From 76361c74176be28e978daffc3758b6db06e3d840 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Mon, 22 Apr 2024 13:11:54 +0100 Subject: [PATCH 01/84] Rework section for clarity, first pass on read-only content Signed-off-by: Dj Walker-Morgan --- .../pgd/5/reference/catalogs-internal.mdx | 103 ++++++++++++++---- .../docs/pgd/5/routing/administering.mdx | 50 +++++++++ .../docs/pgd/5/routing/configuration.mdx | 74 +++++++++++++ product_docs/docs/pgd/5/routing/index.mdx | 84 +++----------- .../docs/pgd/5/routing/installing_proxy.mdx | 6 +- .../docs/pgd/5/routing/monitoring.mdx | 58 ++++++++++ product_docs/docs/pgd/5/routing/proxy.mdx | 83 ++++++-------- product_docs/docs/pgd/5/routing/readonly.mdx | 68 ++++++++++++ 8 files changed, 380 insertions(+), 146 deletions(-) create mode 100644 product_docs/docs/pgd/5/routing/administering.mdx create mode 100644 product_docs/docs/pgd/5/routing/configuration.mdx create mode 100644 product_docs/docs/pgd/5/routing/monitoring.mdx create mode 100644 product_docs/docs/pgd/5/routing/readonly.mdx diff --git a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx index 699a20199cd..88d884b6408 100644 --- a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx +++ b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx @@ -80,26 +80,58 @@ An internal catalog table with per node group configuration options. Per node group routing configuration options. +#### `bdr.node_group_routing_config_summary` columns + +| Name | Type | Description | +|-------------------------|---------|-------------------------------------------------------------------------| +| node_group_name | name | Node group name | +| location | name | Node group location | +| enable_proxy_routing | boolean | Group proxy routing enabled? | +| node_group_type | text | Node group type (one of "global", "data", "shard" or "subscriber-only") | +| route_writer_max_lag | bigint | Maximum write lag accepted | +| route_reader_max_lag | bigint | Maximum read lag accepted | +| route_writer_wait_flush | boolean | Wait for flush | + + ### `bdr.node_group_routing_info` An internal catalog table holding current routing information for a proxy. #### `bdr.node_group_routing_info` columns -| Name | Type | Description | -| -------------------- | -------- | -------------------------------------- | -| node_group_id | oid | The node group id tha this proxy uses | -| write_node_id | oid | Current write node | -| prev_write_node_id | oid | Previous write node | -| read_node_ids | oid[] | List of read nodes IDs | +| Name | Type | Description | +|--------------------|-------|-----------------------------| +| node_group_id | oid | The node group id | +| write_node_id | oid | Current write node | +| prev_write_node_id | oid | Previous write node | +| read_node_ids | oid[] | List of read-only nodes IDs | ### `bdr.node_group_routing_summary` A view of `bdr.node_group_routing_info` catalog which shows the information in more friendly way +#### `bdr.node_group_routing_summary` columns +| Name | Type | Description | +|---------------------|--------|-----------------------------| +| node_group_name | name | The node group name | +| write_lead | name | The current write lead | +| previous_write_lead | name | The previous write lead | +| read_nodes | name[] | The current read-only nodes | + ### `bdr.node_routing_config_summary` -Per node routing configuration options. +A friendly view of the per node routing configuration options. +Shows the node name rather than the oid and shorter field names. + +#### `bdr.node_routing_config_summary` columns +| Name | Type | Description | +|----------------|---------|--------------------------------| +| node_name | name | The node name | +| route_priority | int | Priority assigned to this node | +| route_fence | boolean | Switch to fence this node | +| route_writes | boolean | Switch to allow writes | +| route_reads | boolean | Switch to allow reads | +| route_dsn | text | The interface of this node | ### `bdr.proxy_config` @@ -107,24 +139,53 @@ An internal catalog table holding proxy specific configurations. #### `bdr.proxy_config` columns -| Name | Type | Description | -| ---------------------- | -------- | ------------------------------------------------------------------------ | -| proxy_name | name | The name of the proxy | -| node_group_id | oid | The node group id that this proxy uses | -| listen_port | int | Port that the proxy uses | -| max_client_conn | int | Number of maximum client connections that the proxy accepts | -| max_server_conn | int | Number of maximum connections that the server accepts | -| server_conn_timeout | interval | Timeout for the server connections | -| server_conn_keepalive | interval | Interval between the server connection keep alive | -| fallback_group_timeout | interval | Timeout needed for the fallback | -| fallback_group_ids | oid[] | List of group IDs to be used for the fallback | -| listen_addrs | text[] | Listen address | +| Name | Type | Description | +|-----------------------------|----------|-------------------------------------------------------------------------------| +| proxy_name | name | The name of the proxy | +| node_group_id | oid | The node group id that this proxy uses | +| listen_port | int | Port that the proxy uses for read-write connections (set to -1 disables port) | +| max_client_conn | int | Number of maximum read-write client connections that the proxy accepts | +| max_server_conn | int | Number of maximum read-write connections that the server accepts | +| server_conn_timeout | interval | Timeout for the read-write server connections | +| server_conn_keepalive | interval | Interval between the server connection keep alive | +| fallback_group_timeout | interval | Timeout needed for the fallback | +| fallback_group_ids | oid[] | List of group IDs to be used for the fallback | +| listen_addrs | text[] | Listen address | +| read_listen_port | int | Port that the proxy uses for read-only connections (set to -1 disables port) | +| read_max_client_conn | int | Number of maximum read-only client connections that the proxy accepts | +| read_max_server_conn | int | Number of maximum read-only connections that the server accepts | +| read_server_conn_timeout | interval | Timeout for the server read-only connections | +| read_server_conn_keepalive | interval | Interval between the server read-only connection keep alive | +| read_listen_addrs | text[] | Listen address for read-only connections | +| read_consensus_grace_period | interval | Duration for which proxy continues to route even upon loss of consensus | ### `bdr.proxy_config_summary` -Per proxy instance specific configuration options. - +A friendly view of per proxy instance specific configuration options. + +#### `bdr.proxy_config_summary` columns +| Name | Type | Description | +|---------------------------------|----------|-------------------------------------------------------------------------------| +| proxy_name | name | The name of the proxy | +| node_group_name | name | The node group name that this proxy uses | +| listen_port | int | Port that the proxy uses for read-write connections (set to -1 disables port) | +| max_client_conn | int | Number of maximum read-write client connections that the proxy accepts | +| max_server_conn | int | Number of maximum read-write connections that the server accepts | +| server_conn_timeout | interval | Timeout for the read-write server connections | +| server_conn_keepalive | interval | Interval between the server connection keep alive | +| node_group_enable_proxy_routing | boolean | Does the group the proxy is in enable proxy routing | +| node_group_location | name | The group's location value | +| fallback_group_timeout | interval | Timeout needed for the fallback | +| fallback_group_ids | oid[] | List of group IDs to be used for the fallback | +| listen_addrs | text[] | Listen address | +| read_listen_port | int | Port that the proxy uses for read-only connections (set to -1 disables port) | +| read_max_client_conn | int | Number of maximum read-only client connections that the proxy accepts | +| read_max_server_conn | int | Number of maximum read-only connections that the server accepts | +| read_server_conn_timeout | interval | Timeout for the server read-only connections | +| read_server_conn_keepalive | interval | Interval between the server read-only connection keep alive | +| read_listen_addrs | text[] | Listen address for read-only connections | +| read_consensus_grace_period | interval | Duration for which proxy continues to route even upon loss of consensus | ### `bdr.sequence_kind` diff --git a/product_docs/docs/pgd/5/routing/administering.mdx b/product_docs/docs/pgd/5/routing/administering.mdx new file mode 100644 index 00000000000..3bf09ed6e79 --- /dev/null +++ b/product_docs/docs/pgd/5/routing/administering.mdx @@ -0,0 +1,50 @@ +--- +title: Administering PGD Proxy +navTitle: Administering +--- + +## Switching the write leader + +Switching the write leader is a manual operation that you can perform to change the node that's the write leader. +It can be useful when you want to perform maintenance on the current write leader node or when you want to change the write leader for any other reason. +When changing write leader, there are two modes, `strict` and `fast`. +In `strict` mode, the maximum lag is checked before switching the write leader. This is the default. +In `fast` mode, the write leader is switched without checking the maximum lag. +There is also a timeout parameter that can be set to specify the time to wait for the switchover to complete. + +### Using SQL + +You can perform a switchover operation that explicitly changes the node that's +the write leader to another node. + +Use the [`bdr.routing_leadership_transfer()`](/pgd/latest/reference/routing#bdrrouting_leadership_transfer) function. + +For example, to switch the write leader to node `node1` in group `group1`, use the following SQL command: + +```sql +SELECT bdr.routing_leadership_transfer('group1', 'node1','strict','10s'); +``` +This command will switch the write leader using `strict` mode and wait for up to 10 seconds for the switchover to complete. Those are default settings and can be omitted. + +```sql +SELECT bdr.routing_leadership_transfer('group1', 'node1'); +``` + +### Using PGD CLI + +You can use the [`switchover`](/pgd/latest/cli/command_ref/pgd_switchover/) command to perform a switchover operation. + +For example, to switch the write leader from node `node1` to node `node2` in group `group1`, use the following command: + +```sh +pgd switchover --node-group group1 --node-name node1 --method strict --timeout 10s +``` + +This command will switch the write leader using `strict` mode and wait for up to 10 seconds for the switchover to complete. Those are default settings and can be omitted. + +```sh +pgd switchover --node-group group1 --node-name node1 +``` + + + diff --git a/product_docs/docs/pgd/5/routing/configuration.mdx b/product_docs/docs/pgd/5/routing/configuration.mdx new file mode 100644 index 00000000000..2b666f9038e --- /dev/null +++ b/product_docs/docs/pgd/5/routing/configuration.mdx @@ -0,0 +1,74 @@ +--- +title: "PGD Proxy configuration" +navTitle: "Configuration" +--- + +## Group level configuration + +Configuring the routing is done through either SQL interfaces or through +PGD-CLI. + +You can enable routing decisions by calling the [`bdr.alter_node_group_option()`](/pgd/latest/reference/nodes-management-interfaces#bdralter_node_group_option) function. +For example: + +```text +SELECT bdr.alter_node_group_option('region1-group', 'enable_proxy_routing', 'true') +``` + +You can disable it by setting the same option to `false`. + +Additional group-level options affect the routing decisions: + +- `route_writer_max_lag` — Maximum lag in bytes of the new write candidate to be + selected as write leader. If no candidate passes this, no writer is + selected automatically. +- `route_reader_max_lag` — Maximum lag in bytes for a node to be considered a viable + read-only node. Currently reserved for future use. + +## Node level configuration + +Per-node configuration of routing is set using [`bdr.alter_node_option()`](/pgd/latest/reference/nodes-management-interfaces#bdralter_node_option). The +available options that affect routing are the following: + +- `route_dsn` — The dsn used by proxy to connect to this node. +- `route_priority` — Relative routing priority of the node against other nodes in + the same node group. +- `route_fence` — Whether the node is fenced from routing, that is, it can't receive connections + from PGD Proxy. +- `route_writes` — Whether writes can be routed to this node, that is, whether the node + can become write leader. +- `route_reads` — Whether read-only connections can be routed to this node. Currently + reserved for future use. + +## Proxy level configuration + +You can configure the proxies using SQL interfaces. + +### Creating and dropping proxy configurations + +You can add a proxy configuration using [`bdr.create_proxy`](/pgd/latest/reference/routing#bdrcreate_proxy). +For example, `SELECT bdr.create_proxy('region1-proxy1', 'region1-group');` +creates the default configuration for a proxy named `region1-proxy1` in the PGD group `region1-group`. + +The name of the proxy given here must be same as the name given in the proxy configuration file. + +You can remove a proxy configuration using `SELECT bdr.drop_proxy('region1-proxy1')`. +Dropping a proxy deactivates it. + +### Altering proxy configurations + +You can configure options for each proxy using the [`bdr.alter_proxy_option()`](/pgd/latest/reference/routing#bdralter_proxy_option) function. + +The available options are: + +- `listen_address` — Address for the proxy to listen on. +- `listen_port` — Port for the proxy to listen on. +- `max_client_conn` — Maximum number of connections for the proxy to accept. +- `max_server_conn` — Maximum number of connections the proxy can make to the + Postgres node. +- `server_conn_timeout` — Connection timeout for server connections. +- `server_conn_keepalive` — Keepalive interval for server connections. +- `consensus_grace_period` — Duration for which proxy continues to route even upon loss +of a Raft leader. If set to `0s`, proxy stops routing immediately. + + diff --git a/product_docs/docs/pgd/5/routing/index.mdx b/product_docs/docs/pgd/5/routing/index.mdx index 8601350c100..ecc7f36238f 100644 --- a/product_docs/docs/pgd/5/routing/index.mdx +++ b/product_docs/docs/pgd/5/routing/index.mdx @@ -1,88 +1,30 @@ --- -title: "Application connection management" -navTitle: "Connection management" +title: "PGD Proxy"" +navTitle: "PGD Proxy" indexCards: none description: How to use PGD Proxy to maintain consistent connections to the PGD cluster. navigation: - proxy - installing_proxy + - configuration + - administering + - monitoring - raft --- -Managing application connections is an important part of high availability. +Managing application connections is an important part of high availability. PGD Proxy offers a way to manage connections to the EDB Postgres Distributed cluster. It acts as a proxy layer between the client application and the Postgres database. -Especially with asynchronous replication, having a consistent write lead node is -important to avoid conflicts and guarantee availability for the -application. +* [PGD Proxy Overview](/pgd/latest/routing/proxy) provides an overview of the PGD Proxy, its processes and how it interacts with the EDB Postgres Distributed cluster. -EDB Postgres Distributed provides a proxy layer called PGD Proxy, which is -normally installed in highly available configuration (at least two instances per -region). +* [Installing the PGD Proxy daemon](/pgd/latest/routing/installing_proxy) covers installation of the PGD Proxy daemon on a host. -The PGD Proxy connects to one of the EDB Postgres Distributed nodes and monitors -routing configuration changes as decided by the EDB Postgres Distributed cluster. -It ensures that the connections are routed to the correct nodes consistently. +* [Configuring PGD Proxy](/pgd/latest/routing/configuration) details the three levels (group, node, and proxy) of configuration on a cluster that control how the PGD Proxy daemons behave. -## Configuration +* [Administering PGD Proxy](/pgd/latest/routing/administering) shows how to switch the write leader and manage the PGD Proxy. -Configuring the routing is done through either SQL interfaces or through -PGD-CLI. +* [Monitoring PGD Proxy](/pgd/latest/routing/monitoring) looks at how to monitor PGD Proxy through the cluster and at a daemon level. -You can enable routing decisions by calling the `bdr.alter_node_group_option()` function. -For example: +* [Read-Only Routing](/pgd/latest/routing/readonly) explains how the read-only routing feature in PGD Proxy enables read scalability. -```text -SELECT bdr.alter_node_group_option('region1-group', 'enable_proxy_routing', 'true') -``` +* [RAFT](/pgd/latest/routing/raft) provides an overview of the RAFT consensus mechanism used to coordinate PGD Proxy. -You can disable it by setting the same option to `false`. - -Additional group-level options affect the routing decisions: - -- `route_writer_max_lag` — Maximum lag in bytes of the new write candidate to be - selected as write leader. If no candidate passes this, no writer is - selected automatically. -- `route_reader_max_lag` — Maximum lag in bytes for a node to be considered a viable - read-only node. Currently reserved for future use. - -Per-node configuration of routing is set using `bdr.alter_node_option()`. The -available options that affect routing are the following: - -- `route_dsn` — The dsn used by proxy to connect to this node. -- `route_priority` — Relative routing priority of the node against other nodes in - the same node group. -- `route_fence` — Whether the node is fenced from routing, that is, it can't receive connections - from PGD Proxy. -- `route_writes` — Whether writes can be routed to this node, that is, whether the node - can become write leader. -- `route_reads` — Whether read-only connections can be routed to this node. Currently - reserved for future use. - -You can also configure the proxies using SQL interfaces. You can add proxy configuration -using `bdr.create_proxy`. For example, `SELECT bdr.create_proxy('region1-proxy1', 'region1-group');` -adds the default configuration for a proxy named `region1-proxy1` that's a member -of PGD group `region1-group`. The name of the proxy given here must be same -as the name given in the proxy configuration file. You can remove a proxy configuration -using `SELECT bdr.drop_proxy('region1-proxy1')`. The proxy is -deactivated as a result. - -You can configure options for each proxy using the `bdr.alter_proxy_option()` function. -The available options are: - -- `listen_address` — Address for the proxy to listen on. -- `listen_port` — Port for the proxy to listen on. -- `max_client_conn` — Maximum number of connections for the proxy to accept. -- `max_server_conn` — Maximum number of connections the proxy can make to the - Postgres node. -- `server_conn_timeout` — Connection timeout for server connections. -- `server_conn_keepalive` — Keepalive interval for server connections. -- `consensus_grace_period` — Duration for which proxy continues to route even upon loss -of a Raft leader. If set to `0s`, proxy stops routing immediately. - -The current configuration of every group is visible in the -`bdr.node_group_routing_config_summary` view. Similarly, the -`bdr.node_routing_config_summary` view shows current per-node routing -configuration. `bdr.proxy_config_summary` shows per-proxy configuration. - -You can also do a switchover operation to explicitly change the node that's -the write leader. To do so, use the `bdr.routing_leadership_transfer()` function. diff --git a/product_docs/docs/pgd/5/routing/installing_proxy.mdx b/product_docs/docs/pgd/5/routing/installing_proxy.mdx index 971b6105e29..bb031e8c6a1 100644 --- a/product_docs/docs/pgd/5/routing/installing_proxy.mdx +++ b/product_docs/docs/pgd/5/routing/installing_proxy.mdx @@ -33,9 +33,9 @@ log-level: debug cluster: name: cluster-name endpoints: - - "host=bdr-a1 port=5432 dbname=bdrdb user=pgdproxy " - - "host=bdr-a3 port=5432 dbname=bdrdb user=pgdproxy " - - "host=bdr-a2 port=5432 dbname=bdrdb user=pgdproxy " + - "host=bdr-a1 port=5432 dbname=bdrdb user=pgdproxy" + - "host=bdr-a3 port=5432 dbname=bdrdb user=pgdproxy" + - "host=bdr-a2 port=5432 dbname=bdrdb user=pgdproxy" proxy: name: "proxy-a1" ``` diff --git a/product_docs/docs/pgd/5/routing/monitoring.mdx b/product_docs/docs/pgd/5/routing/monitoring.mdx new file mode 100644 index 00000000000..530583661fa --- /dev/null +++ b/product_docs/docs/pgd/5/routing/monitoring.mdx @@ -0,0 +1,58 @@ +--- +title: Monitoring PGD Proxy +navTitle: Monitoring +--- + +Proxies can be monitored at the cluster and group level or at the process level. + +## Monitoring through the cluster + +### Using SQL + +The current configuration of every group is visible in the [`bdr.node_group_routing_config_summary`](/pgd/latest/reference/catalogs-internal#bdrnode_group_routing_config_summary) view. + +The [`bdr.node_routing_config_summary`](/pgd/latest/reference/catalogs-internal#bdrnode_routing_config_summary) view shows current per-node routing configuration. + +[`bdr.proxy_config_summary`](/pgd/latest/reference/catalogs-internal#bdrproxy_config_summary) shows per-proxy configuration. + +### Using PGD CLI + +You can use the `show-proxies` command to view the current status of all proxies in the PGD cluster. +The `show-proxy` command to view the status of a specific proxy. + +## Monitoring at the process level + +### Proxy health check + +PGD Proxy provides the following HTTP(s) health check API endpoints. The API endpoints respond to `GET` requests. You need to enable and configure the endpoints before using them. See [Configurations](installing_proxy#configuring-health-check). + +``` +GET /health/is-ready +GET /health/is-live +``` + +#### Readiness + +On receiving a valid `GET` request, the proxy checks if it can successfully route connections to the current write leader. If the check returns successfully, the API responds with a body containing `true` and an HTTP status code `200 (OK)`. Otherwise, it returns a body containing `false` with the HTTP status code `500 (Internal Server Error)`. + +#### Liveness + +Liveness checks return either `true` with HTTP status code `200 (OK)` or an error. They never return `false` because the HTTP server listening for the request is stopped if the PGD Proxy service fails to start or exits. + +## Proxy log location + +Proxies also write logs to system logging where they can be monitored with other system services. + +### syslog + +- Debian based - `/var/log/syslog` +- Red Hat based - `/var/log/messages` + +Use the `journalctl` command to filter and view logs for troubleshooting PGD Proxy. The following are few sample commands for quick reference: + +```sh +journalctl -u pgd-proxy -n100 -f +journalctl -u pgd-proxy --since today +journalctl -u pgd-proxy --since "10 min ago" +journalctl -u pgd-proxy --since "2022-10-20 16:21:50" --until "2022-10-20 16:21:55" +``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/routing/proxy.mdx b/product_docs/docs/pgd/5/routing/proxy.mdx index 66373ec1456..59afb47ed42 100644 --- a/product_docs/docs/pgd/5/routing/proxy.mdx +++ b/product_docs/docs/pgd/5/routing/proxy.mdx @@ -1,6 +1,6 @@ --- -title: "EDB Postgres Distributed Proxy" -navTitle: "PGD Proxy" +title: "EDB Postgres Distributed Proxy Overview" +navTitle: "PGD Proxy Overview" indexCards: none navigation: - installing_proxy @@ -9,30 +9,52 @@ directoryDefaults: description: "The PGD Proxy is a service that acts as proxy layer between the client application and Postgres for your EDB Postgres Distributed cluster" --- -EDB Postgres Distributed Proxy is a daemon that acts as an abstraction layer between the client application and Postgres. It interfaces with the PGD consensus mechanism to get the identity of the current write leader node and redirects traffic to that node. -The PGD cluster always has at least one global group and one data group. PGD elects the write leader for each data group that has the `enable_proxy_routing` and `enable_raft` options set to true. You can attach Proxy to a global group or data group. You can attach multiple proxies to each group. +Especially with asynchronous replication, having a consistent write lead node is +important to avoid conflicts and guarantee availability for the application. + +There are two parts to EDB Postgres Distributed's proxy layer: + +* Proxy configuration and routing information which is maintained by the PGD consensus mechanism. +* The PGD Proxy daemon which is installed on a host. It connects to the PGD cluster where it reads its configuration and listens for changes to the routing information. + +This is normally installed in highly available configuration (at least two instances of the daemon per +region). + +Once configured, the PGD Proxy daemon monitors routing configuration changes as decided by the EDB Postgres Distributed cluster. It acts on these changes to ensure that connections are consistently routed to the correct nodes. + +## PGD Proxy cluster configuration + +The PGD cluster always has at least one top-level group and one data group. PGD elects the write leader for each data group that has the `enable_proxy_routing` and `enable_raft` options set to true. + +The cluster also maintains Proxy configurations for each group. Each configuration has a name and is associated with a group. You can attach Proxy to a top-level group or data group. You can attach multiple proxies to each group. + +When a PGD Proxy daemon starts running on a host, it has a name in its local configuration file and it will connect to a node in a group, from where it will use the name to look up its complete configuration as stored on the group. + + +## PGD Proxy daemon + +The PGD Proxy daemon (EDB Postgres Distributed Proxy) is a process that acts as an abstraction layer between the client application and Postgres. It interfaces with the PGD consensus mechanism to get the identity of the current write leader node and redirects traffic to that node. PGD Proxy is a TCP layer 4 proxy. -## How it works +## How they work together Upon starting, PGD Proxy connects to one of the endpoints given in the local config file. It fetches: - DB connection information for all nodes - Proxy options like listen address, listen port -- Routing details like current write leader - -Endpoints given in the config file are used only at startup. After that, actual endpoints are taken from the PGD catalog's `route_dsn` field in `bdr.node_routing_config_summary`. +- Routing details including the current write leader +The endpoints given in the config file are used only at startup. After that, actual endpoints are taken from the PGD catalog's `route_dsn` field in [`bdr.node_routing_config_summary`](/pgd/latest/reference/catalogs-internal#bdrnode_routing_config_summary). PGD manages write leader election. PGD Proxy interacts with PGD to get write leader change events notifications on Postgres notify/listen channels and routes client traffic to the current write leader. PGD Proxy disconnects all existing client connections on write leader change or when write leader is unavailable. Write leader election is a Raft-backed activity and is subject to Raft leader availability. PGD Proxy closes the new client connections if write leader is unavailable. PGD Proxy responds to write leader change events that can be categorized into two modes of operation: *failover* and *switchover*. -Automatic transfer of write leadership from the current write leader node to a new node in the event of Postgres or operating system crash is called failover. PGD elects a new write leader when the current write leader goes down or becomes unresponsive. Once the new write leader is elected by PGD, PGD Proxy closes existing client connections to the old write leader and redirects new client connections to the newly elected write leader. +Automatic transfer of write leadership from the current write leader node to a new node in the event of Postgres or operating system crash is called *failover*. PGD elects a new write leader when the current write leader goes down or becomes unresponsive. Once the new write leader is elected by PGD, PGD Proxy closes existing client connections to the old write leader and redirects new client connections to the newly elected write leader. -User-controlled, manual transfer of write leadership from the current write leader to a new target leader is called switchover. Switchover is triggered through the [PGD CLI switchover](../cli/command_ref/pgd_switchover) command. The command is submitted to PGD, which attempts to elect the given target node as the new write leader. Similar to failover, PGD Proxy closes existing client connections and redirects new client connections to the newly elected write leader. This is useful during server maintenance, for example, if the current write leader node needs to be stopped for maintenance like a server update or OS patch update. +User-controlled, manual transfer of write leadership from the current write leader to a new target leader is called *switchover*. Switchover is triggered through the [PGD CLI switchover](../cli/command_ref/pgd_switchover) command. The command is submitted to PGD, which attempts to elect the given target node as the new write leader. Similar to failover, PGD Proxy closes existing client connections and redirects new client connections to the newly elected write leader. This is useful during server maintenance, for example, if the current write leader node needs to be stopped for maintenance like a server update or OS patch update. ### Consensus grace period @@ -40,7 +62,6 @@ PGD Proxy provides the `consensus_grace_period` proxy option that can be used to The main purpose of this option is to allow users to configure the write behavior when the Raft leader is lost. When the Raft leader isn't present in the cluster, it's not always guaranteed that the current write leader seen by the proxy is the correct one. In some cases, like network partition in the following example, it is possible that the two write leaders may be seen by two different proxies attached to the same group increasing the chances of write conflicts. If this isn't the desired behavior, then the previously mentioned `consensus_grace_period` can be set to 0s. This setting configures the proxy to stop routing and closes existing open connections immediately when it detects the Raft leader is lost. - #### Network partition example Consider a 3-data node group with a proxy on each data node. In this case, if the current write leader gets network partitioned or isolated, then the data nodes present in the majority partition elects a new write leader. If `consensus_grace_period` is set to a non-zero value, say `10s`, then the proxy present on the previous write leader continues to route writes for this duration. @@ -55,43 +76,3 @@ The PostgreSQL C client library (libpq) allows you to specify multiple host name However, if the PGD Proxy instance is accessible but doesn't have access to the write leader, or the write leader for a given instance doesn't exist (that is, because there's no write leader for the given PGD group), the connection simply fails. No other hosts in the multi-host connection string is tried. This behavior is consistent with the behavior of PostgreSQL client libraries with other proxies like HAProxy or pgbouncer. -## Managing PGD Proxy - -PGD CLI provides a few commands to manage proxies in a PGD cluster, such as `create-proxy`, `delete-proxy`, `set-proxy-options`, and `show-proxies`. See [PGD CLI](../cli/command_ref) for more information. - -See [Connection management](../routing) for more information on the PGD side of configuration and management of PGD Proxy. - -### Proxy health check - -PGD Proxy provides the following HTTP(s) health check API endpoints. The API endpoints respond to `GET` requests. You need to enable and configure them before using them. See [Configurations](installing_proxy#configuring-health-check). - - -``` -GET /health/is-ready -GET /health/is-live -``` - -#### Readiness - -On receiving a valid `GET` request, the proxy checks if it can successfully route connections to the current write leader. If the check returns successfully, the API responds with a body containing `true` and an HTTP status code `200 (OK)`. Otherwise, it returns a body containing `false` with the HTTP status code `500 (Internal Server Error)`. - - -#### Liveness - -Liveness checks return either `true` with HTTP status code `200 (OK)` or an error. They never return `false` because the HTTP server listening for the request is stopped if the PGD Proxy service fails to start or exits. - -## Proxy log location - -### syslog - -- Debian based - `/var/log/syslog` -- Red Hat based - `/var/log/messages` - -Use the `journalctl` command to filter and view logs for troubleshooting Proxy. The following are few sample commands for quick reference: - -```sh -journalctl -u pgd-proxy -n100 -f -journalctl -u pgd-proxy --since today -journalctl -u pgd-proxy --since "10 min ago" -journalctl -u pgd-proxy --since "2022-10-20 16:21:50" --until "2022-10-20 16:21:55" -``` diff --git a/product_docs/docs/pgd/5/routing/readonly.mdx b/product_docs/docs/pgd/5/routing/readonly.mdx new file mode 100644 index 00000000000..58c098a66bf --- /dev/null +++ b/product_docs/docs/pgd/5/routing/readonly.mdx @@ -0,0 +1,68 @@ +--- +title: Read-Only Routing with PGD Proxy +navTitle: Read-Only Routing +--- + +## Background + +By default, PGD Proxy routes connections to the currently selected write leader in the cluster. This allows the write traffic conflicts to be rapidly and consistently resolved. Just routing everything to a single node, the write leader, is a natural fit for traditional high availability deployments where system throughput is typically limited to the throughput of what a single node can handle. + +But for some use cases this behavior also means that clients which are only querying the data are also placing a load on the current write leader which could equally well be served by one of the non-write leader nodes in the cluster. + +If you could move traffic that was read-only queries to the non-write leader nodes, you could, at least in theory, handle a multiple of single node throughput. An approach like this would typically require changes to applications knowing details of cluster topology and current node status to detect write lead. + +## Read-Only Routing in PGD Proxy + +From PGD 5.4.1, PGD Proxy addresses this requirement, to utilize read capacity while minimizing application exposure to the cluster status. It does this by offering a new read_listen_port on proxies which complement the existing listen port. Proxies can be configured with either or both of these ports. + +When a proxy is configured with a read_listen_port, connections to that particular port are routed to available data nodes that are not the current write leader. If an application only queries and reads from the database, using a read_listen_port ensures that your queries are not answered by the write leader. + +Because PGD Proxy is a TCP Layer 4 proxy, it does not interfere with traffic passing through it. That means, in turn, that it cannot detect attempts to write passing through the read_listen_port connections. It is entirely possible to write through a read-only port. + +The active-active nature of PGD means that any write operation will be performed and replicated and conflict resolution may, or may not have to take place. It is entirely down to the application to avoid this and make sure that it only uses read_listen_ports for read-only traffic. + +### Valid read-only nodes + +Only data nodes which are not the write leader are valid as read-only nodes. For reference, the following node types are not eligible to be a read-only node: + +* Witness nodes cannot be eligible because they do not contain data. +* Logical standbys cannot be eligible because they are standbys and prioritize replicating. +* Subscriber-only nodes are also not currently eligible. + +## Configuring Read-Only Routing + +### Using SQL + +To configure a read-only proxy port on a proxy, use the `bdr.alter_proxy_options` function. + +```sql +SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','7432'); +``` + +This will configure a read-only proxy port on proxy-a1 on port 7432. + +To remove the read-only proxy set the port to -1. + +```sql +SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','-1'); +``` + +### Using PGD CLI + +To configure a read-only proxy port on a proxy, use the `pgd alter-proxy` command. + +```sh +pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=7432 +``` + +This will configure a read-only proxy port on proxy-a1 on port 7432. + +To remove the read-only proxy set the port to -1. + +```sh +pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=-1 +``` + + + + From 91cd51b5c0166d741d16d61b0d1782d33e581d17 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Mon, 22 Apr 2024 14:03:20 +0100 Subject: [PATCH 02/84] Fix double quote issue Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/routing/index.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/routing/index.mdx b/product_docs/docs/pgd/5/routing/index.mdx index ecc7f36238f..550777a07ac 100644 --- a/product_docs/docs/pgd/5/routing/index.mdx +++ b/product_docs/docs/pgd/5/routing/index.mdx @@ -1,5 +1,5 @@ --- -title: "PGD Proxy"" +title: "PGD Proxy" navTitle: "PGD Proxy" indexCards: none description: How to use PGD Proxy to maintain consistent connections to the PGD cluster. @@ -9,6 +9,7 @@ navigation: - configuration - administering - monitoring + - readonly - raft --- From a56a3387f28880f51ca984851b98558f39e76e33 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Mon, 22 Apr 2024 15:51:10 +0100 Subject: [PATCH 03/84] Small changes for nav and index Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/index.mdx | 2 +- product_docs/docs/pgd/5/routing/index.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5/index.mdx b/product_docs/docs/pgd/5/index.mdx index a10646ea420..b1194e3ad25 100644 --- a/product_docs/docs/pgd/5/index.mdx +++ b/product_docs/docs/pgd/5/index.mdx @@ -31,9 +31,9 @@ navigation: - parallelapply - repsets - routing + - cli - backup - monitoring - - cli - transaction-streaming - testingandtuning - striggers diff --git a/product_docs/docs/pgd/5/routing/index.mdx b/product_docs/docs/pgd/5/routing/index.mdx index 550777a07ac..77bc0557e6f 100644 --- a/product_docs/docs/pgd/5/routing/index.mdx +++ b/product_docs/docs/pgd/5/routing/index.mdx @@ -15,7 +15,7 @@ navigation: Managing application connections is an important part of high availability. PGD Proxy offers a way to manage connections to the EDB Postgres Distributed cluster. It acts as a proxy layer between the client application and the Postgres database. -* [PGD Proxy Overview](/pgd/latest/routing/proxy) provides an overview of the PGD Proxy, its processes and how it interacts with the EDB Postgres Distributed cluster. +* [PGD Proxy overview](/pgd/latest/routing/proxy) provides an overview of the PGD Proxy, its processes and how it interacts with the EDB Postgres Distributed cluster. * [Installing the PGD Proxy daemon](/pgd/latest/routing/installing_proxy) covers installation of the PGD Proxy daemon on a host. From 30800a309bec99c52948317af77e4afa4987536a Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Mon, 22 Apr 2024 17:45:34 +0100 Subject: [PATCH 04/84] Fix Raft from RAFT Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/routing/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/routing/index.mdx b/product_docs/docs/pgd/5/routing/index.mdx index 77bc0557e6f..3fb82efd242 100644 --- a/product_docs/docs/pgd/5/routing/index.mdx +++ b/product_docs/docs/pgd/5/routing/index.mdx @@ -27,5 +27,5 @@ Managing application connections is an important part of high availability. PGD * [Read-Only Routing](/pgd/latest/routing/readonly) explains how the read-only routing feature in PGD Proxy enables read scalability. -* [RAFT](/pgd/latest/routing/raft) provides an overview of the RAFT consensus mechanism used to coordinate PGD Proxy. +* [Raft](/pgd/latest/routing/raft) provides an overview of the Raft consensus mechanism used to coordinate PGD Proxy. From 344e7ca479ab19107318d5b6e0ce4e5d6f4322ab Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 24 Apr 2024 18:10:35 +0100 Subject: [PATCH 05/84] Routing function reference updated Signed-off-by: Dj Walker-Morgan --- .../pgd/5/reference/catalogs-internal.mdx | 38 +++++------ product_docs/docs/pgd/5/reference/routing.mdx | 66 +++++++++++++------ product_docs/docs/pgd/5/routing/readonly.mdx | 39 +++++++++-- 3 files changed, 100 insertions(+), 43 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx index 88d884b6408..62e0293ba5a 100644 --- a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx +++ b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx @@ -139,25 +139,25 @@ An internal catalog table holding proxy specific configurations. #### `bdr.proxy_config` columns -| Name | Type | Description | -|-----------------------------|----------|-------------------------------------------------------------------------------| -| proxy_name | name | The name of the proxy | -| node_group_id | oid | The node group id that this proxy uses | -| listen_port | int | Port that the proxy uses for read-write connections (set to -1 disables port) | -| max_client_conn | int | Number of maximum read-write client connections that the proxy accepts | -| max_server_conn | int | Number of maximum read-write connections that the server accepts | -| server_conn_timeout | interval | Timeout for the read-write server connections | -| server_conn_keepalive | interval | Interval between the server connection keep alive | -| fallback_group_timeout | interval | Timeout needed for the fallback | -| fallback_group_ids | oid[] | List of group IDs to be used for the fallback | -| listen_addrs | text[] | Listen address | -| read_listen_port | int | Port that the proxy uses for read-only connections (set to -1 disables port) | -| read_max_client_conn | int | Number of maximum read-only client connections that the proxy accepts | -| read_max_server_conn | int | Number of maximum read-only connections that the server accepts | -| read_server_conn_timeout | interval | Timeout for the server read-only connections | -| read_server_conn_keepalive | interval | Interval between the server read-only connection keep alive | -| read_listen_addrs | text[] | Listen address for read-only connections | -| read_consensus_grace_period | interval | Duration for which proxy continues to route even upon loss of consensus | +| Name | Type | Description | +|-----------------------------|----------|------------------------------------------------------------------------------| +| proxy_name | name | The name of the proxy | +| node_group_id | oid | The node group id that this proxy uses | +| listen_port | int | Port that the proxy uses for read-write connections (set to 0 disables port) | +| max_client_conn | int | Number of maximum read-write client connections that the proxy accepts | +| max_server_conn | int | Number of maximum read-write connections that the server accepts | +| server_conn_timeout | interval | Timeout for the read-write server connections | +| server_conn_keepalive | interval | Interval between the server connection keep alive | +| fallback_group_timeout | interval | Timeout needed for the fallback | +| fallback_group_ids | oid[] | List of group IDs to be used for the fallback | +| listen_addrs | text[] | Listen address | +| read_listen_port | int | Port that the proxy uses for read-only connections (set to 0 disables port) | +| read_max_client_conn | int | Number of maximum read-only client connections that the proxy accepts | +| read_max_server_conn | int | Number of maximum read-only connections that the server accepts | +| read_server_conn_timeout | interval | Timeout for the server read-only connections | +| read_server_conn_keepalive | interval | Interval between the server read-only connection keep alive | +| read_listen_addrs | text[] | Listen address for read-only connections | +| read_consensus_grace_period | interval | Duration for which proxy continues to route even upon loss of consensus | ### `bdr.proxy_config_summary` diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index a91627bed1a..5156b0d12ac 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -14,13 +14,20 @@ Create a proxy #### Synopsis ```sql -bdr.create_proxy(proxy_name text, node_group text); +bdr.create_proxy(proxy_name text, node_group text, proxy_mode text); ``` #### Parameters -- `proxy_name` — Name of the new proxy. -- `node_group` — Name of the group to be used by the proxy. + +| Name | Type | Default | Description | +|--------------|------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `proxy_name` | text | | Name of the new proxy. | +| `node_group` | text | | Name of the group to be used by the proxy. | +| `proxy_mode` | text | 'default' | Mode of the proxy. It can be 'default' (listen_port connections follow write leader, no read_listen_port), 'read-only' (no listen_port, read_listen_port connections follow read-only nodes) or 'any' (listen_port connections follow write_leader, read_listen_port connections follow read-only nodes). Default is 'default'. | + + + ### `bdr.alter_proxy_option` @@ -34,18 +41,33 @@ bdr.alter_proxy_option(proxy_name text, config_key text, config_value text); #### Parameters -- `proxy_name` — Name of the proxy to be changed. -- `config_key` — Key of the option in the proxy to be changed. -- `config_value` — New value to be set for the given key. - -The proxy options that can be changed using this function are: -- `listen_address` — Address for the proxy to listen on. Default is '{0.0.0.0}'. -- `listen_port` — Port for the proxy to listen on. Default is '6432'. -- `max_client_conn` — Maximum number of connections for the proxy to accept. Default is '32767'. -- `max_server_conn` — Maximum number of connections the proxy can make to the Postgres node. Default is '32767'. -- `server_conn_timeout` — Connection timeout for server connections. Default is '2' (seconds). -- `server_conn_keepalive` — Keepalive interval for server connections. Default is '10' (seconds). -- `consensus_grace_period` — Duration for which proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overriden if `raft_response_timeout`, `raft_global_election_timeout` or `raft_group_election_timeout` are changed from their defaults. +| Name | Type | Default | Description | +|----------------|------|---------|-----------------------------------------------| +| `proxy_name` | text | | Name of the proxy to be changed. | +| `config_key` | text | | Key of the option in the proxy to be changed. | +| `config_value` | text | | New value to be set for the given key. | + + +The proxy options (`config_key`) that can be changed using this function are: + +| Option | Description | +|-------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `listen_address` | Address for the proxy to listen on. Default is '{0.0.0.0}' | +| `listen_port` | Port for the proxy to listen on. Default is '6432' in 'default' or 'any' mode and '0' in 'read-only' mode which disables the write leader following port. | +| `max_client_conn` | Maximum number of connections for the proxy to accept. Default is '32767'. | +| `max_server_conn` | Maximum number of connections the proxy can make to the Postgres node. Default is '32767'. | +| `server_conn_timeout` | Connection timeout for server connections. Default is '2' (seconds). | +| `server_conn_keepalive` | Keepalive interval for server connections. Default is '10' (seconds). | +| `consensus_grace_period` | Duration for which proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overriden if `raft_response_timeout`, `raft_global_election_timeout` or `raft_group_election_timeout` are changed from their defaults. | +| `read_listen_address` | Address for the read-only proxy to listen on. Default is '{0.0.0.0}'. | +| `read_listen_port` | Port for the read-only proxy to listen on. Default is '7432' in 'read-only' or 'any' mode and '0' in 'default' mode which disables the read-only port . | +| `read_max_client_conn` | Maximum number of connections for the read-only proxy to accept. Default is '32767'. | +| `read_max_server_conn` | Maximum number of connections the read-only proxy can make to the Postgres node. Default is '32767'. | +| `read_server_conn_keepalive` | Keepalive interval for read-only server connections. Default is '10' (seconds). | +| `read_server_conn_timeout` | Connection timeout for read-only server connections. Default is '2' (seconds). | +| `read_consensus_grace_period` | Duration for which read-only proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overriden if `raft_response_timeout`, `raft_global_election_timeout` or `raft_group_election_timeout` are changed from their defaults. | + +Changing any of theses values requires a restart of the proxy. ### `bdr.drop_proxy` @@ -59,7 +81,9 @@ bdr.drop_proxy(proxy_name text); #### Parameters -- `proxy_name` — Name of the proxy to be dropped. +| Name | Type | Default | Description | +|--------------|------|---------|-----------------------------------------------| +| `proxy_name` | text | | Name of the proxy to be dropped. | ### `bdr.routing_leadership_transfer` @@ -76,7 +100,9 @@ bdr.routing_leadership_transfer(node_group_name text, #### Parameters -- `node_group_name` — Name of group where the leadership transfer is requested. -- `leader_name` — Name of node that will become write leader. -- `transfer_method` — Type of the transfer, it can be "fast" or the default "strict" that checks the maximum lag. -- `transfer_timeout` — Timeout of the leadership transfer, default is 10 seconds. +| Name | Type | Default | Description | +|--------------------|----------|----------|---------------------------------------------------------------------------------------------| +| `node_group_name` | text | | Name of group where the leadership transfer is requested. | +| `leader_name` | text | | Name of node that will become write leader. | +| `transfer_method` | text | 'strict' | Type of the transfer, it can be "fast" or the default "strict" that checks the maximum lag. | +| `transfer_timeout` | interval | '10s' | Timeout of the leadership transfer, default is 10 seconds. | diff --git a/product_docs/docs/pgd/5/routing/readonly.mdx b/product_docs/docs/pgd/5/routing/readonly.mdx index 58c098a66bf..7fc069ce657 100644 --- a/product_docs/docs/pgd/5/routing/readonly.mdx +++ b/product_docs/docs/pgd/5/routing/readonly.mdx @@ -29,8 +29,39 @@ Only data nodes which are not the write leader are valid as read-only nodes. For * Logical standbys cannot be eligible because they are standbys and prioritize replicating. * Subscriber-only nodes are also not currently eligible. +## Creating a Proxy + +Proxy creation functions in PGD take an optional `proxy-mode` parameter. This parameter can be set to one of the following values: +* `default`: This is the default value. It creates a proxy that can handles traffic that follows the write leader on port 6432. +* `read-only`: This creates a read-only proxy which routes traffic to nodes that are not the write leader. It only handles this read-only traffic on port 7432. +* `any`: This creates create a proxy that can handle both read-only and write leader following traffic on seperate ports; 6432 for write leader following traffic and 7432 for read-only traffic. + +## Creating a Read-Only Proxy + +### Using SQL + +To create a new read-only proxy, use the `bdr.create_proxy` function. + +```sql +SELECT bdr.create_proxy('proxy-ro1','group-a','read-only'); +``` + +This will create a new read-only proxy named `proxy-ro1` in group `group-a` and, by default, will listen on port 7432 for read-only traffic. + +### Using PGD CLI + +To create a new read-only proxy, use the `pgd create-proxy` command with the optional `--proxy_mode` flag set to `read-only`. + +```sh +pgd create-proxy --proxy-name proxy-ro1 --node-group group-a --proxy-mode read-only +``` + ## Configuring Read-Only Routing +!!! Note +To change a proxy's configuration, the proxy must be restarted after changes have been made. +!!! + ### Using SQL To configure a read-only proxy port on a proxy, use the `bdr.alter_proxy_options` function. @@ -41,10 +72,10 @@ SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','7432'); This will configure a read-only proxy port on proxy-a1 on port 7432. -To remove the read-only proxy set the port to -1. +To remove the read-only proxy set the port to 0. ```sql -SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','-1'); +SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','0'); ``` ### Using PGD CLI @@ -57,10 +88,10 @@ pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=7432 This will configure a read-only proxy port on proxy-a1 on port 7432. -To remove the read-only proxy set the port to -1. +To remove the read-only proxy set the port to 0. ```sh -pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=-1 +pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=0 ``` From eb104935e26d7e6117dbde68dbf95dfd1c8b8367 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 25 Apr 2024 11:30:47 +0100 Subject: [PATCH 06/84] Tidy up Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/reference/routing.mdx | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index 5156b0d12ac..e3561c42781 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -5,8 +5,6 @@ indexdepth: 3 rootisheading: false --- - - ### `bdr.create_proxy` Create a proxy @@ -19,7 +17,6 @@ bdr.create_proxy(proxy_name text, node_group text, proxy_mode text); #### Parameters - | Name | Type | Default | Description | |--------------|------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `proxy_name` | text | | Name of the new proxy. | @@ -106,3 +103,4 @@ bdr.routing_leadership_transfer(node_group_name text, | `leader_name` | text | | Name of node that will become write leader. | | `transfer_method` | text | 'strict' | Type of the transfer, it can be "fast" or the default "strict" that checks the maximum lag. | | `transfer_timeout` | interval | '10s' | Timeout of the leadership transfer, default is 10 seconds. | + From 64caa3f083447d975270268ae2709499919fd078 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> Date: Tue, 30 Apr 2024 09:39:38 +0100 Subject: [PATCH 07/84] Update product_docs/docs/pgd/5/reference/routing.mdx --- product_docs/docs/pgd/5/reference/routing.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index e3561c42781..a137130bc48 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -57,7 +57,7 @@ The proxy options (`config_key`) that can be changed using this function are: | `server_conn_keepalive` | Keepalive interval for server connections. Default is '10' (seconds). | | `consensus_grace_period` | Duration for which proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overriden if `raft_response_timeout`, `raft_global_election_timeout` or `raft_group_election_timeout` are changed from their defaults. | | `read_listen_address` | Address for the read-only proxy to listen on. Default is '{0.0.0.0}'. | -| `read_listen_port` | Port for the read-only proxy to listen on. Default is '7432' in 'read-only' or 'any' mode and '0' in 'default' mode which disables the read-only port . | +| `read_listen_port` | Port for the read-only proxy to listen on. Default is '6433' in 'read-only' or 'any' mode and '0' in 'default' mode which disables the read-only port . | | `read_max_client_conn` | Maximum number of connections for the read-only proxy to accept. Default is '32767'. | | `read_max_server_conn` | Maximum number of connections the read-only proxy can make to the Postgres node. Default is '32767'. | | `read_server_conn_keepalive` | Keepalive interval for read-only server connections. Default is '10' (seconds). | From f5dabca358b446981ca0f1b5d48916366489596d Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> Date: Tue, 30 Apr 2024 09:39:55 +0100 Subject: [PATCH 08/84] Update product_docs/docs/pgd/5/reference/routing.mdx --- product_docs/docs/pgd/5/reference/routing.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index a137130bc48..8301979fb96 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -62,7 +62,7 @@ The proxy options (`config_key`) that can be changed using this function are: | `read_max_server_conn` | Maximum number of connections the read-only proxy can make to the Postgres node. Default is '32767'. | | `read_server_conn_keepalive` | Keepalive interval for read-only server connections. Default is '10' (seconds). | | `read_server_conn_timeout` | Connection timeout for read-only server connections. Default is '2' (seconds). | -| `read_consensus_grace_period` | Duration for which read-only proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overriden if `raft_response_timeout`, `raft_global_election_timeout` or `raft_group_election_timeout` are changed from their defaults. | +| `read_consensus_grace_period` | Duration for which read-only proxy continues to route even upon loss of a Raft leader.Default is 1 hour. | Changing any of theses values requires a restart of the proxy. From ea2b91a11c097be62bc29c0f8738fb1418ea53e6 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> Date: Tue, 30 Apr 2024 09:40:08 +0100 Subject: [PATCH 09/84] Update product_docs/docs/pgd/5/reference/routing.mdx --- product_docs/docs/pgd/5/reference/routing.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index 8301979fb96..c0ef5e1b374 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -23,7 +23,7 @@ bdr.create_proxy(proxy_name text, node_group text, proxy_mode text); | `node_group` | text | | Name of the group to be used by the proxy. | | `proxy_mode` | text | 'default' | Mode of the proxy. It can be 'default' (listen_port connections follow write leader, no read_listen_port), 'read-only' (no listen_port, read_listen_port connections follow read-only nodes) or 'any' (listen_port connections follow write_leader, read_listen_port connections follow read-only nodes). Default is 'default'. | - +When proxy_mode is set to `default` all read options in the proxy config will be set to NULL. When it is set to `read-only` all write options in the proxy config will be set to NULL. When set to `any` all options will be set to their defaults. ### `bdr.alter_proxy_option` From 5989932ad90dab063f1e5107727b7e65d6c30827 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 1 May 2024 15:21:17 +0100 Subject: [PATCH 10/84] Review fixes and clarifications Signed-off-by: Dj Walker-Morgan --- .../docs/pgd/5/routing/administering.mdx | 8 +++++-- .../docs/pgd/5/routing/configuration.mdx | 21 ++++++++++++------- 2 files changed, 19 insertions(+), 10 deletions(-) diff --git a/product_docs/docs/pgd/5/routing/administering.mdx b/product_docs/docs/pgd/5/routing/administering.mdx index 3bf09ed6e79..8a173438b94 100644 --- a/product_docs/docs/pgd/5/routing/administering.mdx +++ b/product_docs/docs/pgd/5/routing/administering.mdx @@ -8,10 +8,14 @@ navTitle: Administering Switching the write leader is a manual operation that you can perform to change the node that's the write leader. It can be useful when you want to perform maintenance on the current write leader node or when you want to change the write leader for any other reason. When changing write leader, there are two modes, `strict` and `fast`. -In `strict` mode, the maximum lag is checked before switching the write leader. This is the default. -In `fast` mode, the write leader is switched without checking the maximum lag. +In `strict` mode, the lag is checked before switching the write leader. It will wait until the lag is less than `route_writer_max_lag` before starting the switchover. This is the default. +In `fast` mode, the write leader is switched immediatly. There is also a timeout parameter that can be set to specify the time to wait for the switchover to complete. +!!! Note + The switchover operation is not a guaranteed operation. If due to a timeout or for other reasons, the switchover to the given target node fails, PGD may elect another node as write leader in its place. This other node can include the current write leader node. PGD will always try to elect a new write leader if the switchover operation fails. +!!! + ### Using SQL You can perform a switchover operation that explicitly changes the node that's diff --git a/product_docs/docs/pgd/5/routing/configuration.mdx b/product_docs/docs/pgd/5/routing/configuration.mdx index 2b666f9038e..713fce08053 100644 --- a/product_docs/docs/pgd/5/routing/configuration.mdx +++ b/product_docs/docs/pgd/5/routing/configuration.mdx @@ -23,7 +23,7 @@ Additional group-level options affect the routing decisions: selected as write leader. If no candidate passes this, no writer is selected automatically. - `route_reader_max_lag` — Maximum lag in bytes for a node to be considered a viable - read-only node. Currently reserved for future use. + read-only node. PGD 5.5.0 and later. ## Node level configuration @@ -32,13 +32,12 @@ available options that affect routing are the following: - `route_dsn` — The dsn used by proxy to connect to this node. - `route_priority` — Relative routing priority of the node against other nodes in - the same node group. -- `route_fence` — Whether the node is fenced from routing, that is, it can't receive connections - from PGD Proxy. + the same node group. Only used when electing a Write Leader. +- `route_fence` — Determines Whether the node is fenced from routing. When fenced the node can't receive connections + from PGD Proxy. It therefore cannot become the Write Leader or be available in read-only node pool. - `route_writes` — Whether writes can be routed to this node, that is, whether the node can become write leader. -- `route_reads` — Whether read-only connections can be routed to this node. Currently - reserved for future use. +- `route_reads` — Whether read-only connections can be routed to this node. PGD 5.5.0 and later. ## Proxy level configuration @@ -70,5 +69,11 @@ The available options are: - `server_conn_keepalive` — Keepalive interval for server connections. - `consensus_grace_period` — Duration for which proxy continues to route even upon loss of a Raft leader. If set to `0s`, proxy stops routing immediately. - - +- `read_listen_address` — Address for the read-only proxy to listen on. +- `read_listen_port` — Port for the read-only proxy to listen on. +- `read_max_client_conn` — Maximum number of connections for the read-only proxy to accept. +- `read_max_server_conn` — Maximum number of connections the read-only proxy can make to the + Postgres node. +- `read_server_conn_keepalive` — Keepalive interval for read-only server connections. +- `read_server_conn_timeout` — Connection timeout for read-only server connections. +- `read_consensus_grace_period` — Duration for which read-only proxy continues to route even upon loss of a Raft leader. From 58ffb9b0a19a9c421a6c7088b26515b898f3210a Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 1 May 2024 15:56:24 +0100 Subject: [PATCH 11/84] More review changes Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/routing/readonly.mdx | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/product_docs/docs/pgd/5/routing/readonly.mdx b/product_docs/docs/pgd/5/routing/readonly.mdx index 7fc069ce657..d53cd09c0df 100644 --- a/product_docs/docs/pgd/5/routing/readonly.mdx +++ b/product_docs/docs/pgd/5/routing/readonly.mdx @@ -13,14 +13,16 @@ If you could move traffic that was read-only queries to the non-write leader nod ## Read-Only Routing in PGD Proxy -From PGD 5.4.1, PGD Proxy addresses this requirement, to utilize read capacity while minimizing application exposure to the cluster status. It does this by offering a new read_listen_port on proxies which complement the existing listen port. Proxies can be configured with either or both of these ports. +From PGD 5.5.0, PGD Proxy addresses this requirement, to utilize read capacity while minimizing application exposure to the cluster status. It does this by offering a new read_listen_port on proxies which complement the existing listen port. Proxies can be configured with either or both of these ports. When a proxy is configured with a read_listen_port, connections to that particular port are routed to available data nodes that are not the current write leader. If an application only queries and reads from the database, using a read_listen_port ensures that your queries are not answered by the write leader. -Because PGD Proxy is a TCP Layer 4 proxy, it does not interfere with traffic passing through it. That means, in turn, that it cannot detect attempts to write passing through the read_listen_port connections. It is entirely possible to write through a read-only port. +Because PGD Proxy is a TCP Layer 4 proxy, it does not interfere with traffic passing through it. That means that it cannot detect attempts to write passing through the read_listen_port connections. As it cannot distinguish between a SELECT or an INSERT, it is entirely possible to write through a read-only port. The active-active nature of PGD means that any write operation will be performed and replicated and conflict resolution may, or may not have to take place. It is entirely down to the application to avoid this and make sure that it only uses read_listen_ports for read-only traffic. +Where available, the problem can be mitigated on the client side by passing [`default_transaction_read_only=on`](https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-DEFAULT-TRANSACTION-READ-ONLY) in the connection string or equivalent for the driver in use. + ### Valid read-only nodes Only data nodes which are not the write leader are valid as read-only nodes. For reference, the following node types are not eligible to be a read-only node: @@ -33,8 +35,8 @@ Only data nodes which are not the write leader are valid as read-only nodes. For Proxy creation functions in PGD take an optional `proxy-mode` parameter. This parameter can be set to one of the following values: * `default`: This is the default value. It creates a proxy that can handles traffic that follows the write leader on port 6432. -* `read-only`: This creates a read-only proxy which routes traffic to nodes that are not the write leader. It only handles this read-only traffic on port 7432. -* `any`: This creates create a proxy that can handle both read-only and write leader following traffic on seperate ports; 6432 for write leader following traffic and 7432 for read-only traffic. +* `read-only`: This creates a read-only proxy which routes traffic to nodes that are not the write leader. It only handles this read-only traffic on port 6433. +* `any`: This creates create a proxy that can handle both read-only and write leader following traffic on seperate ports; 6432 for write leader following traffic and 6433 for read-only traffic. ## Creating a Read-Only Proxy @@ -67,10 +69,10 @@ To change a proxy's configuration, the proxy must be restarted after changes hav To configure a read-only proxy port on a proxy, use the `bdr.alter_proxy_options` function. ```sql -SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','7432'); +SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','6433'); ``` -This will configure a read-only proxy port on proxy-a1 on port 7432. +This will configure a read-only proxy port on proxy-a1 on port 6433. To remove the read-only proxy set the port to 0. @@ -83,10 +85,10 @@ SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','0'); To configure a read-only proxy port on a proxy, use the `pgd alter-proxy` command. ```sh -pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=7432 +pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=6433 ``` -This will configure a read-only proxy port on proxy-a1 on port 7432. +This will configure a read-only proxy port on proxy-a1 on port 6433. To remove the read-only proxy set the port to 0. From dfff8086ae4ddfbdb4673dc5e3109352451ca29c Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 25 Apr 2024 14:51:34 +0100 Subject: [PATCH 12/84] Added group applicability details for alter_node_group_option Signed-off-by: Dj Walker-Morgan --- .../reference/nodes-management-interfaces.mdx | 69 +++++-------------- 1 file changed, 16 insertions(+), 53 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx index 613634dcd9f..8fc410511da 100644 --- a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx +++ b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx @@ -112,62 +112,25 @@ bdr.alter_node_group_option(node_group_name text, - `config_value` — New value to be set for the given key. `config_value` will be parsed into the data type appropriate for the option. -Note that some parameters can only be applied to the top-level node group. +Note that some parameters can only be applied to the top-level node group. These are denoted in the table below with 'Top' appearing in the Groups coluumn. Where the parameter can be applied to the top-level node group and sub-groups, 'All' appears in the Groups column. 'Sub' in the Groups column means the parameter can only be applied to subgroups. The group options which can be changed using this function are: -- `apply_delay` (`interval`) — How long nodes wait to apply incoming - changes. This is useful mainly to setup a special sub-group with delayed - subscriber-only nodes. Don't set this on groups which contain data nodes or - on the top-level group. Default is `0s`. -- `check_constraints`(`boolean`) — Whether the apply process checks the - constraints when writing replicated data. It's recommended to keep this to - default value, otherwise you risk data loss. Valid values are either `on` or - `off`. Default is `on`. - This option can only be changed for the top-level node group. -- `default_commit_scope` (`text`) — The commit scope to use by default, - initially the `local` commit scope. This applies only to the - top-level node group. You can use individual rules for different - origin groups of the same commit scope. See - [Origin groups](../durability/commit-scopes/#origin-groups) for more details. -- `enable_proxy_routing`(`boolean`) — Where [`pgd-proxy`](../routing/proxy) - through the group leader is enabled for given group. Valid values are `on` or `off`. - Default is `off`. -- `enable_raft` (`boolean`) — Whether group has its own Raft consensus. This - is necessary for setting `enable_proxy_routing` to `on`. This is always `on` - for the top-level group. Valid values are `on` or `off`. Default is `off` for subgroups. -- `enable_wal_decoder` (`boolean`) — Enables/disables the decoding worker process. - You can't enable the decoding worker process if `streaming_mode` is already enabled. - Valid values are either `on` or `off`. Default is `off`. -- `location` (`text`) — Information about node location, this is purely metadata for - monitoring. Default is `''` (empty string). -- `num_writers` (`integer`) — Number of parallel writers for the subscription backing this - node group. Valid values are either `-1` or a positive integer. `-1` means the value - specified by the GUC [`bdr.writers_per_subscription`](pgd-settings#bdrwriters_per_subscription) - is used. `-1` is the default, - This option can only be changed for the top-level node group. -- `route_reader_max_lag` (`integer`) — Maximum lag in bytes for a node to be considered a viable - read-only node. Currently reserved for future use. -- `route_writer_max_lag` (`integer`) — Maximum lag in bytes of the new write candidate to be - selected as write leader. If no candidate passes this, no writer is - selected automatically. Default is `-1`. -- `route_writer_wait_flush` (`boolean`) — Whether to switch if we need to wait for the flush. - Currently reserved for future use. -- `streaming_mode` (`text`) — Enables/disables streaming of large transactions. - When set to `off`, streaming is disabled. When set to any other value, - large transactions are decoded while they're still in progress, and the - changes are sent to the downstream. If the value is set to `file`, - then the incoming changes of streaming transactions are stored in a file - and applied only after the transaction is committed on upstream. If the - value is set to `writer`, then the incoming changes are directly sent to - one of the writers, if available. If [parallel apply](../parallelapply) - is disabled or no writer is free to handle streaming transactions, then the - changes are written to a file and applied after the transaction is - committed. If the value is set to `auto`, PGD tries to intelligently pick - between `file` and `writer`, depending on the transaction property and - available resources. You can't enable `streaming_mode` if the WAL decoder - is already enabled. Default is `auto` - For more details, see [Transaction streaming](../transaction-streaming). +| Name | Type | Groups | Description | +| ---- | ---- | ------ | ----------- | +| `apply_delay` | `interval` | All | How long nodes wait to apply incoming changes. This is useful mainly to setup a special sub-group with delayed subscriber-only nodes. Don't set this on groups which contain data nodes or on the top-level group. Default is `0s`. | +| `check_constraints` | `boolean` | Top | Whether the apply process checks the constraints when writing replicated data. It's recommended to keep this to default value, otherwise you risk data loss. Valid values are either `on` or `off`. Default is `on`. | +| `default_commit_scope` | `text` | All | The commit scope to use by default, initially the `local` commit scope. This applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | +| `enable_proxy_routing` | `boolean` | All | Where [`pgd-proxy`](../routing/proxy) through the group leader is enabled for given group. Valid values are `on` or `off`. Default is `off`. | +| `enable_raft` | `boolean` | Sub | Whether group has its own Raft consensus. This is necessary for setting `enable_proxy_routing` to `on`. This is always `on` for the top-level group. Valid values are `on` or `off`. Default is `off` for subgroups. | +| `enable_wal_decoder` | `boolean` | Top | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are either `on` or `off`. Default is `off`. | +| `location` | `text` | All | Information about group location, this is purely metadata for monitoring. Default is `''` (empty string). | +| `num_writers` | `integer` | Top | Number of parallel writers for the subscription backing this node group. Valid values are either `-1` or a positive integer. `-1` means the value specified by the GUC [`bdr.writers_per_subscription`](pgd-settings#bdrwriters_per_subscription) is used. `-1` is the default. | +| `route_reader_max_lag` | `integer` | All | Maximum lag in bytes for a node to be considered a viable read-only node. Currently reserved for future use. | +| `route_writer_max_lag` | `integer` | All | Maximum lag in bytes of the new write candidate to be selected as write leader. If no candidate passes this, no writer is selected automatically. Default is `-1`. | +| `route_writer_wait_flush` | `boolean` | All | Whether to switch if we need to wait for the flush. Currently reserved for future use. | +| `streaming_mode` | `text` | Top | Enables/disables streaming of large transactions. When set to `off`, streaming is disabled. When set to any other value, large transactions are decoded while they're still in progress, and the changes are sent to the downstream. If the value is set to `file`, then the incoming changes of streaming transactions are stored in a file and applied only after the transaction is committed on upstream. If the value is set to `writer`, then the incoming changes are directly sent to one of the writers, if available.
If [parallel apply](../parallelapply) is disabled or no writer is free to handle streaming transactions, then the changes are written to a file and applied after the transaction is committed. If the value is set to `auto`, PGD tries to intelligently pick between `file` and `writer`, depending on the transaction property and available resources. You can't enable `streaming_mode` if the WAL decoder is already enabled. Default is `auto`.

For more details, see [Transaction streaming](../transaction-streaming). | + ### Return value From cefae400da096629c08e8cd0ad6deb55c0c2a284 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 25 Apr 2024 19:17:28 +0100 Subject: [PATCH 13/84] Adjusted reference for better group information Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/reference/index.json | 2 +- product_docs/docs/pgd/5/reference/index.mdx | 2 +- .../reference/nodes-management-interfaces.mdx | 147 ++++++++---------- 3 files changed, 63 insertions(+), 88 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index cfbe6393bb9..fdaa8f28993 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -172,7 +172,6 @@ "list-of-node-states": "/pgd/latest/reference/nodes#list-of-node-states", "node-management-commands": "/pgd/latest/reference/nodes#node-management-commands", "bdr_init_physical": "/pgd/latest/reference/nodes#bdr_init_physical", - "bdralter_node_group_config": "/pgd/latest/reference/nodes-management-interfaces#bdralter_node_group_config", "bdralter_node_group_option": "/pgd/latest/reference/nodes-management-interfaces#bdralter_node_group_option", "bdralter_node_interface": "/pgd/latest/reference/nodes-management-interfaces#bdralter_node_interface", "bdralter_node_option": "/pgd/latest/reference/nodes-management-interfaces#bdralter_node_option", @@ -185,6 +184,7 @@ "bdrpromote_node": "/pgd/latest/reference/nodes-management-interfaces#bdrpromote_node", "bdrswitch_node_group": "/pgd/latest/reference/nodes-management-interfaces#bdrswitch_node_group", "bdrwait_for_join_completion": "/pgd/latest/reference/nodes-management-interfaces#bdrwait_for_join_completion", + "bdralter_node_group_config": "/pgd/latest/reference/nodes-management-interfaces#bdralter_node_group_config", "bdrcreate_proxy": "/pgd/latest/reference/routing#bdrcreate_proxy", "bdralter_proxy_option": "/pgd/latest/reference/routing#bdralter_proxy_option", "bdrdrop_proxy": "/pgd/latest/reference/routing#bdrdrop_proxy", diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index 219ca41f3c6..ed0d54fd9fe 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -242,7 +242,6 @@ The reference section is a definitive listing of all functions, views, and comma ## [Node management interfaces](nodes-management-interfaces) - * [`bdr.alter_node_group_config`](nodes-management-interfaces#bdralter_node_group_config) * [`bdr.alter_node_group_option`](nodes-management-interfaces#bdralter_node_group_option) * [`bdr.alter_node_interface`](nodes-management-interfaces#bdralter_node_interface) * [`bdr.alter_node_option`](nodes-management-interfaces#bdralter_node_option) @@ -255,6 +254,7 @@ The reference section is a definitive listing of all functions, views, and comma * [`bdr.promote_node`](nodes-management-interfaces#bdrpromote_node) * [`bdr.switch_node_group`](nodes-management-interfaces#bdrswitch_node_group) * [`bdr.wait_for_join_completion`](nodes-management-interfaces#bdrwait_for_join_completion) + * [`bdr.alter_node_group_config`](nodes-management-interfaces#bdralter_node_group_config) ## [Routing functions](routing) diff --git a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx index 8fc410511da..63c6495e9af 100644 --- a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx +++ b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx @@ -6,96 +6,11 @@ indexdepth: 2 You can add and remove nodes dynamically using the SQL interfaces. -## `bdr.alter_node_group_config` - -This function changes the configuration parameters of an existing PGD group. -Options with NULL value (default for all of them) aren't modified. - -!!! Warning - This function only exists for compatibility with PGD4 and 3.7. - Please use [`bdr.alter_node_group_option`](#bdralter_node_group_option) instead. - -### Synopsis - -```sql -bdr.alter_node_group_config(node_group_name text, - insert_to_update boolean DEFAULT NULL, - update_to_insert boolean DEFAULT NULL, - ignore_redundant_updates boolean DEFAULT NULL, - check_full_tuple boolean DEFAULT NULL, - apply_delay interval DEFAULT NULL, - check_constraints boolean DEFAULT NULL, - num_writers int DEFAULT NULL, - enable_wal_decoder boolean DEFAULT NULL, - streaming_mode text DEFAULT NULL, - default_commit_scope text DEFAULT NULL) -``` - -### Parameters - -- `node_group_name` — Name of an existing PGD group. The local node must be part - of the group. -- `insert_to_update` — Reserved for backward compatibility. -- `update_to_insert` — Reserved for backward compatibility. - This option is deprecated and will be disabled or removed in future - versions of PGD. Use `bdr.alter_node_set_conflict_resolver` instead. -- `ignore_redundant_updates` — Reserved for backward compatibility. -- `check_full_tuple` — Reserved for backward compatibility. -- `apply_delay` — Reserved for backward compatibility. -- `check_constraints` — Whether the apply process checks the constraints - when writing replicated data. -- `num_writers` — Number of parallel writers for subscription backing - this node group. -1 means the default (as specified by the - GUC `bdr.writers_per_subscription`) is used. Valid values - are either -1 or a positive integer. -- `enable_wal_decoder` — Enables/disables the decoding worker process. - You can't enable the decoding worker process if `streaming_mode` is - already enabled. -- `streaming_mode` — Enables/disables streaming of large transactions. - When set to `off`, streaming is disabled. When set to any other value, - large transactions are decoded while they're still in progress, and the - changes are sent to the downstream. If the value is set to `file`, - then the incoming changes of streaming transactions are stored in a file - and applied only after the transaction is committed on upstream. If the - value is set to `writer`, then the incoming changes are directly sent to - one of the writers, if available. If parallel apply is disabled or no - writer is free to handle streaming transaction, then the changes are - written to a file and applied after the transaction is committed. If the - value is set to `auto`, PGD tries to intelligently pick between - `file` and `writer`, depending on the transaction property and available - resources. You can't enable `streaming_mode` if the WAL - decoder is already enabled. - - For more details, see [Transaction streaming](../transaction-streaming). - -- `default_commit_scope` — The commit scope to use by default, - initially the `local` commit scope. This applies only to the - top-level node group. You can use individual rules for different - origin groups of the same commit scope. See - [Origin groups](../durability/commit-scopes/#origin-groups) for more details. - -### Notes - -This function passes a request to the group consensus mechanism to change -the defaults. The changes made are replicated globally using the consensus -mechanism. - -The function isn't transactional. The request is processed in the background -so you can't roll back the function call. Also, the changes might not be -immediately visible to the current transaction. - -This function doesn't hold any locks. - -!!! Warning - When you use this function to change the `apply_delay` value, the - change doesn't apply to nodes that are already members of the - group. - ## `bdr.alter_node_group_option` -Modify the PGD node group routing configuration. +Modify a PGD node group configuration. ### Synopsis @@ -576,3 +491,63 @@ bdr.wait_for_join_completion(verbose_progress boolean DEFAULT false) This function waits until the checks state of the local node reaches the target state, which was set by `bdr.create_node_group`, `bdr.join_node_group`, or `bdr.promote_node`. + + +## `bdr.alter_node_group_config` + +This function changes the configuration parameters of an existing PGD group. +Options with NULL value (default for all of them) aren't modified. + +!!! Warning + This function only exists for compatibility with PGD4 and 3.7. + Please use [`bdr.alter_node_group_option`](#bdralter_node_group_option) instead. + +### Synopsis + +```sql +bdr.alter_node_group_config(node_group_name text, + insert_to_update boolean DEFAULT NULL, + update_to_insert boolean DEFAULT NULL, + ignore_redundant_updates boolean DEFAULT NULL, + check_full_tuple boolean DEFAULT NULL, + apply_delay interval DEFAULT NULL, + check_constraints boolean DEFAULT NULL, + num_writers int DEFAULT NULL, + enable_wal_decoder boolean DEFAULT NULL, + streaming_mode text DEFAULT NULL, + default_commit_scope text DEFAULT NULL) +``` + +### Parameters + +| Name | Description | +| ---- | ------------| +| `node_group_name` | Name of an existing PGD group. The local node must be part of the group. | +| `insert_to_update` | Reserved for backward compatibility. | +| `update_to_insert` | Reserved for backward compatibility. This option is deprecated and will be disabled or removed in future versions of PGD. Use `bdr.alter_node_set_conflict_resolver` instead. | +| `ignore_redundant_updates` | Reserved for backward compatibility. | +| `check_full_tuple` | Reserved for backward compatibility. | +| `apply_delay` | How long nodes wait to apply incoming changes. This is useful mainly to set up a special sub-group with delayed subscriber-only nodes. Don't set this on groups that contain data nodes or on the top-level group. Default is `0s`. | +| `check_constraints` | Whether the apply process checks the constraints when writing replicated data. It's recommended to keep this to the default value, otherwise you risk data loss. Valid values are either `on` or `off`. Default is `on`. | +| `num_writers` | Number of parallel writers for the subscription backing this node group. Valid values are either `-1` or a positive integer. `-1` means the value specified by the GUC `bdr.writers_per_subscription` is used. `-1` is the default. | +| `enable_wal_decoder` | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are either `on` or `off`. Default is `off`. | +| `streaming_mode` | Enables/disables streaming of large transactions. When set to `off`, streaming is disabled. When set to any other value, large transactions are decoded while they're still in progress, and the changes are sent to the downstream. If the value is set to `file`, then the incoming changes of streaming transactions are stored in a file and applied only after the transaction is committed on upstream. If the value is set to `writer`, then the incoming changes are directly sent to one of the writers, if available. If parallel apply is disabled or no writer is free to handle streaming transaction, then the changes are written to a file and applied after the transaction is committed. If the value is set to `auto`, PGD tries to intelligently pick between `file` and `writer`, depending on the transaction property and available resources. You can't enable `streaming_mode` if the WAL decoder is already enabled.

For more details, see [Transaction streaming](../transaction-streaming). | +| `default_commit_scope` | The commit scope to use by default, initially the `local` commit scope. This applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | + +### Notes + +This function passes a request to the group consensus mechanism to change +the defaults. The changes made are replicated globally using the consensus +mechanism. + +The function isn't transactional. The request is processed in the background +so you can't roll back the function call. Also, the changes might not be +immediately visible to the current transaction. + +This function doesn't hold any locks. + +!!! Warning + When you use this function to change the `apply_delay` value, the + change doesn't apply to nodes that are already members of the + group. + From 55c8320cab11adb78a5825e3e7a743d2cc38895a Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 1 May 2024 09:57:36 +0100 Subject: [PATCH 14/84] fixes as per review Signed-off-by: Dj Walker-Morgan --- .../pgd/5/reference/nodes-management-interfaces.mdx | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx index 63c6495e9af..ff0b0e97ae2 100644 --- a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx +++ b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx @@ -528,10 +528,10 @@ bdr.alter_node_group_config(node_group_name text, | `ignore_redundant_updates` | Reserved for backward compatibility. | | `check_full_tuple` | Reserved for backward compatibility. | | `apply_delay` | How long nodes wait to apply incoming changes. This is useful mainly to set up a special sub-group with delayed subscriber-only nodes. Don't set this on groups that contain data nodes or on the top-level group. Default is `0s`. | -| `check_constraints` | Whether the apply process checks the constraints when writing replicated data. It's recommended to keep this to the default value, otherwise you risk data loss. Valid values are either `on` or `off`. Default is `on`. | -| `num_writers` | Number of parallel writers for the subscription backing this node group. Valid values are either `-1` or a positive integer. `-1` means the value specified by the GUC `bdr.writers_per_subscription` is used. `-1` is the default. | -| `enable_wal_decoder` | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are either `on` or `off`. Default is `off`. | -| `streaming_mode` | Enables/disables streaming of large transactions. When set to `off`, streaming is disabled. When set to any other value, large transactions are decoded while they're still in progress, and the changes are sent to the downstream. If the value is set to `file`, then the incoming changes of streaming transactions are stored in a file and applied only after the transaction is committed on upstream. If the value is set to `writer`, then the incoming changes are directly sent to one of the writers, if available. If parallel apply is disabled or no writer is free to handle streaming transaction, then the changes are written to a file and applied after the transaction is committed. If the value is set to `auto`, PGD tries to intelligently pick between `file` and `writer`, depending on the transaction property and available resources. You can't enable `streaming_mode` if the WAL decoder is already enabled.

For more details, see [Transaction streaming](../transaction-streaming). | +| `check_constraints` | Whether the apply process checks the constraints when writing replicated data. It's recommended to keep this to the default value, otherwise you risk data loss. Valid values are either `on` or `off`. Default is `on`. Applies to top-level group only. | +| `num_writers` | Number of parallel writers for the subscription backing this node group. Valid values are either `-1` or a positive integer. `-1` means the value specified by the GUC `bdr.writers_per_subscription` is used. `-1` is the default. Applies to top-level group only.| +| `enable_wal_decoder` | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are either `on` or `off`. Default is `off`. Applies to top-level group only.| +| `streaming_mode` | Enables/disables streaming of large transactions. When set to `off`, streaming is disabled. When set to any other value, large transactions are decoded while they're still in progress, and the changes are sent to the downstream. If the value is set to `file`, then the incoming changes of streaming transactions are stored in a file and applied only after the transaction is committed on upstream. If the value is set to `writer`, then the incoming changes are directly sent to one of the writers, if available. If parallel apply is disabled or no writer is free to handle streaming transaction, then the changes are written to a file and applied after the transaction is committed. If the value is set to `auto`, PGD tries to intelligently pick between `file` and `writer`, depending on the transaction property and available resources. You can't enable `streaming_mode` if the WAL decoder is already enabled.

For more details, see [Transaction streaming](../transaction-streaming). Applies to top-level group only.| | `default_commit_scope` | The commit scope to use by default, initially the `local` commit scope. This applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | ### Notes @@ -546,8 +546,5 @@ immediately visible to the current transaction. This function doesn't hold any locks. -!!! Warning - When you use this function to change the `apply_delay` value, the - change doesn't apply to nodes that are already members of the - group. + From ecbce1a50077432b631f6f8a8faf917c6c2c01c9 Mon Sep 17 00:00:00 2001 From: smaher-edb Date: Thu, 18 Apr 2024 15:37:05 +0530 Subject: [PATCH 15/84] BDR-4933 - cli/docs changes for v5.5.0 Includes read scalability feature changes and others --- .../5/cli/command_ref/pgd_create-proxy.mdx | 19 +++- .../cli/command_ref/pgd_set-group-options.mdx | 3 +- .../cli/command_ref/pgd_set-proxy-options.mdx | 15 ++- .../pgd/5/cli/command_ref/pgd_show-groups.mdx | 12 +-- .../pgd/5/cli/command_ref/pgd_show-nodes.mdx | 94 ++++++------------- .../5/cli/command_ref/pgd_show-proxies.mdx | 14 +-- 6 files changed, 71 insertions(+), 86 deletions(-) diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_create-proxy.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_create-proxy.mdx index 9a05b8f82df..5dff042d0ce 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_create-proxy.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_create-proxy.mdx @@ -10,6 +10,10 @@ Creates proxy in the EDB Postgres Distributed cluster and attaches it to the given group. The proxy name must be unique across the cluster and match with the name given in the corresponding proxy config file. +Use the proxy mode to route connections to Write Leader (default), Read Nodes +(read-only), or both (any). Proxy listens on 'listen_port' for Write Leader +connections while on 'read_listen_port' for Read Nodes connections. + ```sh pgd create-proxy [flags] @@ -19,9 +23,14 @@ pgd create-proxy [flags] ```text - Example 1 (attach new proxy called proxy-a1 to group bdrgroup) + Example 1 (attach new proxy called proxy-a1 to group group_a, with 'default' mode) + + $ pgd create-proxy --proxy-name proxy-a1 --group-name group_a + proxy created successfully + + Example 2 (attach new proxy called proxy-b1 to group group_b, with 'any' mode) - $ pgd create-proxy --proxy-name proxy-a1 --group-name bdrgroup + $ pgd create-proxy --proxy-name proxy-b1 --group-name group_b --proxy-mode any proxy created successfully ``` @@ -31,13 +40,17 @@ pgd create-proxy [flags] ```text --group-name string group name -h, --help help for create-proxy + --proxy-mode string proxy mode (default, read-only, any); proxy will route connections to - + default - Write Leader + read-only - Read Nodes + any - both Write Leader and Read Nodes (default "default") --proxy-name string proxy name ``` ### Options inherited from parent commands ```text - -f, --config-file string config file; ignored if + -f, --config-file string config file; ignored if --dsn flag is present (default "/etc/edb/pgd-cli/pgd-cli-config.yml") --dsn string database connection string e.g."host=bdr-a1 port=5432 dbname=bdrdb user=postgres " diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_set-group-options.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_set-group-options.mdx index 5f5444bf3e0..0e52a6e9077 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_set-group-options.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_set-group-options.mdx @@ -12,6 +12,7 @@ You can set the following group options with this command: - 'enable_proxy_routing' - 'location' - 'route_writer_max_lag' +- 'route_reader_max_lag' Both 'enable_raft' and 'enable_proxy_routing' must be true if proxy is attached to the group. @@ -57,7 +58,7 @@ pgd set-group-options [flags] ### Options inherited from parent commands ```text - -f, --config-file string config file; ignored if + -f, --config-file string config file; ignored if --dsn flag is present (default "/etc/edb/pgd-cli/pgd-cli-config.yml") --dsn string database connection string e.g."host=bdr-a1 port=5432 dbname=bdrdb user=postgres " diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_set-proxy-options.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_set-proxy-options.mdx index d3d4038dece..f5f5b083ee8 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_set-proxy-options.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_set-proxy-options.mdx @@ -15,9 +15,20 @@ You can set the following proxy options with this command: - 'server_conn_keepalive' - 'server_conn_timeout' - 'consensus_grace_period' +- 'read_listen_address' +- 'read_listen_port' +- 'read_max_client_conn' +- 'read_max_server_conn' +- 'read_server_conn_keepalive' +- 'read_server_conn_timeout' +- 'read_consensus_grace_period' After updating any of these options, restart proxy. +Set 'listen_port' to non-zero value to route traffic to the Write Leader and +set 'read_listen_port' to non-zero value to route traffic to Read nodes. +Setting it to zero will disable the respective routing. + Use 'pgd show-proxies -o json' to view option values for each proxy. @@ -37,13 +48,13 @@ pgd set-proxy-options [flags] Example 2 (multiple --option flags are allowed) - $ pgd set-proxy-options --proxy-name proxy-a1 --option listen_address=0.0.0.0 --option listen_port=6432 + $ pgd set-proxy-options --proxy-name proxy-a1 --option listen_address=0.0.0.0 --option listen_port=0 proxy options updated successfully, please restart proxy service Example 3 (use double quote if option value has spaces or special characters) - $ pgd set-proxy-options --proxy-name proxy-a1 --option "listen_address = 0.0.0.0" --option "listen_port = 6432" + $ pgd set-proxy-options --proxy-name proxy-a1 --option "listen_address = 0.0.0.0" --option "consensus_grace_period=1h 30m 5s" proxy options updated successfully, please restart proxy service ``` diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-groups.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-groups.mdx index a36359a89a5..63f2a97f9f6 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-groups.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-groups.mdx @@ -30,11 +30,11 @@ pgd show-groups [flags] $ pgd show-groups - Group Group ID Type Parent Group Location Raft Routing Write Leader - ----- -------- ---- ------------ -------- ---- ------- ------------ - bdrgroup 1360502012 global true false - group_a 3618712053 data bdrgroup a true true bdr-a1 - group_b 402614658 data bdrgroup b true true bdr-b1 + Group Group ID Type Parent Group Location Raft Routing Raft Leader Write Leader + ----- -------- ---- ------------ -------- ---- ------- ----------- ------------ + bdrgroup 1360502012 global world true false bdr-a2 + group_a 3618712053 data bdrgroup a true true bdr-a2 bdr-a1 + group_b 402614658 data bdrgroup b true true bdr-b1 bdr-b1 group_c 2808307099 data bdrgroup c false false group_so 2123208041 subscriber-only bdrgroup c false false @@ -49,7 +49,7 @@ pgd show-groups [flags] ### Options inherited from parent commands ```text - -f, --config-file string config file; ignored if + -f, --config-file string config file; ignored if --dsn flag is present (default "/etc/edb/pgd-cli/pgd-cli-config.yml") --dsn string database connection string e.g."host=bdr-a1 port=5432 dbname=bdrdb user=postgres " diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx index 7281c5d271a..8db3d6d7245 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx @@ -1,101 +1,61 @@ --- -title: show-nodes +title: set-node-options --- -Shows all nodes in the EDB Postgres Distributed cluster and their summary. +Sets node options such as 'route_fence', 'route_priority', and 'route_writes'. ### Synopsis -Shows all nodes in the EDB Postgres Distributed cluster and their summary, -including name, node id, group, and current/target state. - -Node States - -* NONE: Node state is unset when the worker starts, expected to be set -quickly to the current known state. -* CREATED: bdr.create_node() has been executed, but the node is not a -member of any EDB Postgres Distributed cluster yet. -* JOIN_START: bdr.join_node_group() begins to join the local node to an -existing EDB Postgres Distributed cluster. -* JOINING: The node join has started and is currently at the initial sync -phase, creating the schema and data on the node. -* CATCHUP: Initial sync phase is completed; now the join is at the last step -of retrieving and applying transactions that were performed on the upstream -peer node since the join started. -* STANDBY: Node join has finished, but not yet started to broadcast changes. -All joins spend some time in this state, but if defined as a Logical -Standby, the node will continue in this state. -* PROMOTE: Node was a logical standby and we just called bdr.promote_node to -move the node state to ACTIVE. These two PROMOTE states have to be -coherent to the fact, that only one node can be with a state higher than -STANDBY but lower than ACTIVE. -* PROMOTING: Promotion from logical standby to full BDR node is in progress. -* ACTIVE: The node is a full BDR node and is currently ACTIVE. This is the -most common node status. -* PART_START: Node was ACTIVE or STANDBY and we just called bdr.part_node -to remove the node from the EDB Postgres Distributed cluster. -* PARTING: Node disconnects from other nodes and plays no further part in -consensus or replication. -* PART_CATCHUP: Non-parting nodes synchronize any missing data from the -recently parted node. -* PARTED: Node parting operation is now complete on all nodes. - -Only one node at a time can be in either of the states PROMOTE or PROMOTING. - -Note that the read-only state of a node, as shown in the Current State or in -the Target State columns, is indicated as STANDBY. +You can set the following node options with this command: + +- 'route_dsn' +- 'route_fence' +- 'route_priority' +- 'route_writes' +- 'route_reads' + +Use 'pgd show-nodes -o json' to view option values for each node. + ```sh -pgd show-nodes [flags] +pgd set-node-options [flags] ``` ### Examples ```text - Example 1 (3 node cluster, bdr-a1 and witness-a are up, bdr-a2 is down) + Example 1 (comma separated multiple options, spaces are not allowed) + + $ pgd set-node-options --node-name bdr-a1 --option route_priority=100,route_fence=true + node options updated successfully - $ pgd show-nodes - Node Node ID Group Type Current State Target State Status Seq ID - ---- ------- ----- ---- ------------- ------------ ------ ------ - bdr-a1 3136956818 group_a data ACTIVE ACTIVE Up 1 - bdr-a2 2133699692 group_a data ACTIVE ACTIVE Unreachable 2 - witness-a 3889635963 group_a witness ACTIVE ACTIVE Up 3 + Example 2 (multiple --option flags are allowed) + $ pgd set-node-options --node-name bdr-a1 --option route_priority=100 --option route_fence=true + node options updated successfully - Example 2 (multi-group cluster with witness, logical standby and - subscriber-only nodes) - Note: In contrast to logical standby, the subscriber-only nodes are fully - joined node to the cluster - $ pgd show-nodes + Example 3 (use double quote if option value has spaces or special characters) - Node Node ID Group Type Current State Target State Status Seq ID - ---- ------- ----- ---- ------------- ------------ ------ ------ - bdr-a1 3136956818 group_a data ACTIVE ACTIVE Up 6 - bdr-a2 2133699692 group_a data ACTIVE ACTIVE Up 3 - logical-standby-a1 1140256918 group_a standby STANDBY STANDBY Up 9 - witness-a 3889635963 group_a witness ACTIVE ACTIVE Up 7 - bdr-b1 2380210996 group_b data ACTIVE ACTIVE Up 1 - bdr-b2 2244996162 group_b data ACTIVE ACTIVE Up 2 - logical-standby-b1 3541792022 group_b standby STANDBY STANDBY Up 10 - witness-b 661050297 group_b witness ACTIVE ACTIVE Up 5 - witness-c 1954444188 group_c witness ACTIVE ACTIVE Up 4 - subscriber-only-c1 2448841809 group_so subscriber-only ACTIVE ACTIVE Up 8 + $ pgd set-node-options --node-name bdr-a1 --option "route_priority = 100" --option "route_fence = true" + node options updated successfully ``` ### Options ```text - -h, --help help for show-nodes + -h, --help help for set-node-options + --node-name string node name + --option strings option in name=value format ``` ### Options inherited from parent commands ```text - -f, --config-file string config file; ignored if + -f, --config-file string config file; ignored if --dsn flag is present (default "/etc/edb/pgd-cli/pgd-cli-config.yml") --dsn string database connection string e.g."host=bdr-a1 port=5432 dbname=bdrdb user=postgres " diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-proxies.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-proxies.mdx index 685cc1b7d10..f916395e9e1 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-proxies.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-proxies.mdx @@ -24,12 +24,12 @@ pgd show-proxies [flags] $ pgd show-proxies - Proxy Group Listen Addresses Listen Port - ----- ----- ---------------- ----------- - proxy-a1 group_a [0.0.0.0] 6432 - proxy-a2 group_a [0.0.0.0] 6432 - proxy-b1 group_b [0.0.0.0] 6432 - proxy-b2 group_b [0.0.0.0] 6432 + Proxy Group Listen Addrs Listen Port Read Listen Addrs Read Listen Port + ----- ----- ------------ ----------- ----------------- ---------------- + proxy-a1 group_a [0.0.0.0] 6432 [0.0.0.0] 6433 + proxy-a2 group_a [0.0.0.0] 6432 [0.0.0.0] 6433 + proxy-b1 group_b [0.0.0.0] 6432 [0.0.0.0] 6433 + proxy-b2 group_b [0.0.0.0] 6432 [0.0.0.0] 6433 ``` @@ -42,7 +42,7 @@ pgd show-proxies [flags] ### Options inherited from parent commands ```text - -f, --config-file string config file; ignored if + -f, --config-file string config file; ignored if --dsn flag is present (default "/etc/edb/pgd-cli/pgd-cli-config.yml") --dsn string database connection string e.g."host=bdr-a1 port=5432 dbname=bdrdb user=postgres " From eeb9fdcc5e288d950299a6f3dab03d72c06aeeec Mon Sep 17 00:00:00 2001 From: smaher-edb Date: Mon, 22 Apr 2024 14:28:23 +0530 Subject: [PATCH 16/84] Incorrectly overriden the show-nodes file. Revert the changes. --- .../cli/command_ref/pgd_set-node-options.mdx | 3 +- .../pgd/5/cli/command_ref/pgd_show-nodes.mdx | 92 +++++++++++++------ 2 files changed, 68 insertions(+), 27 deletions(-) diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_set-node-options.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_set-node-options.mdx index d7c03d390d1..8db3d6d7245 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_set-node-options.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_set-node-options.mdx @@ -12,6 +12,7 @@ You can set the following node options with this command: - 'route_fence' - 'route_priority' - 'route_writes' +- 'route_reads' Use 'pgd show-nodes -o json' to view option values for each node. @@ -54,7 +55,7 @@ pgd set-node-options [flags] ### Options inherited from parent commands ```text - -f, --config-file string config file; ignored if + -f, --config-file string config file; ignored if --dsn flag is present (default "/etc/edb/pgd-cli/pgd-cli-config.yml") --dsn string database connection string e.g."host=bdr-a1 port=5432 dbname=bdrdb user=postgres " diff --git a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx index 8db3d6d7245..bad1ff00df8 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/pgd_show-nodes.mdx @@ -1,55 +1,95 @@ --- -title: set-node-options +title: show-nodes --- -Sets node options such as 'route_fence', 'route_priority', and 'route_writes'. +Shows all nodes in the EDB Postgres Distributed cluster and their summary. ### Synopsis -You can set the following node options with this command: - -- 'route_dsn' -- 'route_fence' -- 'route_priority' -- 'route_writes' -- 'route_reads' - -Use 'pgd show-nodes -o json' to view option values for each node. - +Shows all nodes in the EDB Postgres Distributed cluster and their summary, +including name, node id, group, and current/target state. + +Node States + +* NONE: Node state is unset when the worker starts, expected to be set +quickly to the current known state. +* CREATED: bdr.create_node() has been executed, but the node is not a +member of any EDB Postgres Distributed cluster yet. +* JOIN_START: bdr.join_node_group() begins to join the local node to an +existing EDB Postgres Distributed cluster. +* JOINING: The node join has started and is currently at the initial sync +phase, creating the schema and data on the node. +* CATCHUP: Initial sync phase is completed; now the join is at the last step +of retrieving and applying transactions that were performed on the upstream +peer node since the join started. +* STANDBY: Node join has finished, but not yet started to broadcast changes. +All joins spend some time in this state, but if defined as a Logical +Standby, the node will continue in this state. +* PROMOTE: Node was a logical standby and we just called bdr.promote_node to +move the node state to ACTIVE. These two PROMOTE states have to be +coherent to the fact, that only one node can be with a state higher than +STANDBY but lower than ACTIVE. +* PROMOTING: Promotion from logical standby to full BDR node is in progress. +* ACTIVE: The node is a full BDR node and is currently ACTIVE. This is the +most common node status. +* PART_START: Node was ACTIVE or STANDBY and we just called bdr.part_node +to remove the node from the EDB Postgres Distributed cluster. +* PARTING: Node disconnects from other nodes and plays no further part in +consensus or replication. +* PART_CATCHUP: Non-parting nodes synchronize any missing data from the +recently parted node. +* PARTED: Node parting operation is now complete on all nodes. + +Only one node at a time can be in either of the states PROMOTE or PROMOTING. + +Note that the read-only state of a node, as shown in the Current State or in +the Target State columns, is indicated as STANDBY. ```sh -pgd set-node-options [flags] +pgd show-nodes [flags] ``` ### Examples ```text - Example 1 (comma separated multiple options, spaces are not allowed) - - $ pgd set-node-options --node-name bdr-a1 --option route_priority=100,route_fence=true - node options updated successfully + Example 1 (3 node cluster, bdr-a1 and witness-a are up, bdr-a2 is down) + $ pgd show-nodes - Example 2 (multiple --option flags are allowed) + Node Node ID Group Type Current State Target State Status Seq ID + ---- ------- ----- ---- ------------- ------------ ------ ------ + bdr-a1 3136956818 group_a data ACTIVE ACTIVE Up 1 + bdr-a2 2133699692 group_a data ACTIVE ACTIVE Unreachable 2 + witness-a 3889635963 group_a witness ACTIVE ACTIVE Up 3 - $ pgd set-node-options --node-name bdr-a1 --option route_priority=100 --option route_fence=true - node options updated successfully + Example 2 (multi-group cluster with witness, logical standby and + subscriber-only nodes) + Note: In contrast to logical standby, the subscriber-only nodes are fully + joined node to the cluster - Example 3 (use double quote if option value has spaces or special characters) + $ pgd show-nodes - $ pgd set-node-options --node-name bdr-a1 --option "route_priority = 100" --option "route_fence = true" - node options updated successfully + Node Node ID Group Type Current State Target State Status Seq ID + ---- ------- ----- ---- ------------- ------------ ------ ------ + bdr-a1 3136956818 group_a data ACTIVE ACTIVE Up 6 + bdr-a2 2133699692 group_a data ACTIVE ACTIVE Up 3 + logical-standby-a1 1140256918 group_a standby STANDBY STANDBY Up 9 + witness-a 3889635963 group_a witness ACTIVE ACTIVE Up 7 + bdr-b1 2380210996 group_b data ACTIVE ACTIVE Up 1 + bdr-b2 2244996162 group_b data ACTIVE ACTIVE Up 2 + logical-standby-b1 3541792022 group_b standby STANDBY STANDBY Up 10 + witness-b 661050297 group_b witness ACTIVE ACTIVE Up 5 + witness-c 1954444188 group_c witness ACTIVE ACTIVE Up 4 + subscriber-only-c1 2448841809 group_so subscriber-only ACTIVE ACTIVE Up 8 ``` ### Options ```text - -h, --help help for set-node-options - --node-name string node name - --option strings option in name=value format + -h, --help help for show-nodes ``` ### Options inherited from parent commands From 7cfa7ff92dab10fda8da49418e107f788caa86ba Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 2 May 2024 11:24:50 +0100 Subject: [PATCH 17/84] Reorg and add macOS installer instructions Signed-off-by: Dj Walker-Morgan --- .../docs/pgd/5/cli/command_ref/index.mdx | 4 +-- .../docs/pgd/5/cli/configuring_cli.mdx | 1 + .../docs/pgd/5/cli/discover_connections.mdx | 1 + product_docs/docs/pgd/5/cli/index.mdx | 26 ++++++++++++---- .../docs/pgd/5/cli/installing/index.mdx | 12 ++++++++ .../linux.mdx} | 30 ++++++++----------- .../docs/pgd/5/cli/installing/macos.mdx | 21 +++++++++++++ .../docs/pgd/5/cli/installing/tpa.mdx | 13 ++++++++ product_docs/docs/pgd/5/cli/using_cli.mdx | 1 + 9 files changed, 84 insertions(+), 25 deletions(-) create mode 100644 product_docs/docs/pgd/5/cli/installing/index.mdx rename product_docs/docs/pgd/5/cli/{installing_cli.mdx => installing/linux.mdx} (57%) create mode 100644 product_docs/docs/pgd/5/cli/installing/macos.mdx create mode 100644 product_docs/docs/pgd/5/cli/installing/tpa.mdx diff --git a/product_docs/docs/pgd/5/cli/command_ref/index.mdx b/product_docs/docs/pgd/5/cli/command_ref/index.mdx index 1ef87e61391..6668a1e0d64 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/index.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/index.mdx @@ -1,7 +1,7 @@ --- title: Command reference -redirects: - - /pgd/latest/cli/command_ref/pgd_show-camo/ +navTitle: Command reference +description: A reference guide to the commands available in the (PGD CLI). --- pgd is the command name for the PGD command line interface. diff --git a/product_docs/docs/pgd/5/cli/configuring_cli.mdx b/product_docs/docs/pgd/5/cli/configuring_cli.mdx index 37422d707d4..895ec4560e0 100644 --- a/product_docs/docs/pgd/5/cli/configuring_cli.mdx +++ b/product_docs/docs/pgd/5/cli/configuring_cli.mdx @@ -1,6 +1,7 @@ --- title: "Configuring PGD CLI" navTitle: "Configuring PGD CLI" +description: "Configuring PGD CLI for simpler connections to your PGD cluster" --- PGD CLI can be installed on any system that can connect to the PGD cluster. To use PGD CLI, you need a user with PGD superuser privileges or equivalent. The PGD user with superuser privileges is the [bdr_superuser role](../security). An example of an equivalent user is edb_admin on a BigAnimal distributed high-availability cluster. diff --git a/product_docs/docs/pgd/5/cli/discover_connections.mdx b/product_docs/docs/pgd/5/cli/discover_connections.mdx index 04e2a3689d8..d1147559b2f 100644 --- a/product_docs/docs/pgd/5/cli/discover_connections.mdx +++ b/product_docs/docs/pgd/5/cli/discover_connections.mdx @@ -3,6 +3,7 @@ title: "Discovering connection strings" navTitle: "Discovering connection strings" indexdepth: 2 deepToC: true +description: "How to obtain the correct connection strings for your PGD-powered deployment." --- You can install PGD CLI on any system that can connect to the PGD cluster. To use PGD CLI, you need a user with PGD superuser privileges or equivalent. The PGD user with superuser privileges is the [bdr_superuser role](../security). An example of an equivalent user is edb_admin on an EDB BigAnimal distributed high-availability cluster. diff --git a/product_docs/docs/pgd/5/cli/index.mdx b/product_docs/docs/pgd/5/cli/index.mdx index 34e8c3a061a..61ed5b10ad5 100644 --- a/product_docs/docs/pgd/5/cli/index.mdx +++ b/product_docs/docs/pgd/5/cli/index.mdx @@ -1,9 +1,9 @@ --- -title: "EDB Postgres Distributed Command Line Interface" +title: "EDB Postgres Distributed Command Line Interface (PGD CLI)" navTitle: "PGD CLI" -indexCards: none +indexCards: simple navigation: -- installing_cli +- installing - using_cli - configuring_cli - discover_connections @@ -13,9 +13,23 @@ directoryDefaults: description: "The PGD Command Line Interface (CLI) is a tool to manage your EDB Postgres Distributed cluster" --- -The EDB Postgres Distributed Command Line Interface (PGD CLI) is a tool for managing your EDB Postgres Distributed cluster. It allows you to run commands against EDB Postgres Distributed clusters. It's installed automatically on systems in a TPA-deployed PGD cluster. Or it can be installed manually on systems that can connect to any PGD cluster, such as EDB BigAnimal distributed high-availability clusters or PGD clusters deployed using the EDB PGD for Kubernetes operator. +The EDB Postgres Distributed Command Line Interface (PGD CLI) is a tool for managing your EDB Postgres Distributed cluster. It is the key tool for inspecting and managing cluster resources. -* [Installing PGD CLI](installing_cli) has instructions on manually installing PGD CLI on various systems. +It allows you to run commands against EDB Postgres Distributed clusters + * to determine the health of the cluster, inspect the cluster's configuration, and manage the cluster's resources. + * to inspect and manage the cluster's nodes, groups, and proxies. + * to perform switchover operations on the write leaders of groups. + +PGD CLI is installed automatically on systems in a TPA-deployed PGD cluster. + +It can also be installed manually on Linux and macOS systems that can connect to a PGD cluster, including: + * EDB BigAnimal distributed high-availability clusters. + * PGD clusters deployed using the EDB PGD for Kubernetes operator. + * Manually deployed PGD clusters. + * TPA-deployed PGD clusters. + + + diff --git a/product_docs/docs/pgd/5/cli/installing/index.mdx b/product_docs/docs/pgd/5/cli/installing/index.mdx new file mode 100644 index 00000000000..af78ef30dde --- /dev/null +++ b/product_docs/docs/pgd/5/cli/installing/index.mdx @@ -0,0 +1,12 @@ +--- +title: "Installing PGD CLI" +navTitle: "Installing PGD CLI" +redirects: + - /pgd/latest/cli/installing_cli +deepToC: true +indexCards: simple +description: Installing the PGD CLI on various systems. +--- + +You can install PGD CLI on any system that can connect to the PGD cluster. Linux and macOS are currently supported platforms to install PGD CLI on. + diff --git a/product_docs/docs/pgd/5/cli/installing_cli.mdx b/product_docs/docs/pgd/5/cli/installing/linux.mdx similarity index 57% rename from product_docs/docs/pgd/5/cli/installing_cli.mdx rename to product_docs/docs/pgd/5/cli/installing/linux.mdx index 1695307879a..4bf0ad1573d 100644 --- a/product_docs/docs/pgd/5/cli/installing_cli.mdx +++ b/product_docs/docs/pgd/5/cli/installing/linux.mdx @@ -1,24 +1,16 @@ --- -title: "Installing PGD CLI" -navTitle: "Installing PGD CLI" -deepToC: true +title: Installing PGD CLI on Linux +navTitle: Linux +description: Installing PGD CLI on Linux --- -You can install PGD CLI on any system that can connect to the PGD cluster. To use PGD CLI, you need a user with PGD superuser privileges or equivalent. The PGD user with superuser privileges is the [bdr_superuser role](../security). An example of an equivalent user is edb_admin on an EDB BigAnimal distributed high-availability cluster. + PGD CLI is available for most Linux distributions. It is installable from the EDB repositories, which you can access with your EDB account. PGD users and BigAnimal users, including those on a free trial, have an EDB account and access to PGD CLI. -## Installing automatically with Trusted Postgres Architect (TPA) - -By default, Trusted Postgres Architect installs and configures PGD CLI on each PGD node. If you want to install PGD CLI on any non-PGD instance in the cluster, attach the pgdcli role to that instance in Trusted Postgres Architect's configuration file before deploying. See [Trusted Postgres Architect](/tpa/latest/) for more information. - -## Installing manually on Linux - -PGD CLI is installable from the EDB repositories, which you can access with your EDB account. PGD users and BigAnimal users, including those on a free trial, have an EDB account and access to PGD CLI. - -### Obtain your EDB subscription token +## Obtain your EDB subscription token These repositories require a token to enable downloads from them. To obtain your token, log in to [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads). If this is your first time visiting the EDB Repos 2.0 page, you must select **Request Access** to generate your token. Once a generated token is available, select the **Copy** icon to copy it to your clipboard, or select the eye icon to view it. -### Set the EDB_SUBSCRIPTION_TOKEN environment variable +## Set the EDB_SUBSCRIPTION_TOKEN environment variable Once you have the token, execute the command shown for your operating system, substituting your token for ``. @@ -28,7 +20,7 @@ export EDB_SUBSCRIPTION_TOKEN= Then run the appropriate commands for your operating system. -### Install on Debian or Ubuntu +## Install on Debian or Ubuntu ```bash curl -1sSLf "https://downloads.enterprisedb.com/$EDB_SUBSCRIPTION_TOKEN/postgres_distributed/setup.deb.sh" | sudo -E bash @@ -49,7 +41,7 @@ You can now install the PGD CLI package using the command: sudo apt-get install edb-pgd5-cli ``` -### Install on RHEL, Rocky, AlmaLinux, or Oracle Linux +## Install on RHEL, Rocky, AlmaLinux, or Oracle Linux ```bash curl -1sSLf "https://downloads.enterprisedb.com/$EDB_SUBSCRIPTION_TOKEN/postgres_distributed/setup.rpm.sh" | sudo -E bash @@ -70,4 +62,8 @@ You can now install the PGD CLI package using the command: sudo yum install edb-pgd5-cli ``` -[Next: Using PGD CLI](using_cli) +[Next: Using PGD CLI](../using_cli) + + + + diff --git a/product_docs/docs/pgd/5/cli/installing/macos.mdx b/product_docs/docs/pgd/5/cli/installing/macos.mdx new file mode 100644 index 00000000000..e98cdcca5a6 --- /dev/null +++ b/product_docs/docs/pgd/5/cli/installing/macos.mdx @@ -0,0 +1,21 @@ +--- +title: Installing PGD CLI on macOS +navTitle: macOS +description: Installing PGD CLI on macOS +--- + +PGD CLI is available for macOS as a [Homebrew](https://brew.sh/) formula. To install it, run the following commands: + +```shell +brew tap enterprisedb/tap +brew install pgd-cli +``` + +Verify the installation by running: + +```shell +pgd --version +``` + + +[Next: Using PGD CLI](../using_cli) diff --git a/product_docs/docs/pgd/5/cli/installing/tpa.mdx b/product_docs/docs/pgd/5/cli/installing/tpa.mdx new file mode 100644 index 00000000000..eaf43b0ee7f --- /dev/null +++ b/product_docs/docs/pgd/5/cli/installing/tpa.mdx @@ -0,0 +1,13 @@ +--- +title: "Installing PGD CLI with TPA" +navTitle: "TPA" +description: "Installing PGD CLI with Trusted Postgres Architect" +--- + +By default, Trusted Postgres Architect installs and configures PGD CLI on each PGD node. + +If you want to install PGD CLI on any non-PGD instance in the cluster, attach the pgdcli role to that instance in Trusted Postgres Architect's configuration file before deploying. + +See [Trusted Postgres Architect](/tpa/latest/) for more information. + +[Next: Using PGD CLI](../using_cli) diff --git a/product_docs/docs/pgd/5/cli/using_cli.mdx b/product_docs/docs/pgd/5/cli/using_cli.mdx index fa1c89e3c71..511d98767fc 100644 --- a/product_docs/docs/pgd/5/cli/using_cli.mdx +++ b/product_docs/docs/pgd/5/cli/using_cli.mdx @@ -1,6 +1,7 @@ --- title: "Using PGD CLI" navTitle: "Using PGD CLI" +description: "How to use the PGD Command Line Interface (CLI) to manage your EDB Postgres Distributed cluster." --- ## What is the PGD CLI? From 1730b4895ad2554e0ee8c53d4d9eb1ce088c7d18 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Fri, 3 May 2024 12:00:51 +0100 Subject: [PATCH 18/84] Added Template Release Notes Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/rel_notes/index.mdx | 8 ++- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 55 +++++++++++++++++++ 2 files changed, 60 insertions(+), 3 deletions(-) create mode 100644 product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx diff --git a/product_docs/docs/pgd/5/rel_notes/index.mdx b/product_docs/docs/pgd/5/rel_notes/index.mdx index f7851b087cb..c13a2afc7f6 100644 --- a/product_docs/docs/pgd/5/rel_notes/index.mdx +++ b/product_docs/docs/pgd/5/rel_notes/index.mdx @@ -3,6 +3,7 @@ title: "EDB Postgres Distributed release notes" navTitle: "Release notes" description: "Release notes for EDB Postgres Distributed" navigation: +- pgd_5.5.0_rel_notes - pgd_5.4.1_rel_notes - pgd_5.4.0_rel_notes - pgd_5.3.0_rel_notes @@ -22,12 +23,13 @@ provide information on what was new in each release. For new functionality introduced in a minor or patch release, the content also indicates the release that introduced the feature. -| Release Date | EDB Postgres Distributed | BDR extension | PGD CLI | PGD Proxy | -| ------------- | ---------------------------- | ------------- | ------- | --------- | +| Release Date | EDB Postgres Distributed | BDR extension | PGD CLI | PGD Proxy | +|--------------|------------------------------|---------------|---------|-----------| +| TBD | [5.5.0](pgd_5.5.0_rel_notes) | 5.5.0 | 5.5.0 | 5.5.0 | | 03 Apr 2024 | [5.4.1](pgd_5.4.1_rel_notes) | 5.4.1 | 5.4.0 | 5.4.0 | | 05 Mar 2024 | [5.4.0](pgd_5.4.0_rel_notes) | 5.4.0 | 5.4.0 | 5.4.0 | | 14 Nov 2023 | [5.3.0](pgd_5.3.0_rel_notes) | 5.3.0 | 5.3.0 | 5.3.0 | -| 04 Aug 2023 | [5.2.0](pgd_5.2.0_rel_notes) | 5.2.0 | 5.2.0 | 5.2.0 | +| 04 Aug 2023 | [5.2.0](pgd_5.2.0_rel_notes) | 5.2.0 | 5.2.0 | 5.2.0 | | 16 May 2023 | [5.1.0](pgd_5.1.0_rel_notes) | 5.1.0 | 5.1.0 | 5.1.0 | | 21 Mar 2023 | [5.0.1](pgd_5.0.1_rel_notes) | 5.0.0 | 5.0.1 | 5.0.1 | | 21 Feb 2023 | [5.0.0](pgd_5.0.0_rel_notes) | 5.0.0 | 5.0.0 | 5.0.0 | diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx new file mode 100644 index 00000000000..7e7c12975d5 --- /dev/null +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -0,0 +1,55 @@ +--- +title: "EDB Postgres Distributed 5.5.0 release notes" +navTitle: "Version 5.5.0" +--- + +Released: TBD + +EDB Postgres Distributed version 5.5.0 is a minor version of EDB Postgres Distributed. + +!!! Important Recommended upgrade +We recommend that all users of PGD 5 upgrade to PGD 5.5. See [PGD/TPA upgrades](../upgrades/tpa_overview) for details. +!!! + + +## Highlights of EDB Postgres Distributed 5.5.0 + +Highlights of this 5.5.0 release include: + +* read scalability enhancements in PGD Proxy which allow read-only queries to be routed to nodes that are not the write leader, improving the overall performance of the cluster. + +## Compatibility + +!!! Note EDB Server version compatibility +This version requires the recently released Postgres versions 14.10, 15.4, +or 16.1 (or later) of EDB Postgres Advanced Server or EDB Postgres Extended +Server. No such restrictions exist for PostgreSQL Server. + +Package managers on Debian, RHEL, and SLES pull in the required EDB Postgres +Advanced Server or EDB Postgres Extended upgrades with an upgrade of EDB +Postgres Distributed. +!!! + +## Features + +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| +| | | | | +| | | | | + + +## Enhancements + +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| +| | | | | +| | | | | + + + +## Bug fixes + +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------| +| | | | | +| | | | | From af9aa2eaef420ee0a940875ad6a62747e956e8db Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Mon, 6 May 2024 18:19:47 +0100 Subject: [PATCH 19/84] Release notes draft 0 Signed-off-by: Dj Walker-Morgan --- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 43 +++++++++++++------ 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index 7e7c12975d5..b6fcf3f9a91 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -32,24 +32,43 @@ Postgres Distributed. ## Features -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| -| | | | | -| | | | | +| Component | Version | Description | Addresses | +|-----------|---------|----------------------------------------|-----------| +| BDR | 5.5.0 | Add support for readonly proxy routing | | +| | | | | ## Enhancements -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| -| | | | | -| | | | | +| Component | Version | Description | Addresses | +|-----------|---------|-------------------------------------------------------------------------------------------------------------------------|-----------| +| BDR | 5.5.0 | Improve bulk INSERT/UPDATE/DELETE performance by sending multiple messages together in a group rather than individually | | +| BDR | 5.5.0 | Log completion of an extension upgrade | | +| BDR | 5.5.0 | Add restrictions for group commit options | | +| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | +| BDR | 5.5.0 | Execute each autopartition task in its own transaction | RT101407 | +| BDR | 5.5.0 | Use DETACH CONCURRENTLY to drop partitions. | RT101407 | +| BDR | 5.5.0 | Disallow node group creation on a node bad state | | +| BDR | 5.5.0 | Grant additional object permissions to role "bdr_read_all_stats" | | +| BDR | 5.5.0 | Add support for readonly proxy routing | | +| BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | +| BDR | 5.5.0 | Report current manager activity in pg_stat_activity | | +| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | +| BDR | 5.5.0 | Restart the replication connection for bdr_init_physical | | +| BDR | 5.5.0 | Add replication parameter to connectdb() | | + ## Bug fixes -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------| -| | | | | -| | | | | +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------|-----------| +| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956 | +| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | +| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | +| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times() | | +| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | +| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql| | + From d6f94dcc736080792a983ae4b0bcb6c67cafce57 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Mon, 6 May 2024 18:20:07 +0100 Subject: [PATCH 20/84] Fixes for readonly docs Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/routing/readonly.mdx | 31 +++++++++++++------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/product_docs/docs/pgd/5/routing/readonly.mdx b/product_docs/docs/pgd/5/routing/readonly.mdx index d53cd09c0df..76e559a6918 100644 --- a/product_docs/docs/pgd/5/routing/readonly.mdx +++ b/product_docs/docs/pgd/5/routing/readonly.mdx @@ -38,9 +38,9 @@ Proxy creation functions in PGD take an optional `proxy-mode` parameter. This pa * `read-only`: This creates a read-only proxy which routes traffic to nodes that are not the write leader. It only handles this read-only traffic on port 6433. * `any`: This creates create a proxy that can handle both read-only and write leader following traffic on seperate ports; 6432 for write leader following traffic and 6433 for read-only traffic. -## Creating a Read-Only Proxy +### Creating a Read-Only Proxy -### Using SQL +#### Using SQL To create a new read-only proxy, use the `bdr.create_proxy` function. @@ -48,9 +48,9 @@ To create a new read-only proxy, use the `bdr.create_proxy` function. SELECT bdr.create_proxy('proxy-ro1','group-a','read-only'); ``` -This will create a new read-only proxy named `proxy-ro1` in group `group-a` and, by default, will listen on port 7432 for read-only traffic. +This will create a new read-only proxy named `proxy-ro1` in group `group-a` and, by default, will listen on port 6433 for read-only traffic. -### Using PGD CLI +#### Using PGD CLI To create a new read-only proxy, use the `pgd create-proxy` command with the optional `--proxy_mode` flag set to `read-only`. @@ -58,13 +58,26 @@ To create a new read-only proxy, use the `pgd create-proxy` command with the opt pgd create-proxy --proxy-name proxy-ro1 --node-group group-a --proxy-mode read-only ``` -## Configuring Read-Only Routing - +## Configuring running proxies + !!! Note To change a proxy's configuration, the proxy must be restarted after changes have been made. !!! -### Using SQL +Activating read-only routing on a proxy is done by setting the `read_listen_port` option to a port number. This port number is the port on which the proxy will listen for read-only traffic. +If the proxy already has a listen_port set, then the proxy will listen on both ports, routing read/write and read-only traffic respectively on each port. +This is equivalent to creating a proxy with `proxy-mode` set to `any`. + +If you set a `read_listen_port` on a proxy and then set the `listen_port` to 0, the proxy will only listen on the `read_listen_port` and will only route read-only traffic. +This is equivalent to creating a proxy with `proxy-mode` set to `read-only`. +The configuration elements related to the read/write port will be cleared (set to null). + +If you set a `listen_port` on a proxy and then set the `read_listen_port` to 0, the proxy will only listen on the `listen_port` and will only route read/write traffic. +This is equivalent to creating a proxy with `proxy-mode` set to `default`. +The configuration elements related to the read-only port will be cleared (set to null). + + +### Configuring using SQL To configure a read-only proxy port on a proxy, use the `bdr.alter_proxy_options` function. @@ -80,7 +93,7 @@ To remove the read-only proxy set the port to 0. SELECT bdr.alter_proxy_options('proxy-a1','read_listen_port','0'); ``` -### Using PGD CLI +### Configuring using PGD CLI To configure a read-only proxy port on a proxy, use the `pgd alter-proxy` command. @@ -97,5 +110,3 @@ pgd set-proxy-options --proxy-name proxy-a1 --option read_listen_port=0 ``` - - From 4bafa42369ce66a41013dbe02e566aff36799a55 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 7 May 2024 08:50:46 +0100 Subject: [PATCH 21/84] Updated monitoring with new endpoints Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/routing/monitoring.mdx | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/product_docs/docs/pgd/5/routing/monitoring.mdx b/product_docs/docs/pgd/5/routing/monitoring.mdx index 530583661fa..ff68715db4f 100644 --- a/product_docs/docs/pgd/5/routing/monitoring.mdx +++ b/product_docs/docs/pgd/5/routing/monitoring.mdx @@ -26,12 +26,14 @@ The `show-proxy` command to view the status of a specific proxy. PGD Proxy provides the following HTTP(s) health check API endpoints. The API endpoints respond to `GET` requests. You need to enable and configure the endpoints before using them. See [Configurations](installing_proxy#configuring-health-check). -``` -GET /health/is-ready -GET /health/is-live -``` - -#### Readiness +| Endpoint | Description | +| --- | --- | +| `/health/is-ready` | Checks if the proxy can successfully route connections to the current write leader. | +| `/health/is-live` | Checks if the proxy is running. | +| `/health/is-write-ready` | Checks if the proxy can successfully route connections to the current write leader. PGD 5.5.0 and later | +| `/health/is-read-only-ready` | Checks if the proxy can successfully route read-only connections. PGD 5.5.0 and later | + +#### Readiness On receiving a valid `GET` request, the proxy checks if it can successfully route connections to the current write leader. If the check returns successfully, the API responds with a body containing `true` and an HTTP status code `200 (OK)`. Otherwise, it returns a body containing `false` with the HTTP status code `500 (Internal Server Error)`. From 0565e28c2e89dfbac796d0f62f297f119f2f0455 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 7 May 2024 13:21:28 +0100 Subject: [PATCH 22/84] Fixes from Petr comments Signed-off-by: Dj Walker-Morgan --- .../pgd/5/reference/catalogs-internal.mdx | 2 +- .../reference/nodes-management-interfaces.mdx | 4 +-- product_docs/docs/pgd/5/routing/proxy.mdx | 33 ++++++++++++++++--- 3 files changed, 30 insertions(+), 9 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx index 62e0293ba5a..551d5631670 100644 --- a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx +++ b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx @@ -87,7 +87,7 @@ Per node group routing configuration options. | node_group_name | name | Node group name | | location | name | Node group location | | enable_proxy_routing | boolean | Group proxy routing enabled? | -| node_group_type | text | Node group type (one of "global", "data", "shard" or "subscriber-only") | +| node_group_type | text | Node group type (one of "global", "data", or "subscriber-only") | | route_writer_max_lag | bigint | Maximum write lag accepted | | route_reader_max_lag | bigint | Maximum read lag accepted | | route_writer_wait_flush | boolean | Wait for flush | diff --git a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx index ff0b0e97ae2..4278dd840da 100644 --- a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx +++ b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx @@ -255,12 +255,10 @@ bdr.create_node_group(node_group_name text, creating a subgroup means the local node won't join the new group, for example, when creating a independent remote group. In this case, you must specify `parent_group_name`. -- `node_group_type` — The valid values are `NULL`, `subscriber-only`, - and `shard`. `NULL` (the default) is for creating a normal, general-purpose +- `node_group_type` — The valid values are `NULL` or `subscriber-only`. `NULL` (the default) is for creating a normal, general-purpose node group. `subscriber-only` is for creating [subscriber-only groups](../node_management/subscriber_only/) whose members receive changes only from the fully joined nodes in the cluster but that never send changes to other nodes. - `shard` is reserved for future use. ### Notes diff --git a/product_docs/docs/pgd/5/routing/proxy.mdx b/product_docs/docs/pgd/5/routing/proxy.mdx index 59afb47ed42..a60f76ffd72 100644 --- a/product_docs/docs/pgd/5/routing/proxy.mdx +++ b/product_docs/docs/pgd/5/routing/proxy.mdx @@ -1,12 +1,9 @@ --- title: "EDB Postgres Distributed Proxy Overview" navTitle: "PGD Proxy Overview" -indexCards: none -navigation: -- installing_proxy - +indexCards: simple directoryDefaults: - description: "The PGD Proxy is a service that acts as proxy layer between the client application and Postgres for your EDB Postgres Distributed cluster" + description: "The PGD Proxy service acts as proxy layer between the client application and Postgres for your PGD cluster" --- @@ -23,6 +20,32 @@ region). Once configured, the PGD Proxy daemon monitors routing configuration changes as decided by the EDB Postgres Distributed cluster. It acts on these changes to ensure that connections are consistently routed to the correct nodes. +The information about currently selected write and read nodes is visible in +`bdr.node_group_routing_summary`. Note that this is node-local view, the proxy +always reads from Raft leader to get current and consistent view. + +## Leader selection + +The write leader is selected by the current Raft leader (either subgroup one or top-level group one, +depending on whether the leader for sub-group or the cluster's top-level group is being selected). + +Leader is selected from candidate nodes which are reachable and meet the criteria based +on the configuration as described bellow. To be a viable candidate, node has to have +`route_writes` enabled and `route_fence` disabled and be within `route_writer_max_lag` +(if enabled) from previous leader. The candidates are ordered by their `route_priority` +in descending order and by the lag from previous leader in ascending order. + +New leader selection process is started when there is either no existing leader currently +(this could be because there were no valid candidates or because Raft was down), or when +connectivity is lost to the existing leader. + +A Node is considered connected if the last Raft protocol message received from by the leader +it is not older than Raft election timeout +(see [Internal settings - Raft timeouts](/pgd/latest/reference/pgd-settings#internal-settings---raft-timeouts) ). + +Since the Raft leader is sending heartbeat 3 times every election timeout limit, the leader +node needs to miss reply to 3 heartbeats before it's considered disconnected. + ## PGD Proxy cluster configuration The PGD cluster always has at least one top-level group and one data group. PGD elects the write leader for each data group that has the `enable_proxy_routing` and `enable_raft` options set to true. From 42cb265b6ed09ca1416e883ac9b65f20cea27dbe Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> Date: Tue, 7 May 2024 13:25:07 +0100 Subject: [PATCH 23/84] Typo fix on product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx --- product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index b6fcf3f9a91..3085c07c12e 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -68,7 +68,7 @@ Postgres Distributed. | BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | | BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | | BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | -| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times() | | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | | BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | | BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql| | From 06744e69173e5ad9abbcbafef04050092dab29ad Mon Sep 17 00:00:00 2001 From: Jagdish Kewat Date: Tue, 7 May 2024 20:34:51 +0530 Subject: [PATCH 24/84] Add release notes for PGD Proxy and CLI v5.5.0 --- .../docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index 3085c07c12e..6c4afe4b9bc 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -32,10 +32,11 @@ Postgres Distributed. ## Features -| Component | Version | Description | Addresses | -|-----------|---------|----------------------------------------|-----------| -| BDR | 5.5.0 | Add support for readonly proxy routing | | -| | | | | +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------|-----------| +| BDR | 5.5.0 | Add support for readonly proxy routing | | +| CLI | 5.5.0 | Create PGD CLI binaries for MacOS and Windows | | +| Proxy | 5.5.0 | Add support for readonly proxy routing | | ## Enhancements @@ -56,6 +57,11 @@ Postgres Distributed. | BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | | BDR | 5.5.0 | Restart the replication connection for bdr_init_physical | | | BDR | 5.5.0 | Add replication parameter to connectdb() | | +| CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | +| CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | +| CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | +| CLI | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | +| Proxy | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | From 6ec1e0b8989bb8681add323a590ed54f84f763b5 Mon Sep 17 00:00:00 2001 From: Jagdish Kewat Date: Tue, 7 May 2024 20:35:26 +0530 Subject: [PATCH 25/84] Add release notes for HARP v2.4.1 --- .../01_release_notes/harp2.4.1_rel_notes.mdx | 9 +++++++++ .../pgd/3.7/harp/01_release_notes/index.mdx | 2 ++ product_docs/docs/pgd/4/rel_notes/index.mdx | 2 ++ .../pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx | 18 ++++++++++++++++++ 4 files changed, 31 insertions(+) create mode 100644 product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx create mode 100644 product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx diff --git a/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx b/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx new file mode 100644 index 00000000000..cad3e5e64b7 --- /dev/null +++ b/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx @@ -0,0 +1,9 @@ +--- +title: "Version 2.4.1" +--- + +This is a patch release of HARP 2 that includes internal maintenance fixes. + +| Type | Description | +| ---- |------------ | +| Change | Routine security library upgrades and bump golang version | diff --git a/product_docs/docs/pgd/3.7/harp/01_release_notes/index.mdx b/product_docs/docs/pgd/3.7/harp/01_release_notes/index.mdx index 8467591676c..6421139e38b 100644 --- a/product_docs/docs/pgd/3.7/harp/01_release_notes/index.mdx +++ b/product_docs/docs/pgd/3.7/harp/01_release_notes/index.mdx @@ -1,6 +1,7 @@ --- title: Release Notes navigation: +- harp2.4.1_rel_notes - harp2.4.0_rel_notes - harp2.3.2_rel_notes - harp2.3.1_rel_notes @@ -27,6 +28,7 @@ The release notes in this section provide information on what was new in each re | Version | Release Date | | ----------------------- | ------------ | +| [2.4.1](harp2.4.1_rel_notes) | 14 May 2024 | | [2.4.0](harp2.4.0_rel_notes) | 05 Mar 2024 | | [2.3.2](harp2.3.2_rel_notes) | 17 Oct 2023 | | [2.3.1](harp2.3.1_rel_notes) | 27 Jul 2023 | diff --git a/product_docs/docs/pgd/4/rel_notes/index.mdx b/product_docs/docs/pgd/4/rel_notes/index.mdx index 64abe56586a..cee1c4c51ac 100644 --- a/product_docs/docs/pgd/4/rel_notes/index.mdx +++ b/product_docs/docs/pgd/4/rel_notes/index.mdx @@ -2,6 +2,7 @@ title: "EDB Postgres Distributed Release notes" navTitle: "Release notes" navigation: +- pgd_4.3.5_rel_notes - pgd_4.3.4_rel_notes - pgd_4.3.3_rel_notes - pgd_4.3.2+p1_rel_notes @@ -29,6 +30,7 @@ The EDB Postgres Distributed documentation describes the latest version of EDB P | Release Date | EDB Postgres Distributed | BDR | HARP | CLI | TPAexec | |--------------|-------------------------------------|-------|-------|-------|--------------------------------------------------------------| +| 14 May 2024 | [4.3.5](pgd_4.3.5_rel_notes) | 4.3.4 | 2.4.1 | 1.1.2 | [23.31](/tpa/latest/rel_notes/tpa_23.31_rel_notes/) | | 05 Mar 2024 | [4.3.4](pgd_4.3.4_rel_notes) | 4.3.4 | 2.4 | 1.1.2 | [23.29](/tpa/latest/rel_notes/tpa_23.29_rel_notes/) | | 14 Nov 2023 | [4.3.3](pgd_4.3.3_rel_notes) | 4.3.3 | 2.3.2 | 1.1.2 | [23.24](/tpa/latest/rel_notes/tpa_23.24_rel_notes) | | 17 Oct 2023 | [4.3.2+p1](pgd_4.3.2+p1_rel_notes) | 4.3.2 | 2.3.2 | 1.1.1 | [23.20](/tpa/latest/rel_notes/tpa_23.20_rel_notes) | diff --git a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx new file mode 100644 index 00000000000..cb2e435dd37 --- /dev/null +++ b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx @@ -0,0 +1,18 @@ +--- +title: "Release notes for EDB Postgres Distributed version 4.3.5" +navTitle: "Version 4.3.5" +redirects: + - /pgd/latest/bdr/release_notes/bdr4.3.5_rel_notes/ +--- + +Released: 14 May 2024 + +EDB Postgres Distributed version 4.3.5 is a patch release of EDB Postgres Distributed 4, which includes bug fixes for issues identified in previous versions. + +!!! Note +This version is required for EDB Postgres Advanced Server versions 12.15, 13.11, 14.10, 15.5 and later. +!!! + +| Component | Version | Type | Description | +|-----------|---------|--------------|---------------------------------------------------------------------------------------------| +| HARP | 2.4.1 | Change | Routine security library upgrades and bump golang version | From 048a7f042dc0df82025000ce703036446100cd17 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 8 May 2024 11:46:05 +0100 Subject: [PATCH 26/84] Reorg front page Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/index.mdx | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/product_docs/docs/pgd/5/index.mdx b/product_docs/docs/pgd/5/index.mdx index b1194e3ad25..1ae3c2913da 100644 --- a/product_docs/docs/pgd/5/index.mdx +++ b/product_docs/docs/pgd/5/index.mdx @@ -13,34 +13,32 @@ navigation: - "#Get Started" - quickstart - planning - - architectures - - choosing_server - - deployments - - other_considerations - - limitations - deploy-config - "#Using" - appusage + - ddl + - sequences + - "#Administration" - node_management - postgres-configuration - - ddl + - routing + - backup - security - - sequences + - monitoring + - testingandtuning + - upgrades + - "#Tools" + - cli + - "#PGD Features" - durability - consistency - parallelapply - repsets - - routing - - cli - - backup - - monitoring - - transaction-streaming - - testingandtuning - striggers - scaling - twophase + - transaction-streaming - tssnapshots - - upgrades - "#Reference" - reference --- @@ -67,4 +65,4 @@ EDB Postgres Advanced Server | 12-16 Postgres 16 support is only available in EDB Postgres Distributed 5.3 and later !!! -See the [compatibility matrix](/pgd/4/#compatibility-matrix) for previous versions +For feature compatibility with compatible servers, see [Choosing a Postgres Distribution](/pgd/latest/planning/choosing_server). \ No newline at end of file From a2c177f4f4d2d1a0d1e240aa529494c9430d81de Mon Sep 17 00:00:00 2001 From: Jagdish Kewat Date: Wed, 8 May 2024 16:39:54 +0530 Subject: [PATCH 27/84] Update description for harp --- .../docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx | 2 +- product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx b/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx index cad3e5e64b7..727e1c4b2a3 100644 --- a/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx +++ b/product_docs/docs/pgd/3.7/harp/01_release_notes/harp2.4.1_rel_notes.mdx @@ -6,4 +6,4 @@ This is a patch release of HARP 2 that includes internal maintenance fixes. | Type | Description | | ---- |------------ | -| Change | Routine security library upgrades and bump golang version | +| Change | Routine security library upgrades | diff --git a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx index cb2e435dd37..c96fba6f0d2 100644 --- a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx +++ b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx @@ -15,4 +15,4 @@ This version is required for EDB Postgres Advanced Server versions 12.15, 13.11, | Component | Version | Type | Description | |-----------|---------|--------------|---------------------------------------------------------------------------------------------| -| HARP | 2.4.1 | Change | Routine security library upgrades and bump golang version | +| HARP | 2.4.1 | Change | Routine security library upgrades | From b433195e67a730e0d277bfac23cc0ad6605588b1 Mon Sep 17 00:00:00 2001 From: Jagdish Kewat Date: Wed, 8 May 2024 17:01:25 +0530 Subject: [PATCH 28/84] Remove the "Windows" part for CLI binaries from description --- product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index 6c4afe4b9bc..6b1ee146102 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -35,7 +35,7 @@ Postgres Distributed. | Component | Version | Description | Addresses | |-----------|---------|-----------------------------------------------|-----------| | BDR | 5.5.0 | Add support for readonly proxy routing | | -| CLI | 5.5.0 | Create PGD CLI binaries for MacOS and Windows | | +| CLI | 5.5.0 | Create PGD CLI binaries for MacOS | | | Proxy | 5.5.0 | Add support for readonly proxy routing | | From 50a21b011f54bc2e0b1bdff7151cfef58381f8fc Mon Sep 17 00:00:00 2001 From: Florin Irion Date: Wed, 8 May 2024 18:25:34 +0200 Subject: [PATCH 29/84] BDR4.3.5 relnotes + some on 5.5.0 --- .../pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx | 14 ++++++++--- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 24 ++++++++++--------- 2 files changed, 24 insertions(+), 14 deletions(-) diff --git a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx index c96fba6f0d2..8c0e0a54add 100644 --- a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx +++ b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx @@ -13,6 +13,14 @@ EDB Postgres Distributed version 4.3.5 is a patch release of EDB Postgres Distri This version is required for EDB Postgres Advanced Server versions 12.15, 13.11, 14.10, 15.5 and later. !!! -| Component | Version | Type | Description | -|-----------|---------|--------------|---------------------------------------------------------------------------------------------| -| HARP | 2.4.1 | Change | Routine security library upgrades | +| Component | Version | Type | Description | +|-----------|---------|--------------|----------------------------------------------------------------------------------------------------| +| HARP | 2.4.1 | Change | Routine security library upgrades. | +| BDR | 5.5.0 | Enhancement | Log completion of an extension upgrade. | +| BDR | 5.5.0 | Enhancement | Execute each autopartition task in its own transaction. | +| BDR | 5.5.0 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | +| BDR | 5.5.0 | Enhancement | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser. | +| BDR | 5.5.0 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | +| BDR | 5.5.0 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | +| BDR | 5.5.0 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | +| BDR | 5.5.0 | Bug Fix | Use bdr.default_sequence_kind when updating sequence kind of existig sequences upon node creation. | \ No newline at end of file diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index 6b1ee146102..f2a2b84ab3d 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -44,6 +44,7 @@ Postgres Distributed. | Component | Version | Description | Addresses | |-----------|---------|-------------------------------------------------------------------------------------------------------------------------|-----------| | BDR | 5.5.0 | Improve bulk INSERT/UPDATE/DELETE performance by sending multiple messages together in a group rather than individually | | +| BDR | 5.5.0 | Don't save changes received by the writer to a temp file | | | BDR | 5.5.0 | Log completion of an extension upgrade | | | BDR | 5.5.0 | Add restrictions for group commit options | | | BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | @@ -55,8 +56,8 @@ Postgres Distributed. | BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | | BDR | 5.5.0 | Report current manager activity in pg_stat_activity | | | BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | -| BDR | 5.5.0 | Restart the replication connection for bdr_init_physical | | -| BDR | 5.5.0 | Add replication parameter to connectdb() | | +| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | | +| BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | | CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | | CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | | CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | @@ -68,13 +69,14 @@ Postgres Distributed. ## Bug fixes -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------------------------------------------|-----------| -| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956 | -| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | -| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | -| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | -| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | -| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | -| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql| | +| Component | Version | Description | Addresses | +|-----------|---------|----------------------------------------------------------------------------------------------------|-----------| +| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956 | +| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | +| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | +| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | +| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | +| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | +| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existig sequences upon node creation. | | From 7e06ad2d74f99787d0f81ad865dd4be880793090 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 09:43:01 +0100 Subject: [PATCH 30/84] Release notes merged Signed-off-by: Dj Walker-Morgan --- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 34 +++++++++---------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index f2a2b84ab3d..cee4d24f7c7 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -32,11 +32,11 @@ Postgres Distributed. ## Features -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------|-----------| -| BDR | 5.5.0 | Add support for readonly proxy routing | | -| CLI | 5.5.0 | Create PGD CLI binaries for MacOS | | -| Proxy | 5.5.0 | Add support for readonly proxy routing | | +| Component | Version | Description | Addresses | +|-----------|---------|----------------------------------------|-----------| +| BDR | 5.5.0 | Add support for readonly proxy routing | | +| CLI | 5.5.0 | Create PGD CLI binaries for MacOS | | +| Proxy | 5.5.0 | Add support for readonly proxy routing | | ## Enhancements @@ -56,7 +56,7 @@ Postgres Distributed. | BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | | BDR | 5.5.0 | Report current manager activity in pg_stat_activity | | | BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | -| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | | +| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | RT102828 | | BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | | CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | | CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | @@ -69,14 +69,14 @@ Postgres Distributed. ## Bug fixes -| Component | Version | Description | Addresses | -|-----------|---------|----------------------------------------------------------------------------------------------------|-----------| -| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956 | -| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | -| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | -| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | -| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | -| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | -| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | -| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existig sequences upon node creation. | | - +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------------------------|-----------| +| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956 | +| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | +| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | +| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | +| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | +| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | +| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | +| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | From f3ce04a62f15b6c89754dcd75ffde0ca643bd5e6 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 10:10:53 +0100 Subject: [PATCH 31/84] Late changes to read_only additions Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/routing/index.mdx | 6 ++--- .../docs/pgd/5/routing/monitoring.mdx | 8 ++++++- product_docs/docs/pgd/5/routing/proxy.mdx | 24 ++++++++++++------- 3 files changed, 26 insertions(+), 12 deletions(-) diff --git a/product_docs/docs/pgd/5/routing/index.mdx b/product_docs/docs/pgd/5/routing/index.mdx index 3fb82efd242..70aa43b77d5 100644 --- a/product_docs/docs/pgd/5/routing/index.mdx +++ b/product_docs/docs/pgd/5/routing/index.mdx @@ -17,13 +17,13 @@ Managing application connections is an important part of high availability. PGD * [PGD Proxy overview](/pgd/latest/routing/proxy) provides an overview of the PGD Proxy, its processes and how it interacts with the EDB Postgres Distributed cluster. -* [Installing the PGD Proxy daemon](/pgd/latest/routing/installing_proxy) covers installation of the PGD Proxy daemon on a host. +* [Installing the PGD Proxy service](/pgd/latest/routing/installing_proxy) covers installation of the PGD Proxy service on a host. -* [Configuring PGD Proxy](/pgd/latest/routing/configuration) details the three levels (group, node, and proxy) of configuration on a cluster that control how the PGD Proxy daemons behave. +* [Configuring PGD Proxy](/pgd/latest/routing/configuration) details the three levels (group, node, and proxy) of configuration on a cluster that control how the PGD Proxy service behaves. * [Administering PGD Proxy](/pgd/latest/routing/administering) shows how to switch the write leader and manage the PGD Proxy. -* [Monitoring PGD Proxy](/pgd/latest/routing/monitoring) looks at how to monitor PGD Proxy through the cluster and at a daemon level. +* [Monitoring PGD Proxy](/pgd/latest/routing/monitoring) looks at how to monitor PGD Proxy through the cluster and at a service level. * [Read-Only Routing](/pgd/latest/routing/readonly) explains how the read-only routing feature in PGD Proxy enables read scalability. diff --git a/product_docs/docs/pgd/5/routing/monitoring.mdx b/product_docs/docs/pgd/5/routing/monitoring.mdx index ff68715db4f..554bb9fffb3 100644 --- a/product_docs/docs/pgd/5/routing/monitoring.mdx +++ b/product_docs/docs/pgd/5/routing/monitoring.mdx @@ -35,7 +35,13 @@ PGD Proxy provides the following HTTP(s) health check API endpoints. The API end #### Readiness -On receiving a valid `GET` request, the proxy checks if it can successfully route connections to the current write leader. If the check returns successfully, the API responds with a body containing `true` and an HTTP status code `200 (OK)`. Otherwise, it returns a body containing `false` with the HTTP status code `500 (Internal Server Error)`. +On receiving a valid `GET` request: + +* When in default (write mode), the proxy checks if it can successfully route connections to the current write leader. +* When in read-only mode, the proxy checks if it can successfully route read-only connections. +* When in any mode, the proxy first checks if it can successfully route connections to the current write leader. If it can, the check is successful. If not, it checks if it can route a read-only connection. If it can, the check is successful. If not, the check fails. + +If the check returns successfully, the API responds with a body containing `true` and an HTTP status code `200 (OK)`. Otherwise, it returns a body containing `false` with the HTTP status code `500 (Internal Server Error)`. #### Liveness diff --git a/product_docs/docs/pgd/5/routing/proxy.mdx b/product_docs/docs/pgd/5/routing/proxy.mdx index a60f76ffd72..ae485a2b759 100644 --- a/product_docs/docs/pgd/5/routing/proxy.mdx +++ b/product_docs/docs/pgd/5/routing/proxy.mdx @@ -13,12 +13,14 @@ important to avoid conflicts and guarantee availability for the application. There are two parts to EDB Postgres Distributed's proxy layer: * Proxy configuration and routing information which is maintained by the PGD consensus mechanism. -* The PGD Proxy daemon which is installed on a host. It connects to the PGD cluster where it reads its configuration and listens for changes to the routing information. +* The PGD Proxy service which is installed on a host. It connects to the PGD cluster where it reads its configuration and listens for changes to the routing information. -This is normally installed in highly available configuration (at least two instances of the daemon per -region). +This is normally installed in highly available configuration (at least two instances of the proxy service per PGD group). -Once configured, the PGD Proxy daemon monitors routing configuration changes as decided by the EDB Postgres Distributed cluster. It acts on these changes to ensure that connections are consistently routed to the correct nodes. +Once configured, the PGD Proxy service monitors routing changes as decided by the EDB Postgres Distributed cluster. It acts on these changes to ensure that connections are consistently routed to the correct nodes. + +Configuration changes to the PGD Proxy service are made through the PGD cluster. +The PGD Proxy service reads its configuration from the PGD cluster, but the proxy service will need to be restarted to apply those changes. The information about currently selected write and read nodes is visible in `bdr.node_group_routing_summary`. Note that this is node-local view, the proxy @@ -52,12 +54,12 @@ The PGD cluster always has at least one top-level group and one data group. PGD The cluster also maintains Proxy configurations for each group. Each configuration has a name and is associated with a group. You can attach Proxy to a top-level group or data group. You can attach multiple proxies to each group. -When a PGD Proxy daemon starts running on a host, it has a name in its local configuration file and it will connect to a node in a group, from where it will use the name to look up its complete configuration as stored on the group. +When a PGD Proxy service starts running on a host, it has a name in its local configuration file and it will connect to a node in a group, from where it will use the name to look up its complete configuration as stored on the group. -## PGD Proxy daemon +## PGD Proxy service -The PGD Proxy daemon (EDB Postgres Distributed Proxy) is a process that acts as an abstraction layer between the client application and Postgres. It interfaces with the PGD consensus mechanism to get the identity of the current write leader node and redirects traffic to that node. +The PGD Proxy service (EDB Postgres Distributed Proxy) is a process that acts as an abstraction layer between the client application and Postgres. It interfaces with the PGD consensus mechanism to get the identity of the current write leader node and redirects traffic to that node. It also optionally supports a read-only mode where it can route read-only queries to nodes that are not the write leader, improving the overall performance of the cluster. PGD Proxy is a TCP layer 4 proxy. @@ -67,7 +69,7 @@ Upon starting, PGD Proxy connects to one of the endpoints given in the local con - DB connection information for all nodes - Proxy options like listen address, listen port -- Routing details including the current write leader +- Routing details including the current write leader in default mode, read nodes in read-only mode, or both in any mode. The endpoints given in the config file are used only at startup. After that, actual endpoints are taken from the PGD catalog's `route_dsn` field in [`bdr.node_routing_config_summary`](/pgd/latest/reference/catalogs-internal#bdrnode_routing_config_summary). @@ -79,6 +81,8 @@ Automatic transfer of write leadership from the current write leader node to a n User-controlled, manual transfer of write leadership from the current write leader to a new target leader is called *switchover*. Switchover is triggered through the [PGD CLI switchover](../cli/command_ref/pgd_switchover) command. The command is submitted to PGD, which attempts to elect the given target node as the new write leader. Similar to failover, PGD Proxy closes existing client connections and redirects new client connections to the newly elected write leader. This is useful during server maintenance, for example, if the current write leader node needs to be stopped for maintenance like a server update or OS patch update. +If the proxy is configured to support read-only routing, it can route read-only queries to a pool of nodes that are not the write leader. The pool of nodes is maintained by the PGD cluster and proxies listen for changes to the pool. When the pool changes, the proxy updates its routing configuration and starts routing read-only queries to the new pool of nodes and disconnecting existing client connections to nodes that have left the pool. + ### Consensus grace period PGD Proxy provides the `consensus_grace_period` proxy option that can be used to configure the routing behavior upon loss of a Raft leader. PGD Proxy continues to route to the current write leader (if it's available) for this duration. If the new Raft leader isn't elected during this period, the proxy stops routing. If set to `0s`, PGD Proxy stops routing immediately. @@ -93,6 +97,10 @@ In this case, if the grace period is kept too high, then writes continue to happ Having said that, most of the time, upon loss of the current Raft leader, the new Raft leader gets elected by BDR within a few seconds if more than half of the nodes (quorum) are still up. Hence, if the Raft leader is down but the write leader is still up, then proxy can be configured to allow routing by keeping `consensus_grace_period` to a non-zero, positive value. The proxy waits for the Raft leader to get elected during this period before stopping routing. This might be helpful in some cases where availability is more important. +### Read consensus grace period + +Similar to the `consensus_grace_period`, a `read_consensus_grace_period` option is available for read-only routing. This option can be used to configure the routing behavior upon loss of a Raft leader for read-only queries. PGD Proxy continues to route to the current read nodes for this duration. If the new Raft leader isn't elected during this period, the proxy stops routing read-only queries. If set to `0s`, PGD Proxy stops routing read-only queries immediately. + ### Multi-host connection strings The PostgreSQL C client library (libpq) allows you to specify multiple host names in a single connection string for simple failover. This is also supported by client libraries (drivers) in some other programming languages. It works well for failing over across PGD Proxy instances that are down or inaccessible. From 2fb6445c4a507b0c9b24b3e3853a7e7345e91077 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 09:27:15 +0100 Subject: [PATCH 32/84] Fix up 4.3.5 relnote to match modern template Signed-off-by: Dj Walker-Morgan --- .../pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx index 8c0e0a54add..daa829567ee 100644 --- a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx +++ b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx @@ -13,14 +13,14 @@ EDB Postgres Distributed version 4.3.5 is a patch release of EDB Postgres Distri This version is required for EDB Postgres Advanced Server versions 12.15, 13.11, 14.10, 15.5 and later. !!! -| Component | Version | Type | Description | -|-----------|---------|--------------|----------------------------------------------------------------------------------------------------| -| HARP | 2.4.1 | Change | Routine security library upgrades. | -| BDR | 5.5.0 | Enhancement | Log completion of an extension upgrade. | -| BDR | 5.5.0 | Enhancement | Execute each autopartition task in its own transaction. | -| BDR | 5.5.0 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | -| BDR | 5.5.0 | Enhancement | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser. | -| BDR | 5.5.0 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | -| BDR | 5.5.0 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | -| BDR | 5.5.0 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | -| BDR | 5.5.0 | Bug Fix | Use bdr.default_sequence_kind when updating sequence kind of existig sequences upon node creation. | \ No newline at end of file +| Component | Version | Type | Description | Addresses | +|-----------|---------|-------------|----------------------------------------------------------------------------------------------------|-----------| +| HARP | 2.4.1 | Change | Routine security library upgrades. | | +| BDR | 4.3.5 | Enhancement | Log completion of an extension upgrade. | | +| BDR | 4.3.5 | Enhancement | Execute each autopartition task in its own transaction. | | +| BDR | 4.3.5 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | | +| BDR | 4.3.5 | Enhancement | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser. | | +| BDR | 4.3.5 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | | +| BDR | 4.3.5 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | | +| BDR | 4.3.5 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | | +| BDR | 4.3.5 | Bug Fix | Use bdr.default_sequence_kind when updating sequence kind of existig sequences upon node creation. | | \ No newline at end of file From 4d1ec46a9dd3ba6fbe2f6f081ab8b9084ec4ac30 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 10:18:05 +0100 Subject: [PATCH 33/84] Snagging dates and formats Signed-off-by: Dj Walker-Morgan --- .../docs/pgd/5/cli/installing/linux.mdx | 1 - .../pgd/5/reference/catalogs-internal.mdx | 16 ++-- product_docs/docs/pgd/5/rel_notes/index.mdx | 2 +- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 79 ++++++++++--------- 4 files changed, 49 insertions(+), 49 deletions(-) diff --git a/product_docs/docs/pgd/5/cli/installing/linux.mdx b/product_docs/docs/pgd/5/cli/installing/linux.mdx index 4bf0ad1573d..8189fb21832 100644 --- a/product_docs/docs/pgd/5/cli/installing/linux.mdx +++ b/product_docs/docs/pgd/5/cli/installing/linux.mdx @@ -66,4 +66,3 @@ sudo yum install edb-pgd5-cli - diff --git a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx index 551d5631670..2cf0fbf2548 100644 --- a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx +++ b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx @@ -82,15 +82,15 @@ Per node group routing configuration options. #### `bdr.node_group_routing_config_summary` columns -| Name | Type | Description | -|-------------------------|---------|-------------------------------------------------------------------------| -| node_group_name | name | Node group name | -| location | name | Node group location | -| enable_proxy_routing | boolean | Group proxy routing enabled? | +| Name | Type | Description | +|-------------------------|---------|-----------------------------------------------------------------| +| node_group_name | name | Node group name | +| location | name | Node group location | +| enable_proxy_routing | boolean | Group proxy routing enabled? | | node_group_type | text | Node group type (one of "global", "data", or "subscriber-only") | -| route_writer_max_lag | bigint | Maximum write lag accepted | -| route_reader_max_lag | bigint | Maximum read lag accepted | -| route_writer_wait_flush | boolean | Wait for flush | +| route_writer_max_lag | bigint | Maximum write lag accepted | +| route_reader_max_lag | bigint | Maximum read lag accepted | +| route_writer_wait_flush | boolean | Wait for flush | ### `bdr.node_group_routing_info` diff --git a/product_docs/docs/pgd/5/rel_notes/index.mdx b/product_docs/docs/pgd/5/rel_notes/index.mdx index c13a2afc7f6..d2a430c566b 100644 --- a/product_docs/docs/pgd/5/rel_notes/index.mdx +++ b/product_docs/docs/pgd/5/rel_notes/index.mdx @@ -25,7 +25,7 @@ that introduced the feature. | Release Date | EDB Postgres Distributed | BDR extension | PGD CLI | PGD Proxy | |--------------|------------------------------|---------------|---------|-----------| -| TBD | [5.5.0](pgd_5.5.0_rel_notes) | 5.5.0 | 5.5.0 | 5.5.0 | +| 14 May 2024 | [5.5.0](pgd_5.5.0_rel_notes) | 5.5.0 | 5.5.0 | 5.5.0 | | 03 Apr 2024 | [5.4.1](pgd_5.4.1_rel_notes) | 5.4.1 | 5.4.0 | 5.4.0 | | 05 Mar 2024 | [5.4.0](pgd_5.4.0_rel_notes) | 5.4.0 | 5.4.0 | 5.4.0 | | 14 Nov 2023 | [5.3.0](pgd_5.3.0_rel_notes) | 5.3.0 | 5.3.0 | 5.3.0 | diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index cee4d24f7c7..a289fa91743 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -3,7 +3,7 @@ title: "EDB Postgres Distributed 5.5.0 release notes" navTitle: "Version 5.5.0" --- -Released: TBD +Released: 14th May 2024 EDB Postgres Distributed version 5.5.0 is a minor version of EDB Postgres Distributed. @@ -32,51 +32,52 @@ Postgres Distributed. ## Features -| Component | Version | Description | Addresses | -|-----------|---------|----------------------------------------|-----------| -| BDR | 5.5.0 | Add support for readonly proxy routing | | -| CLI | 5.5.0 | Create PGD CLI binaries for MacOS | | -| Proxy | 5.5.0 | Add support for readonly proxy routing | | +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------|-----------| +| BDR | 5.5.0 | Add support for read-only proxy routing | | +| CLI | 5.5.0 | Add PGD CLI binaries for MacOS | | +| Proxy | 5.5.0 | Add support for read-only proxy routing | | ## Enhancements -| Component | Version | Description | Addresses | -|-----------|---------|-------------------------------------------------------------------------------------------------------------------------|-----------| -| BDR | 5.5.0 | Improve bulk INSERT/UPDATE/DELETE performance by sending multiple messages together in a group rather than individually | | -| BDR | 5.5.0 | Don't save changes received by the writer to a temp file | | -| BDR | 5.5.0 | Log completion of an extension upgrade | | -| BDR | 5.5.0 | Add restrictions for group commit options | | -| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | -| BDR | 5.5.0 | Execute each autopartition task in its own transaction | RT101407 | -| BDR | 5.5.0 | Use DETACH CONCURRENTLY to drop partitions. | RT101407 | -| BDR | 5.5.0 | Disallow node group creation on a node bad state | | -| BDR | 5.5.0 | Grant additional object permissions to role "bdr_read_all_stats" | | -| BDR | 5.5.0 | Add support for readonly proxy routing | | -| BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | -| BDR | 5.5.0 | Report current manager activity in pg_stat_activity | | -| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | -| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | RT102828 | -| BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | -| CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | -| CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | -| CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | -| CLI | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | -| Proxy | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | +| Component | Version | Description | Addresses | +|-----------|---------|-------------------------------------------------------------------------------------------------------------------------|----------------| +| BDR | 5.5.0 | Improve bulk INSERT/UPDATE/DELETE performance by sending multiple messages together in a group rather than individually | | +| BDR | 5.5.0 | Don't save changes received by the writer to a temp file | | +| BDR | 5.5.0 | Log completion of an extension upgrade | | +| BDR | 5.5.0 | Add restrictions for group commit options | | +| BDR | 5.5.0 | Execute each autopartition task in its own transaction | RT101407/35476 | +| BDR | 5.5.0 | Use DETACH CONCURRENTLY to drop partitions. | RT101407/35476 | +| BDR | 5.5.0 | Disallow node group creation on a node bad state | | +| BDR | 5.5.0 | Grant additional object permissions to role "bdr_read_all_stats" | | +| BDR | 5.5.0 | Add support for read-only proxy routing | | +| BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | +| BDR | 5.5.0 | Report current manager activity in pg_stat_activity | | +| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | +| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | RT102828/35305 | +| BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | +| CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | +| CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | +| CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | +| CLI | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | +| Proxy | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | ## Bug fixes -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------------------------------------------------------------|-----------| -| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956 | -| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290 | -| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | -| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966 | -| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | -| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | -| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | -| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | -| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------------------------|----------------| +| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956/31896 | +| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290/34051 | +| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | +| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | +| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966/35447 | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | +| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | +| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | +| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | +| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | + From 0d6398c0af458f747a377a98327b79da803c9313 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 11:38:44 +0100 Subject: [PATCH 34/84] Remove cgroups and other TPS 23.32 fixes. Signed-off-by: Dj Walker-Morgan --- .../pgd/5/quickstart/quick_start_docker.mdx | 48 +++---------------- 1 file changed, 6 insertions(+), 42 deletions(-) diff --git a/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx b/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx index 247f411daba..98d534043a9 100644 --- a/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx +++ b/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx @@ -108,33 +108,6 @@ You'll use TPA to provision and deploy PGD. If you previously installed TPA, you [TPA supports several distributions of Linux](/tpa/latest/INSTALL/) as a host platform. These examples are written for Ubuntu 22.04, but steps are similar for other supported platforms. -!!! Important - If the Linux host platform you're using is running [cgroups](https://en.wikipedia.org/wiki/Cgroups) v2, you need to disable it and enable cgroups v1 while using TPA to deploy to Docker. - - To check for cgroup v2, run: - - ```shell - mount | grep cgroup | head -1 - ``` - - If you do **not** see a line beginning: - - `tmpfs on /sys/fs/cgroup type tmpfs` - - Then you need to to disable cgroup v2. To do this, run: - - ```shell - echo 'GRUB_CMDLINE_LINUX=systemd.unified_cgroup_hierarchy=false' | sudo tee /etc/default/grub.d/cgroup.cfg - ``` - - Then update the grub bootloader and reboot by running: - - ```shell - sudo update-grub - sudo reboot - ``` - - ### Install the TPA package ```shell @@ -176,7 +149,8 @@ tpaexec configure democluster \ --location-names dc1 \ --pgd-proxy-routing local \ --no-git \ - --hostnames-unsorted + --hostnames-unsorted \ + --keyring-backend legacy ``` You specify the PGD-Always-ON architecture (`--architecture PGD-Always-ON`), which @@ -200,7 +174,9 @@ You set the notional location of the nodes to `dc1` using `--location-names`. Yo By default, TPA commits configuration changes to a Git repository. For this example, you don't need to do that, so pass the `--no-git` flag. -Finally, you ask TPA to generate repeatable hostnames for the nodes by passing `--hostnames-unsorted`. Otherwise, it selects hostnames at random from a predefined list of suitable words. +You also ask TPA to generate repeatable hostnames for the nodes by passing `--hostnames-unsorted`. Otherwise, it selects hostnames at random from a predefined list of suitable words. + +Finally, `--keyring-backend legacy` tells that the keyring backend should be the legacy backend. Secrets are stored with an older keyring backend, as the version of Ubuntu this example is based on doesn't support the newer keyring backend. This command creates a subdirectory in the current working directory called `democluster`. It contains the `config.yml` configuration file TPA uses to create the cluster. You can view it using: @@ -218,22 +194,10 @@ less democluster/config.yml - [`tpaexec configure`](/tpa/latest/tpaexec-configure/) in the Trusted Postgres Architect documentation - [Docker platform](/tpa/latest/platform-docker/) in the Trusted Postgres Architect documentation -### Provisioning the cluster - -Next, allocate the resources needed to run the configuration you just created using the [`tpaexec provision`](/tpa/latest/tpaexec-provision/) command: - -```shell -tpaexec provision democluster -``` - -Since you specified Docker as the platform, TPA creates a Docker image, containers, networks, and so on. - -!!! SeeAlso "Further reading" - - [`tpaexec provision`](/tpa/latest/tpaexec-provision/) in the Trusted Postgres Architect documentation ### Deploying the cluster -With configuration in place and infrastructure provisioned, you can now [deploy](/tpa/latest/tpaexec-deploy/) the distributed cluster: +You can now [deploy](/tpa/latest/tpaexec-deploy/) the distributed cluster. The `tpaexec` command will automatically run its `provision` command to prepare the Docker containers for the deployment: ```shell tpaexec deploy democluster From fb991408ca4ff125a1b22d8a9efec7d2438ffe88 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 12:09:56 +0100 Subject: [PATCH 35/84] Resolve review comments Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index a289fa91743..4fd568b9c9a 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -35,6 +35,7 @@ Postgres Distributed. | Component | Version | Description | Addresses | |-----------|---------|-----------------------------------------|-----------| | BDR | 5.5.0 | Add support for read-only proxy routing | | +| BDR | 5.5.0 | Improve stability of routing leader selection by using Raft hearbeat for connectivity check | | | CLI | 5.5.0 | Add PGD CLI binaries for MacOS | | | Proxy | 5.5.0 | Add support for read-only proxy routing | | @@ -51,9 +52,11 @@ Postgres Distributed. | BDR | 5.5.0 | Use DETACH CONCURRENTLY to drop partitions. | RT101407/35476 | | BDR | 5.5.0 | Disallow node group creation on a node bad state | | | BDR | 5.5.0 | Grant additional object permissions to role "bdr_read_all_stats" | | -| BDR | 5.5.0 | Add support for read-only proxy routing | | +| BDR | 5.5.0 | Improve stability of manager worker and Raft consensus by not throwing error on non-fatal dynamic shared memory read failures | | | BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | -| BDR | 5.5.0 | Report current manager activity in pg_stat_activity | | +| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | +| BDR | 5.5.0 | Expose heatbeat timings in Raft get_raft_status() | | +| BDR | 5.5.0 | Expose follower info in Raft get_raft_status() on Raft leader nodes | | | BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | | BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | RT102828/35305 | | BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | From 2501504539f0588078fddb9a8110684cf880e462 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 12:12:09 +0100 Subject: [PATCH 36/84] Remove internal issues from relnotes Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 4 ---- 1 file changed, 4 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index 4fd568b9c9a..c290e947884 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -63,10 +63,6 @@ Postgres Distributed. | CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | | CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | | CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | -| CLI | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | -| Proxy | 5.5.0 | Upgrade 3rd party dependencies to fix Github dependabot alerts | | - - ## Bug fixes From 53444b744c3866d1062cf01d7f97cd5c3c40066e Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 12:15:01 +0100 Subject: [PATCH 37/84] Final bug additions to rel_notes Signed-off-by: Dj Walker-Morgan --- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 27 ++++++++++--------- 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index c290e947884..ef7eff51171 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -67,16 +67,19 @@ Postgres Distributed. ## Bug fixes -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------------------------------------------------------------------|----------------| -| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956/31896 | -| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290/34051 | -| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | -| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | -| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966/35447 | -| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | -| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | -| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | -| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | -| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | +| Component | Version | Description | Addresses | +|-----------|---------|---------------------------------------------------------------------------------------------------------|----------------| +| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956/31896 | +| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290/34051 | +| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | +| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | +| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966/35447 | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | +| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | +| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | +| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | +| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | +| BDR | 5.5.0 | Fixed a bug preventing some trusted extension management commands (CREATE/ALTER) from being replicated. | | + + From 2fbc093eee59379b1cad9320792eac5914aee444 Mon Sep 17 00:00:00 2001 From: Florin Irion Date: Tue, 14 May 2024 13:39:35 +0200 Subject: [PATCH 38/84] Update pgd_4.3.5_rel_notes.mdx Add customer facing issues --- .../docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx index daa829567ee..c05ebd51aab 100644 --- a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx +++ b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx @@ -18,9 +18,9 @@ This version is required for EDB Postgres Advanced Server versions 12.15, 13.11, | HARP | 2.4.1 | Change | Routine security library upgrades. | | | BDR | 4.3.5 | Enhancement | Log completion of an extension upgrade. | | | BDR | 4.3.5 | Enhancement | Execute each autopartition task in its own transaction. | | -| BDR | 4.3.5 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | | +| BDR | 4.3.5 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | RT101407 | | BDR | 4.3.5 | Enhancement | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser. | | -| BDR | 4.3.5 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | | -| BDR | 4.3.5 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | | -| BDR | 4.3.5 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | | -| BDR | 4.3.5 | Bug Fix | Use bdr.default_sequence_kind when updating sequence kind of existig sequences upon node creation. | | \ No newline at end of file +| BDR | 4.3.5 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | RT102828 | +| BDR | 4.3.5 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | RT99956 | +| BDR | 4.3.5 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | RT102290 | +| BDR | 4.3.5 | Bug Fix | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation.| | From 2c3c2f7f46ced04e96f0ee65d471d2830d720cb8 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 12:39:40 +0100 Subject: [PATCH 39/84] Ensured proxy naming correct Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/reference/routing.mdx | 6 +-- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 54 +++++++++---------- 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index c0ef5e1b374..0a68bb007a5 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -7,7 +7,7 @@ rootisheading: false ### `bdr.create_proxy` -Create a proxy +Create a proxy configuration #### Synopsis @@ -28,7 +28,7 @@ When proxy_mode is set to `default` all read options in the proxy config will be ### `bdr.alter_proxy_option` -Change a proxy +Change a proxy configuration #### Synopsis @@ -68,7 +68,7 @@ Changing any of theses values requires a restart of the proxy. ### `bdr.drop_proxy` -Drop a proxy +Drop a proxy configuration #### Synopsis diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index ef7eff51171..36ba4826089 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -32,37 +32,37 @@ Postgres Distributed. ## Features -| Component | Version | Description | Addresses | -|-----------|---------|-----------------------------------------|-----------| -| BDR | 5.5.0 | Add support for read-only proxy routing | | -| BDR | 5.5.0 | Improve stability of routing leader selection by using Raft hearbeat for connectivity check | | -| CLI | 5.5.0 | Add PGD CLI binaries for MacOS | | -| Proxy | 5.5.0 | Add support for read-only proxy routing | | +| Component | Version | Description | Addresses | +|-----------|---------|-----------------------------------------------------------------------------------------------|-----------| +| BDR | 5.5.0 | Add support for read-only proxy routing | | +| BDR | 5.5.0 | Improve stability of routing leader selection by using Raft heartbeat for connectivity check | | +| CLI | 5.5.0 | Add PGD CLI binaries for MacOS | | +| Proxy | 5.5.0 | Add support for read-only proxy routing | | ## Enhancements -| Component | Version | Description | Addresses | -|-----------|---------|-------------------------------------------------------------------------------------------------------------------------|----------------| -| BDR | 5.5.0 | Improve bulk INSERT/UPDATE/DELETE performance by sending multiple messages together in a group rather than individually | | -| BDR | 5.5.0 | Don't save changes received by the writer to a temp file | | -| BDR | 5.5.0 | Log completion of an extension upgrade | | -| BDR | 5.5.0 | Add restrictions for group commit options | | -| BDR | 5.5.0 | Execute each autopartition task in its own transaction | RT101407/35476 | -| BDR | 5.5.0 | Use DETACH CONCURRENTLY to drop partitions. | RT101407/35476 | -| BDR | 5.5.0 | Disallow node group creation on a node bad state | | -| BDR | 5.5.0 | Grant additional object permissions to role "bdr_read_all_stats" | | -| BDR | 5.5.0 | Improve stability of manager worker and Raft consensus by not throwing error on non-fatal dynamic shared memory read failures | | -| BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | -| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | -| BDR | 5.5.0 | Expose heatbeat timings in Raft get_raft_status() | | -| BDR | 5.5.0 | Expose follower info in Raft get_raft_status() on Raft leader nodes | | -| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | -| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | RT102828/35305 | -| BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | -| CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | -| CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | -| CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | +| Component | Version | Description | Addresses | +|-----------|---------|-------------------------------------------------------------------------------------------------------------------------------|----------------| +| BDR | 5.5.0 | Improve bulk INSERT/UPDATE/DELETE performance by sending multiple messages together in a group rather than individually | | +| BDR | 5.5.0 | Don't save changes received by the writer to a temp file | | +| BDR | 5.5.0 | Log completion of an extension upgrade | | +| BDR | 5.5.0 | Add restrictions for group commit options | | +| BDR | 5.5.0 | Execute each autopartition task in its own transaction | RT101407/35476 | +| BDR | 5.5.0 | Use DETACH CONCURRENTLY to drop partitions. | RT101407/35476 | +| BDR | 5.5.0 | Disallow node group creation on a node bad state | | +| BDR | 5.5.0 | Grant additional object permissions to role "bdr_read_all_stats" | | +| BDR | 5.5.0 | Improve stability of manager worker and Raft consensus by not throwing error on non-fatal dynamic shared memory read failures | | +| BDR | 5.5.0 | Improve stability of Raft consensus and workers by handling dynamic shared memory errors in the right place | | +| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | +| BDR | 5.5.0 | Expose heatbeat timings in Raft get_raft_status() | | +| BDR | 5.5.0 | Expose follower info in Raft get_raft_status() on Raft leader nodes | | +| BDR | 5.5.0 | Expose number of changes processed by writer in a large transaction | | +| BDR | 5.5.0 | Stop the initial replication connection for bdr_init_physical and start it only when needed | RT102828/35305 | +| BDR | 5.5.0 | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser | | +| CLI | 5.5.0 | Add new read scalability related options to JSON output of `show-proxies ` and `show-groups` commands | | +| CLI | 5.5.0 | Add new option called `proxy-mode` to `create-proxy` command for read scalability support | | +| CLI | 5.5.0 | Add raft leader in tabular output of `show-groups` command | | ## Bug fixes From f160ff3735c517269164c8accb51097edf569759 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> Date: Tue, 14 May 2024 12:43:26 +0100 Subject: [PATCH 40/84] Added new issue numbers pgd_4.3.5_rel_notes.mdx --- product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx index c05ebd51aab..b6aaea4a68e 100644 --- a/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx +++ b/product_docs/docs/pgd/4/rel_notes/pgd_4.3.5_rel_notes.mdx @@ -18,9 +18,9 @@ This version is required for EDB Postgres Advanced Server versions 12.15, 13.11, | HARP | 2.4.1 | Change | Routine security library upgrades. | | | BDR | 4.3.5 | Enhancement | Log completion of an extension upgrade. | | | BDR | 4.3.5 | Enhancement | Execute each autopartition task in its own transaction. | | -| BDR | 4.3.5 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | RT101407 | +| BDR | 4.3.5 | Enhancement | Use `DETACH CONCURRENTLY` to drop partitions. | RT101407/35476 | | BDR | 4.3.5 | Enhancement | Grant usage of `pg_file_settings` and `pg_show_all_file_settings()` to bdr_superuser. | | -| BDR | 4.3.5 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | RT102828 | -| BDR | 4.3.5 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | RT99956 | -| BDR | 4.3.5 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | RT102290 | +| BDR | 4.3.5 | Enhancement | Stop the initial replication connection for `bdr_init_physical` and start it only when needed. | RT102828/35305 | +| BDR | 4.3.5 | Bug Fix | Improve handling of node group configuration parameter "check_constraints". | RT99956/31896 | +| BDR | 4.3.5 | Bug Fix | Prevent potential segfault in `bdr.monitor_group_versions()`. | RT102290/34051 | | BDR | 4.3.5 | Bug Fix | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation.| | From a1683aea64852cf39d55edb2ad82df7ccfa9d3c6 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 7 May 2024 11:14:47 +0100 Subject: [PATCH 41/84] cve20244545 added Signed-off-by: Dj Walker-Morgan --- .../security/advisories/cve20244545.mdx | 65 ++++++++++++++++ advocacy_docs/security/advisories/index.mdx | 25 ++++++ advocacy_docs/security/index.mdx | 78 ++----------------- product_docs/docs/pgd/5/reference/index.mdx | 8 +- 4 files changed, 101 insertions(+), 75 deletions(-) create mode 100644 advocacy_docs/security/advisories/cve20244545.mdx diff --git a/advocacy_docs/security/advisories/cve20244545.mdx b/advocacy_docs/security/advisories/cve20244545.mdx new file mode 100644 index 00000000000..dda91c16c32 --- /dev/null +++ b/advocacy_docs/security/advisories/cve20244545.mdx @@ -0,0 +1,65 @@ +--- +title: CVE-2024-4545 - EDB Postgres Advanced Server (EPAS) authenticated file read permissions bypass using edbldr +navTitle: CVE-2024-4545 +affectedProducts: All versions of EDB Postgres Advanced Server (EPAS) edbldr prior to 15.6.1 and 16.2.1 +--- + +First Published: 2024/05/09 + +Last Updated: 2024/05/09 + +## Summary + +All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 15.6.1 and 16.2.1 may allow users using `edbldr` to bypass role permissions from `pg_read_server_files`. This could allow low privilege users to read files to which they would not otherwise have access. + +## Vulnerability details + +CVE-ID: [CVE-2024-4545](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-4545) + +CVSS Base Score: 7.7 + +CVSS Temporal Score: Undefined + +CVSS Environmental Score: Undefined + +CVSS Vector: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:N/A:N + +## Affected products and versions + +* EnterpriseDB Postgres Advanced Server (EPAS) + * All versions prior to 15.6.1 + * All versions prior to 16.2.1 + +## Remediation + +Impacted users must upgrade to a fixed version of EPAS. For questions about updating, users can contact their account representative or [contact EDB](https://www.enterprisedb.com/contact). + +| Product | VRMF | Remediation/First Fix | +|---------|------|-----------------------| +| EPAS | All versions prior to 15.6.1 | [Upgrade EPAS 15 to Minor release](https://www.enterprisedb.com/docs/epas/15/upgrading/04_upgrading_an_installation_with_pg_upgrade/01_performing_an_upgrade/) | +| EPAS | All versions prior to 16.2.1 | [Upgrade EPAS 16 to Minor release](https://www.enterprisedb.com/docs/epas/16/upgrading/04_upgrading_an_installation_with_pg_upgrade/01_performing_an_upgrade/) | + +## References + +* [CVSS Calculator v3.1](https://www.first.org/cvss/calculator/3.1) +* [CWE-284 Improper Access Control](http://cwe.mitre.org/data/definitions/284.html) + + +## Related information + +* [EnterpriseDB](https://www.enterprisedb.com/) +* [PostgreSQL](https://www.postgresql.org/) +* [EDB Postgres Advanced Server (EPAS)](https://www.enterprisedb.com/products/edb-postgres-advanced-server) +* [EDB Blogs link](https://enterprisedb.com/blog/) + +## Acknowledgement + +None + +## Change history + +* 9 May 2024: Original document published + +## Disclaimer + +This document is provided on an "as is" basis and does not imply any kind of guarantee or warranty, including the warranties of merchantability or fitness for a particular use. Your use of the information on the document is at your own risk. EDB reserves the right to change or update this document at any time. Customers are therefore recommended to always view the latest version of this document. \ No newline at end of file diff --git a/advocacy_docs/security/advisories/index.mdx b/advocacy_docs/security/advisories/index.mdx index 0b7733b715f..8d534591473 100644 --- a/advocacy_docs/security/advisories/index.mdx +++ b/advocacy_docs/security/advisories/index.mdx @@ -6,6 +6,7 @@ iconName: Security hideKBLink: true hideToC: false navigation: +- cve20244545 - cve202341120 - cve202341119 - cve202341118 @@ -26,6 +27,30 @@ navigation: +

Updated 2024

+ + + + + + + + + +
+

CVE-2024-4545

+ +  Read Advisory +  Updated: 2024/05/09 +

EDB Postgres Advanced Server (EPAS) authenticated file read permissions bypass using edbldr

+
All versions of EDB Postgres Advanced Server (EPAS) edbldr prior to 15.6.1 and 16.2.1
+ +
+Summary:  +All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 15.6.1 and 16.2.1 may allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access. +
+Read More... +

Updated 2023

diff --git a/advocacy_docs/security/index.mdx b/advocacy_docs/security/index.mdx index 5f469e6c792..eac2c45fdb2 100644 --- a/advocacy_docs/security/index.mdx +++ b/advocacy_docs/security/index.mdx @@ -33,82 +33,18 @@ This policy outlines how EnterpriseDB handles disclosures related to suspected v - - - - - - - - - - - -
-

CVE-2023-41120

+

CVE-2024-4545

-  Read Advisory -  Updated: 2023/08/30 -

EDB Postgres Advanced Server (EPAS) DBMS_PROFILER data may be removed without permission

-
All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 11.21.32, 12.16.20, 13.12.17, 14.9.0, 15.4.0
+  Read Advisory +  Updated: 2024/05/09 +

EDB Postgres Advanced Server (EPAS) authenticated file read permissions bypass using edbldr

+
All versions of EDB Postgres Advanced Server (EPAS) edbldr prior to 15.6.1 and 16.2.1
Summary:  -An issue was discovered in EnterpriseDB Postgres Advanced Server (EPAS) before 11.21.32, 12.x before 12.16.20, 13.x before 13.12.16, 14.x before 14.9.0, and 15.x before 15.4.0. It permits an authenticated user to use DBMS_PROFILER to remove all accumulated profiling data on a system-wide basis, regardless of that user's permissions. +All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 15.6.1 and 16.2.1 may allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access.
-Read More... -
-

CVE-2023-41119

- -  Read Advisory -  Updated: 2023/08/30 -

EDB Postgres Advanced Server (EPAS) dbms_aq helper function may run arbitrary SQL as a superuser

-
All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 11.21.32, 12.16.20, 13.12.17, 14.9.0, 15.4.0
- -
-Summary:  -An issue was discovered in EnterpriseDB Postgres Advanced Server (EPAS) before 11.21.32, 12.x before 12.16.20, 13.x before 13.12.16, 14.x before 14.9.0, and 15.x before 15.4.0. It contains the function _dbms_aq_move_to_exception_queue that may be used to elevate a user's privileges to superuser. This function accepts the OID of a table, and then accesses that table as the superuser by using SELECT and DML commands. -
-Read More... -
-

CVE-2023-41118

- -  Read Advisory -  Updated: 2023/08/30 -

EDB Postgres Advanced Server (EPAS) UTL_FILE permission bypass

-
All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 11.21.32, 12.16.20, 13.12.17, 14.9.0, 15.4.0
- -
-Summary:  -An issue was discovered in EnterpriseDB Postgres Advanced Server (EPAS) before 11.21.32, 12.x before 12.16.20, 13.x before 13.12.16, 14.x before 14.9.0, and 15.x before 15.4.0. It may allow an authenticated user to bypass authorization requirements and access underlying implementation functions. When a superuser has configured file locations using CREATE DIRECTORY, these functions allow users to take a wide range of actions, including read, write, copy, rename, and delete. -
-Read More... -
-

CVE-2023-41117

- -  Read Advisory -  Updated: 2023/08/30 -

EDB Postgres Advanced Server (EPAS) SECURITY DEFINER functions and procedures may be hijacked via search_path

-
All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 11.21.32, 12.16.20, 13.12.17, 14.9.0, 15.4.0
- -
-Summary:  -An issue was discovered in EnterpriseDB Postgres Advanced Server (EPAS) before 11.21.32, 12.x before 12.16.20, 13.x before 13.12.16, 14.x before 14.9.0, and 15.x before 15.4.0. It contain packages, standalone packages, and functions that run SECURITY DEFINER but are inadequately secured against search_path attacks. -
-Read More... -
-

CVE-2023-41116

- -  Read Advisory -  Updated: 2023/08/30 -

EDB Postgres Advanced Server (EPAS) permission bypass for materialized views

-
All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 11.21.32, 12.16.20, 13.12.17, 14.9.0, 15.4.0
- -
-Summary:  -An issue was discovered in EnterpriseDB Postgres Advanced Server (EPAS) before 11.21.32, 12.x before 12.16.20, 13.x before 13.12.16, 14.x before 14.9.0, and 15.x before 15.4.0. It allows an authenticated user to refresh any materialized view, regardless of that user's permissions. -
-Read More... +Read More...
diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index ed0d54fd9fe..fbfe3357360 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -1,8 +1,8 @@ --- -title: "PGD reference" -navTitle: "PGD reference" +title: "PGD Reference" +navTitle: "PGD Reference" description: > - The complete reference to all functions, views, and commands available in EDB Postgres Distributed. + The complete reference to all functions, views and commands available in EDB Postgres Distributed. indexCards: none navigation: - catalogs-visible @@ -23,7 +23,7 @@ navigation: - functions-internal --- -The reference section is a definitive listing of all functions, views, and commands available in EDB Postgres Distributed. +The reference section is a definitive listing of all functions, views and commands available in EDB Postgres Distributed. From d7be0ca7d2dfb767e1cfccd5decfa52f865d7161 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 8 May 2024 11:11:49 +0100 Subject: [PATCH 42/84] First rel note pass - pge good, epas tbd Signed-off-by: Dj Walker-Morgan --- .../security/advisories/cve20244545.mdx | 18 ++++++++++++------ advocacy_docs/security/advisories/index.mdx | 4 ++-- advocacy_docs/security/index.mdx | 4 ++-- .../16/epas_rel_notes/epas16_3_0_rel_notes.mdx | 17 +++++++++++++++++ .../docs/pge/15/release_notes/index.mdx | 2 ++ .../pge/15/release_notes/rel_notes15.7.mdx | 16 ++++++++++++++++ .../docs/pge/16/release_notes/index.mdx | 2 ++ .../pge/16/release_notes/rel_notes16.3.mdx | 12 ++++++++++++ 8 files changed, 65 insertions(+), 10 deletions(-) create mode 100644 product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx create mode 100644 product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx create mode 100644 product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx diff --git a/advocacy_docs/security/advisories/cve20244545.mdx b/advocacy_docs/security/advisories/cve20244545.mdx index dda91c16c32..5f73256e654 100644 --- a/advocacy_docs/security/advisories/cve20244545.mdx +++ b/advocacy_docs/security/advisories/cve20244545.mdx @@ -1,7 +1,7 @@ --- title: CVE-2024-4545 - EDB Postgres Advanced Server (EPAS) authenticated file read permissions bypass using edbldr navTitle: CVE-2024-4545 -affectedProducts: All versions of EDB Postgres Advanced Server (EPAS) edbldr prior to 15.6.1 and 16.2.1 +affectedProducts: All versions of EDB Postgres Advanced Server (EPAS) edbldr from 15.0 and prior to 15.7.0 and from 16.0 and prior to 16.3.0 --- First Published: 2024/05/09 @@ -10,7 +10,7 @@ Last Updated: 2024/05/09 ## Summary -All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 15.6.1 and 16.2.1 may allow users using `edbldr` to bypass role permissions from `pg_read_server_files`. This could allow low privilege users to read files to which they would not otherwise have access. +All versions of EnterpriseDB Postgres Advanced Server (EPAS) from 15.0 and prior to 15.7.0 and from 16.0 and prior to 16.3.0 may allow users using `edbldr` to bypass role permissions from `pg_read_server_files`. This could allow low privilege users to read files to which they would not otherwise have access. ## Vulnerability details @@ -27,8 +27,8 @@ CVSS Vector: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:N/A:N ## Affected products and versions * EnterpriseDB Postgres Advanced Server (EPAS) - * All versions prior to 15.6.1 - * All versions prior to 16.2.1 + * All versions from 15.0 and prior to 15.7.0 + * All versions from 16.0 and prior to 16.3.0 ## Remediation @@ -36,8 +36,14 @@ Impacted users must upgrade to a fixed version of EPAS. For questions about upda | Product | VRMF | Remediation/First Fix | |---------|------|-----------------------| -| EPAS | All versions prior to 15.6.1 | [Upgrade EPAS 15 to Minor release](https://www.enterprisedb.com/docs/epas/15/upgrading/04_upgrading_an_installation_with_pg_upgrade/01_performing_an_upgrade/) | -| EPAS | All versions prior to 16.2.1 | [Upgrade EPAS 16 to Minor release](https://www.enterprisedb.com/docs/epas/16/upgrading/04_upgrading_an_installation_with_pg_upgrade/01_performing_an_upgrade/) | +| EPAS | All versions from 15.0 and prior to 15.7.0 | [Upgrade EPAS 15 to Minor release](https://www.enterprisedb.com/docs/epas/15/upgrading/04_upgrading_an_installation_with_pg_upgrade/01_performing_an_upgrade/) | +| EPAS | All versions from 16.0 and prior to 16.7.0 | [Upgrade EPAS 16 to Minor release](https://www.enterprisedb.com/docs/epas/16/upgrading/04_upgrading_an_installation_with_pg_upgrade/01_performing_an_upgrade/) | + +!!! Warning +If impacted users are currently relying on non-superusers to run edbldr and read data from the server filesystem without any special permissions, the fixed versions of EPAS could break these workflows. It is recommended that users do one of the following: +* Grant such users the `pg_read_server_files` role +* Change the way data is being loaded into the database, such as loading files from standard input rather than specifying a pathname. +!!! ## References diff --git a/advocacy_docs/security/advisories/index.mdx b/advocacy_docs/security/advisories/index.mdx index 8d534591473..1912a042234 100644 --- a/advocacy_docs/security/advisories/index.mdx +++ b/advocacy_docs/security/advisories/index.mdx @@ -38,11 +38,11 @@ navigation:   Read Advisory   Updated: 2024/05/09

EDB Postgres Advanced Server (EPAS) authenticated file read permissions bypass using edbldr

-
All versions of EDB Postgres Advanced Server (EPAS) edbldr prior to 15.6.1 and 16.2.1
+
All versions of EDB Postgres Advanced Server (EPAS) edbldr from 15.0 and prior to 15.7.0 and from 16.0 and prior to 16.3.0
Summary:  -All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 15.6.1 and 16.2.1 may allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access. +All versions of EnterpriseDB Postgres Advanced Server (EPAS) from 15.0 and prior to 15.7.0 and from 16.0 and prior to 16.3.0 may allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access.
Read More... diff --git a/advocacy_docs/security/index.mdx b/advocacy_docs/security/index.mdx index eac2c45fdb2..b6b148eec9e 100644 --- a/advocacy_docs/security/index.mdx +++ b/advocacy_docs/security/index.mdx @@ -38,11 +38,11 @@ This policy outlines how EnterpriseDB handles disclosures related to suspected v   Read Advisory   Updated: 2024/05/09

EDB Postgres Advanced Server (EPAS) authenticated file read permissions bypass using edbldr

-
All versions of EDB Postgres Advanced Server (EPAS) edbldr prior to 15.6.1 and 16.2.1
+
All versions of EDB Postgres Advanced Server (EPAS) edbldr from 15.0 and prior to 15.7.0 and from 16.0 and prior to 16.3.0
Summary:  -All versions of EnterpriseDB Postgres Advanced Server (EPAS) prior to 15.6.1 and 16.2.1 may allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access. +All versions of EnterpriseDB Postgres Advanced Server (EPAS) from 15.0 and prior to 15.7.0 and from 16.0 and prior to 16.3.0 may allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access.
Read More... diff --git a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx new file mode 100644 index 00000000000..ee006e4afc9 --- /dev/null +++ b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx @@ -0,0 +1,17 @@ +--- +title: EDB Postgres Advanced Server 16.3 release notes +navTitle: "Version 16.3" +--- + +Released: 9 May 2024 + +EDB Postgres Advanced Server 16.3 includes the following enhancements and bug fixes: + +| Type | Description | Category | +|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| +| Upstream merge | Merged with community PostgreSQL 16.3. Addresses CVE-2024-4317. See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. | +| Security |edbldr: check pg_read_server_files privilege before data file access. (#35906) +Permission to read data from the server File System should be restricted to superusers or users who possess the pg_read_server_files role. However, in affected versions of EPAS, any user can use edbldr to read data from any server file which is accessible to the OS user account under which EPAS is running. With this fix, a non-superuser using edbldr must either possess the pg_read_server_files role or must load data from standard input, rather than the server filesystem. Customer Advisory: CVE-2024-4545-for-db-2681 | +| Bug fix | +Fix assertion in DROP ROLE statement having duplicate names. | Bug | +| Security fix | Fixed a security vulnerability that could allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access. This issue is tracked as [CVE-2024-4545](linktoadvisory). | Security | \ No newline at end of file diff --git a/product_docs/docs/pge/15/release_notes/index.mdx b/product_docs/docs/pge/15/release_notes/index.mdx index 787e1f8320c..46a3acf4941 100644 --- a/product_docs/docs/pge/15/release_notes/index.mdx +++ b/product_docs/docs/pge/15/release_notes/index.mdx @@ -1,6 +1,7 @@ --- title: "Release notes" navigation: + - rel_notes15.7 - rel_notes15.6 - rel_notes15.5 - rel_notes15.4 @@ -13,6 +14,7 @@ release notes cover what was new in each release. | Version | Release date | | ------------------------ | ------------ | +| [15.7](rel_notes15.7) | 09 May 2024 | | [15.6](rel_notes15.6) | 08 Feb 2024 | | [15.5](rel_notes15.5) | 09 Nov 2023 | | [15.4](rel_notes15.4) | 21 Aug 2023 | diff --git a/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx b/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx new file mode 100644 index 00000000000..23c767dc7f3 --- /dev/null +++ b/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx @@ -0,0 +1,16 @@ +--- +title: "EDB Postgres Extended Server 15.7 release notes" +navTitle: Version 15.7 +--- + +Released: 9 May 2024 + +New features, enhancements, bug fixes, and other changes in EDB Postgres Extended Server 15.7 include: + +| Type | Description | +| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Upstream merge | Merged with community PostgreSQL 15.7. Addresses CVE-2024-4137. See the [PostgreSQL 15 Release Notes](https://www.postgresql.org/docs/15/release-15-6.html) for more information. | + + + + diff --git a/product_docs/docs/pge/16/release_notes/index.mdx b/product_docs/docs/pge/16/release_notes/index.mdx index bb7d8c875b8..7194a5d1b1e 100644 --- a/product_docs/docs/pge/16/release_notes/index.mdx +++ b/product_docs/docs/pge/16/release_notes/index.mdx @@ -1,6 +1,7 @@ --- title: "Release notes" navigation: + - rel_notes16.3 - rel_notes16.2 - rel_notes16.1 --- @@ -10,6 +11,7 @@ cover what was new in each release. | Version | Release date | | ------------------------ | ------------ | +| [16.3](rel_notes16.3) | 09 May 2024 | | [16.2](rel_notes16.2) | 08 Feb 2024 | | [16.1](rel_notes16.1) | 09 Nov 2023 | diff --git a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx new file mode 100644 index 00000000000..37504405e28 --- /dev/null +++ b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx @@ -0,0 +1,12 @@ +--- +title: EDB Postgres Advanced Server 16.3 release notes +navTitle: "Version 16.3" +--- + +Released: 9 May 2024 + +EDB Postgres Extended Server 16.3 includes the following enhancements and bug fixes: + +| Type | Description +| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +| Upstream merge | Merged with community PostgreSQL 16.3. Addresses CVE-2024-4317. See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. From de39609af7cd6e12c6ec8f5608103cca7e3c7071 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 8 May 2024 15:00:25 +0100 Subject: [PATCH 43/84] EPAS/PGE Rel Notes and advisory/assessment added Signed-off-by: Dj Walker-Morgan --- .../security/assessments/cve-2024-4317.mdx | 96 +++++++++++++++++++ advocacy_docs/security/assessments/index.mdx | 20 ++++ advocacy_docs/security/index.mdx | 22 ++++- .../security/templates/securityindex.njs | 4 +- .../epas_rel_notes/epas12_15_19_rel_notes.mdx | 24 ++--- .../epas_rel_notes/epas12_19_24_rel_notes.mdx | 17 ++++ .../docs/epas/12/epas_rel_notes/index.mdx | 2 + .../epas_rel_notes/epas13_15_21_rel_notes.mdx | 17 ++++ .../docs/epas/13/epas_rel_notes/index.mdx | 6 +- .../epas_rel_notes/epas14_12_0_rel_notes.mdx | 17 ++++ .../docs/epas/14/epas_rel_notes/index.mdx | 10 +- .../epas_rel_notes/epas15_7_0_rel_notes.mdx | 19 ++++ .../docs/epas/15/epas_rel_notes/index.mdx | 8 +- .../epas_rel_notes/epas16_3_0_rel_notes.mdx | 26 ++--- .../docs/epas/16/epas_rel_notes/index.mdx | 4 +- .../pge/15/release_notes/rel_notes15.7.mdx | 2 +- .../pge/16/release_notes/rel_notes16.3.mdx | 2 +- 17 files changed, 256 insertions(+), 40 deletions(-) create mode 100644 advocacy_docs/security/assessments/cve-2024-4317.mdx create mode 100644 product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx create mode 100644 product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx create mode 100644 product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx create mode 100644 product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx diff --git a/advocacy_docs/security/assessments/cve-2024-4317.mdx b/advocacy_docs/security/assessments/cve-2024-4317.mdx new file mode 100644 index 00000000000..f1441fcdbc4 --- /dev/null +++ b/advocacy_docs/security/assessments/cve-2024-4317.mdx @@ -0,0 +1,96 @@ +--- +title: CVE-2024-4317 - TBD +navTitle: CVE-2024-4317 +affectedProducts: TBD +--- + +First Published: 2024/05/09 + +Last Updated: 2024/05/09 + +Important: This is an assessment of the impact of CVE-2024-4317 on EDB products and services. It links to and details the CVE and supplements that information with EDB's own assessment. + +## Summary + +TBC + +## Vulnerability details + +CVE-ID: [CVE-2024-4317](https://nvd.nist.gov/vuln/detail/CVE-2024-4317) + +CVSS Base Score: TBC + +CVSS Temporal Score: TBC + +CVSS Environmental Score: TBC + +CVSS Vector: TBC + +## Affected products and versions + +### PostgreSQL + +TBC + +### EnterpriseDB Postgres Advanced Server (EPAS) +TBC + +### EnterpriseDB Postgres Extended +TBC + +## Remediation/fixes + +### PostgreSQL Version Information + +| Affected Version | Fixed In | Fix Published | +|-------------------|----------|---------------| +| 15 | TBC | 2024-05-09 | +| 14 | TBC | 2024-05-09 | +| 13 | TBC | 2024-05-09 | +| 12 | TBC | 2024-05-09 | + +### EPAS Version Information + +TBC + +| Product | VRMF | Remediation/First Fix | +|---------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------| +| EPAS | TBC | TBC | +### PGE Version Information + +| Product | VRMF | Remediation/First Fix | +|---------|--------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| +| PGE | TBC | TBC | + +!!! Note +The exploit referred to in this CVE did not work on PostgreSQL 16. The +same defensive code as other releases has been added in PostgreSQL 16.2, EPAS +16.2 and PGE 16.2 to ensure strength in depth. We strongly recommend upgrading +your PostgreSQL 16, EPAS 16 and PGE 16 deployments to these versions. +!!! + +## References + +* [CVSS Calculator v3.1](https://www.first.org/cvss/calculator/3.1) + + +## Related information + +* [EnterpriseDB](https://www.enterprisedb.com/) +* [EDB Blogs link](https://enterprisedb.com/blog/) + +## Acknowledgement + +Source: PostgreSQL.org + +## Change history + +## Disclaimer + + +This document is provided on an "as is" basis and does not imply any kind of +guarantee or warranty, including the warranties of merchantability or fitness +for a particular use. Your use of the information on the document is at your own +risk. EDB reserves the right to change or update this document at any time. +Customers are therefore recommended to always view the latest version of this +document. diff --git a/advocacy_docs/security/assessments/index.mdx b/advocacy_docs/security/assessments/index.mdx index cab7c7ecb97..16450c0a7b7 100644 --- a/advocacy_docs/security/assessments/index.mdx +++ b/advocacy_docs/security/assessments/index.mdx @@ -6,6 +6,7 @@ iconName: Security hideKBLink: true hideToC: false navigation: +- cve-2024-4317 - cve-2024-1597 - cve-2024-0985 --- @@ -25,6 +26,25 @@ The CVEs listed in this section are from PostgreSQL and other parties who have r + + + + + +
+

CVE-2024-4317

+ +  Read Assessment +  Updated: 2024/05/09 +

TBD

+
TBD
+ +
+Summary:  +TBC +
+Read More... +

CVE-2024-1597

diff --git a/advocacy_docs/security/index.mdx b/advocacy_docs/security/index.mdx index b6b148eec9e..9b6851d16f5 100644 --- a/advocacy_docs/security/index.mdx +++ b/advocacy_docs/security/index.mdx @@ -49,15 +49,31 @@ All versions of EnterpriseDB Postgres Advanced Server (EPAS) from 15.0 and prior
-## Most Recent Assesments +## Most Recent Assessments + + +
+

CVE-2024-4317

+ +  Read Assessment +  Updated: 2024/05/09 +

TBD

+
TBD
+ +
+Summary:  +TBC +
+Read More... +

CVE-2024-1597

-  Read Assesment +  Read Assessment   Updated: 2024/03/08

SQL Injection via line comment generation

pgJDBC all versions prior to 42.7.2, 42.6.1, 42.5.5, 42.4.4, 42.3.9, and 42.2.28 and EDB pgJDBC all versions prior to 42.5.5
@@ -73,7 +89,7 @@ pgjdbc, the PostgreSQL JDBC Driver, allows attacker to inject SQL if using Prefe

CVE-2024-0985

-  Read Assesment +  Read Assessment   Updated: 2024/02/26

PostgreSQL non-owner REFRESH MATERIALIZED VIEW CONCURRENTLY executes arbitrary SQL

PostgreSQL, EPAS all versions prior to 15.6.0,14.11.0,13.14.20 and 12.18.23, PGE all versions prior to 15.6.0
diff --git a/advocacy_docs/security/templates/securityindex.njs b/advocacy_docs/security/templates/securityindex.njs index 3f39d08581f..45acc33522c 100755 --- a/advocacy_docs/security/templates/securityindex.njs +++ b/advocacy_docs/security/templates/securityindex.njs @@ -49,7 +49,7 @@ This policy outlines how EnterpriseDB handles disclosures related to suspected v {% endfor %}
-## Most Recent Assesments +## Most Recent Assessments {% for ass in shortasslist %} @@ -57,7 +57,7 @@ This policy outlines how EnterpriseDB handles disclosures related to suspected v diff --git a/advocacy_docs/security/index.mdx b/advocacy_docs/security/index.mdx index 9b6851d16f5..b198d05e9e7 100644 --- a/advocacy_docs/security/index.mdx +++ b/advocacy_docs/security/index.mdx @@ -59,12 +59,12 @@ All versions of EnterpriseDB Postgres Advanced Server (EPAS) from 15.0 and prior   Read Assessment   Updated: 2024/05/09 -

TBD

+

Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner

TBD
Summary:  -TBC +Missing authorization in PostgreSQL built-in views pg_stats_ext and pg_stats_ext_exprs allows an unprivileged database user to read most common values and other statistics from CREATE STATISTICS commands of other users. The most common values may reveal column values the eavesdropper could not otherwise read or results of functions they cannot execute. Installing an unaffected version only fixes fresh PostgreSQL installations, namely those that are created with the initdb utility after installing that version. Current PostgreSQL installations will remain vulnerable until they follow the instructions in the release notes, which are provided as a convenience in the below section. Within major versions 14-16, minor versions before PostgreSQL 16.3, 15.7, and 14.12 are affected. Versions before PostgreSQL 14 are unaffected.
Read More... diff --git a/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx b/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx index c48e0cebb7c..e2706ce7de1 100644 --- a/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx +++ b/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 12.19.24 includes the following enhancements and bu | Type | Description | Addresses            | |----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| -| Upstream merge | Merged with community PostgreSQL 12.19. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 12.19 Release Notes](https://www.postgresql.org/docs/release/12.19/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | +| Upstream merge | Merged with community PostgreSQL 12.19. See the [PostgreSQL 12.19 Release Notes](https://www.postgresql.org/docs/release/12.19/) for more information. | | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | | Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | diff --git a/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx b/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx index 64a8db33c42..44031b5efec 100644 --- a/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx +++ b/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 13.15.21 includes the following enhancements and bu | Type | Description | Addresses                | |----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 13.15. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 13.15 Release Notes](https://www.postgresql.org/docs/release/14.15/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | +| Upstream merge | Merged with community PostgreSQL 13.15. See the [PostgreSQL 13.15 Release Notes](https://www.postgresql.org/docs/release/14.15/) for more information. | | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | | Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | diff --git a/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx b/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx index 6017ec579d0..8b09fb72acc 100644 --- a/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx +++ b/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 14.12.0 includes the following enhancements and bug | Type | Description | Addresses                | |----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 14.12. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 14.12 Release Notes](https://www.postgresql.org/docs/release/14.12/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | +| Upstream merge | Merged with community PostgreSQL 14.12. This release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 14.12 Release Notes](https://www.postgresql.org/docs/release/14.12/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | | Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | diff --git a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx index 9e354230428..e10641507f9 100644 --- a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx +++ b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 15.7.0 includes the following enhancements and bug | Type | Description | Addresses                | |-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 15.7. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15.7 Release Notes](https://www.postgresql.org/docs/release/15.7/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317)| +| Upstream merge | Merged with community PostgreSQL 15.7. This release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15.7 Release Notes](https://www.postgresql.org/docs/release/15.7/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317)| | Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | diff --git a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx index a84b5607245..eb6b9caaf4c 100644 --- a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx +++ b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 16.3.0 includes the following enhancements and bug | Type | Description | Addresses                | |-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 16.3. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/release/16.3/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | +| Upstream merge | Merged with community PostgreSQL 16.3. This release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/release/16.3/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | | Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | diff --git a/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx b/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx index ce6b83e9182..cf94e96f896 100644 --- a/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx +++ b/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx @@ -9,7 +9,7 @@ New features, enhancements, bug fixes, and other changes in EDB Postgres Extende | Type | Description | | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Upstream merge | Merged with community PostgreSQL 15.7. Includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15 Release Notes](https://www.postgresql.org/docs/15/release-15-6.html) for more information. | +| Upstream merge | Merged with community PostgreSQL 15.7. Includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15 Release Notes](https://www.postgresql.org/docs/15/release-15-7.html) for more information. | diff --git a/product_docs/docs/pge/16/release_notes/rel_notes16.2.mdx b/product_docs/docs/pge/16/release_notes/rel_notes16.2.mdx index 395e8efcf90..6709c591584 100644 --- a/product_docs/docs/pge/16/release_notes/rel_notes16.2.mdx +++ b/product_docs/docs/pge/16/release_notes/rel_notes16.2.mdx @@ -1,5 +1,5 @@ --- -title: EDB Postgres Advanced Server 16.2 release notes +title: EDB Postgres Extended Server 16.2 release notes navTitle: "Version 16.2" --- diff --git a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx index eb1f6406528..11d3b3fcffc 100644 --- a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx +++ b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx @@ -1,5 +1,5 @@ --- -title: EDB Postgres Advanced Server 16.3 release notes +title: EDB Postgres Extended Server 16.3 release notes navTitle: "Version 16.3" --- From e07e7f3a2e71fe0a966a139e04dc13b5f2fd898b Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Wed, 8 May 2024 11:11:49 +0100 Subject: [PATCH 46/84] First rel note pass - pge good, epas tbd Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/reference/index.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index fbfe3357360..ed0d54fd9fe 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -1,8 +1,8 @@ --- -title: "PGD Reference" -navTitle: "PGD Reference" +title: "PGD reference" +navTitle: "PGD reference" description: > - The complete reference to all functions, views and commands available in EDB Postgres Distributed. + The complete reference to all functions, views, and commands available in EDB Postgres Distributed. indexCards: none navigation: - catalogs-visible @@ -23,7 +23,7 @@ navigation: - functions-internal --- -The reference section is a definitive listing of all functions, views and commands available in EDB Postgres Distributed. +The reference section is a definitive listing of all functions, views, and commands available in EDB Postgres Distributed. From 4e0b98849d59faa66b08aa2beab8bd9f4204dd0b Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 12:40:41 +0100 Subject: [PATCH 47/84] Cleaned notes Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx index 11d3b3fcffc..024924237a2 100644 --- a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx +++ b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx @@ -9,4 +9,4 @@ EDB Postgres Extended Server 16.3 includes the following enhancements and bug fi | Type | Description | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -| Upstream merge | Merged with community PostgreSQL 16.3. Includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. +| Upstream merge | Merged with community PostgreSQL 16.3. Includes a security fix. See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. From ba9350c8384f199632d7d4fd79a0699904e13b86 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 13:22:52 +0100 Subject: [PATCH 48/84] Remove template Signed-off-by: Dj Walker-Morgan --- .../security/assessments/cve-2024-4317.mdx | 105 ------------------ advocacy_docs/security/assessments/index.mdx | 1 - 2 files changed, 106 deletions(-) delete mode 100644 advocacy_docs/security/assessments/cve-2024-4317.mdx diff --git a/advocacy_docs/security/assessments/cve-2024-4317.mdx b/advocacy_docs/security/assessments/cve-2024-4317.mdx deleted file mode 100644 index d553264aa94..00000000000 --- a/advocacy_docs/security/assessments/cve-2024-4317.mdx +++ /dev/null @@ -1,105 +0,0 @@ ---- -title: CVE-2024-4317 - Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner -navTitle: CVE-2024-4317 -affectedProducts: TBD ---- - -First Published: 2024/05/09 - -Last Updated: 2024/05/09 - -Important: This is an assessment of the impact of CVE-2024-4317 on EDB products and services. It links to and details the CVE and supplements that information with EDB's own assessment. - -## Summary - -Missing authorization in PostgreSQL built-in views pg_stats_ext and pg_stats_ext_exprs allows an unprivileged database user to read most common values and other statistics from CREATE STATISTICS commands of other users. The most common values may reveal column values the eavesdropper could not otherwise read or results of functions they cannot execute. Installing an unaffected version only fixes fresh PostgreSQL installations, namely those that are created with the initdb utility after installing that version. Current PostgreSQL installations will remain vulnerable until they follow the instructions in the release notes, which are provided as a convenience in the below section. Within major versions 14-16, minor versions before PostgreSQL 16.3, 15.7, and 14.12 are affected. Versions before PostgreSQL 14 are unaffected. - - -## Vulnerability details - -CVE-ID: [CVE-2024-4317](https://www.postgresql.org/support/security/CVE-2024-4317/) - -CVSS Base Score: 3.1 - -CVSS Temporal Score: Undefined - -CVSS Environmental Score: Undefined - -CVSS Vector: TBC - -## Affected products and versions - -### PostgreSQL - -* All versions of PostgreSQL prior to 16.3 -* All versions of PostgreSQL prior to 15.7 -* All versions of PostgreSQL prior to 14.12 - -### EnterpriseDB Postgres Advanced Server (EPAS) - -* All versions of EPAS prior to 16.3 -* All versions of EPAS prior to 15.7 -* All versions of EPAS prior to 14.12 - -### EnterpriseDB Postgres Extended - -* All versions of PGE prior to 16.3 -* All versions of PGE prior to 15.7 -* All versions of PGE prior to 14.12 - -## Remediation/fixes - -The fix is included in the following versions: 16.3, 15.7, and 14.12. - -Installing the fix will not remove the vulnerability from existing installations. To remove the vulnerability, follow the instructions in the [CVE-2024-4317](https://www.postgresql.org/support/security/CVE-2024-4317/) advisory. - -### PostgreSQL Version Information - -| Affected Version | Fixed In | Fix Published | -|-----------------------------|----------|---------------| -| All versions prior to 16.3 | 16.3 | 2024-05-09 | -| All versions prior to 15.7 | 15.7 | 2024-05-09 | -| All versions prior to 14.12 | 14.12 | 2024-05-09 | - -### EPAS Version Information - -| Product | VRMF | Remediation/First Fix | -|---------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------| -| EPAS | All versions prior to 16.3 | Update to version 16.3 or later | -| EPAS | All versions prior to 15.7 | Update to version 15.7 or later | -| EPAS | All versions prior to 14.12 | Update to version 14.12 or later | - -### PGE Version Information - -| Product | VRMF | Remediation/First Fix | -|---------|--------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| -| PGE | All versions prior to 16.3 | Update to version 16.3 or later | -| PGE | All versions prior to 15.7 | Update to version 15.7 or later | -| PGE | All versions prior to 14.12 | Update to version 14.12 or later | - -## References - -* [CVSS Calculator v3.1](https://www.first.org/cvss/calculator/3.1) -* [CWE-284 Improper Access Control](http://cwe.mitre.org/data/definitions/284.html) - -## Related information - -* [EnterpriseDB](https://www.enterprisedb.com/) -* [EDB Blogs link](https://enterprisedb.com/blog/) - -## Acknowledgement - -Source: PostgreSQL.org - -## Change history - -9 May 2024: Original Copy Published - -## Disclaimer - -This document is provided on an "as is" basis and does not imply any kind of -guarantee or warranty, including the warranties of merchantability or fitness -for a particular use. Your use of the information on the document is at your own -risk. EDB reserves the right to change or update this document at any time. -Customers are therefore recommended to always view the latest version of this -document. diff --git a/advocacy_docs/security/assessments/index.mdx b/advocacy_docs/security/assessments/index.mdx index 9202d30d36e..9042a86180f 100644 --- a/advocacy_docs/security/assessments/index.mdx +++ b/advocacy_docs/security/assessments/index.mdx @@ -6,7 +6,6 @@ iconName: Security hideKBLink: true hideToC: false navigation: -- cve-2024-4317 - cve-2024-1597 - cve-2024-0985 --- From ecf30036fa9e884e918d52b8d9da9ca403a25681 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 18:02:46 +0100 Subject: [PATCH 49/84] Snagging fixes for security links Signed-off-by: Dj Walker-Morgan --- .../security/assessments/cve-2024-4317.mdx | 105 ++++++++++++++++++ advocacy_docs/security/assessments/index.mdx | 2 +- advocacy_docs/security/index.mdx | 2 +- .../epas_rel_notes/epas15_7_0_rel_notes.mdx | 2 +- .../epas_rel_notes/epas16_3_0_rel_notes.mdx | 2 +- 5 files changed, 109 insertions(+), 4 deletions(-) create mode 100644 advocacy_docs/security/assessments/cve-2024-4317.mdx diff --git a/advocacy_docs/security/assessments/cve-2024-4317.mdx b/advocacy_docs/security/assessments/cve-2024-4317.mdx new file mode 100644 index 00000000000..80fb9465a8b --- /dev/null +++ b/advocacy_docs/security/assessments/cve-2024-4317.mdx @@ -0,0 +1,105 @@ +--- +title: CVE-2024-4317 - Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner +navTitle: CVE-2024-4317 +affectedProducts: All versions of PostgreSQL, EPAS and PGE prior to 16.3, 15.7, and 14.12 +--- + +First Published: 2024/05/09 + +Last Updated: 2024/05/09 + +Important: This is an assessment of the impact of CVE-2024-4317 on EDB products and services. It links to and details the CVE and supplements that information with EDB's own assessment. + +## Summary + +Missing authorization in PostgreSQL built-in views pg_stats_ext and pg_stats_ext_exprs allows an unprivileged database user to read most common values and other statistics from CREATE STATISTICS commands of other users. The most common values may reveal column values the eavesdropper could not otherwise read or results of functions they cannot execute. Installing an unaffected version only fixes fresh PostgreSQL installations, namely those that are created with the initdb utility after installing that version. Current PostgreSQL installations will remain vulnerable until they follow the instructions in the release notes, which are provided as a convenience in the below section. Within major versions 14-16, minor versions before PostgreSQL 16.3, 15.7, and 14.12 are affected. Versions before PostgreSQL 14 are unaffected. + + +## Vulnerability details + +CVE-ID: [CVE-2024-4317](https://www.postgresql.org/support/security/CVE-2024-4317/) + +CVSS Base Score: 3.1 + +CVSS Temporal Score: Undefined + +CVSS Environmental Score: Undefined + +CVSS Vector: [AV:N/AC:H/PR:L/UI:N/S:U/C:L/I:N/A:N](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:H/PR:L/UI:N/S:U/C:L/I:N/A:N&version=3.0) + +## Affected products and versions + +### PostgreSQL + +* All versions of PostgreSQL prior to 16.3 +* All versions of PostgreSQL prior to 15.7 +* All versions of PostgreSQL prior to 14.12 + +### EnterpriseDB Postgres Advanced Server (EPAS) + +* All versions of EPAS prior to 16.3 +* All versions of EPAS prior to 15.7 +* All versions of EPAS prior to 14.12 + +### EnterpriseDB Postgres Extended + +* All versions of PGE prior to 16.3 +* All versions of PGE prior to 15.7 +* All versions of PGE prior to 14.12 + +## Remediation/fixes + +The fix is included in the following versions: 16.3, 15.7, and 14.12. + +Installing the fix will not remove the vulnerability from existing installations. To remove the vulnerability, follow the instructions in the [CVE-2024-4317](https://www.postgresql.org/support/security/CVE-2024-4317/) advisory. + +### PostgreSQL Version Information + +| Affected Version | Fixed In | Fix Published | +|-----------------------------|----------|---------------| +| All versions prior to 16.3 | 16.3 | 2024-05-09 | +| All versions prior to 15.7 | 15.7 | 2024-05-09 | +| All versions prior to 14.12 | 14.12 | 2024-05-09 | + +### EPAS Version Information + +| Product | VRMF | Remediation/First Fix | +|---------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------| +| EPAS | All versions prior to 16.3 | Update to version 16.3 or later | +| EPAS | All versions prior to 15.7 | Update to version 15.7 or later | +| EPAS | All versions prior to 14.12 | Update to version 14.12 or later | + +### PGE Version Information + +| Product | VRMF | Remediation/First Fix | +|---------|--------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| +| PGE | All versions prior to 16.3 | Update to version 16.3 or later | +| PGE | All versions prior to 15.7 | Update to version 15.7 or later | +| PGE | All versions prior to 14.12 | Update to version 14.12 or later | + +## References + +* [CVSS Calculator v3.1](https://www.first.org/cvss/calculator/3.1) +* [CWE-284 Improper Access Control](http://cwe.mitre.org/data/definitions/284.html) + +## Related information + +* [EnterpriseDB](https://www.enterprisedb.com/) +* [EDB Blogs link](https://enterprisedb.com/blog/) + +## Acknowledgement + +Source: PostgreSQL.org + +## Change history + +9 May 2024: Original Copy Published + +## Disclaimer + +This document is provided on an "as is" basis and does not imply any kind of +guarantee or warranty, including the warranties of merchantability or fitness +for a particular use. Your use of the information on the document is at your own +risk. EDB reserves the right to change or update this document at any time. +Customers are therefore recommended to always view the latest version of this +document. diff --git a/advocacy_docs/security/assessments/index.mdx b/advocacy_docs/security/assessments/index.mdx index 9042a86180f..2586a8c30f1 100644 --- a/advocacy_docs/security/assessments/index.mdx +++ b/advocacy_docs/security/assessments/index.mdx @@ -31,7 +31,7 @@ The CVEs listed in this section are from PostgreSQL and other parties who have r   Read Assessment   Updated: 2024/05/09

Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner

-
TBD
+
All versions of PostgreSQL, EPAS and PGE prior to 16.3, 15.7, and 14.12
Summary:  diff --git a/advocacy_docs/security/index.mdx b/advocacy_docs/security/index.mdx index b198d05e9e7..97b759e447f 100644 --- a/advocacy_docs/security/index.mdx +++ b/advocacy_docs/security/index.mdx @@ -60,7 +60,7 @@ All versions of EnterpriseDB Postgres Advanced Server (EPAS) from 15.0 and prior   Read Assessment   Updated: 2024/05/09

Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner

-
TBD
+
All versions of PostgreSQL, EPAS and PGE prior to 16.3, 15.7, and 14.12
Summary:  diff --git a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx index e10641507f9..13a411b0f0a 100644 --- a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx +++ b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx @@ -10,7 +10,7 @@ EDB Postgres Advanced Server 15.7.0 includes the following enhancements and bug | Type | Description | Addresses                | |-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| | Upstream merge | Merged with community PostgreSQL 15.7. This release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15.7 Release Notes](https://www.postgresql.org/docs/release/15.7/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317)| -| Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | +| Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve20244545/) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | | Bug fix | Fixed an issue to fetch all the attributes correctly from the sublink in `CONNECT BY` processing to avoid the server crash. | #102746 | diff --git a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx index eb6b9caaf4c..df83316226c 100644 --- a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx +++ b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx @@ -10,7 +10,7 @@ EDB Postgres Advanced Server 16.3.0 includes the following enhancements and bug | Type | Description | Addresses                | |-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| | Upstream merge | Merged with community PostgreSQL 16.3. This release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/release/16.3/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | -| Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | +| Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve20244545/) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | | Bug fix | Fixed an issue to fetch all the attributes correctly from the sublink in `CONNECT BY` processing to avoid the server crash. | #102746 | From 24636ec123c23098b4a69417ae1573c7badd4389 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 13:05:37 +0100 Subject: [PATCH 50/84] Add missing release note Signed-off-by: Dj Walker-Morgan --- .../pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx index 36ba4826089..2c06da404f5 100644 --- a/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx +++ b/product_docs/docs/pgd/5/rel_notes/pgd_5.5.0_rel_notes.mdx @@ -67,19 +67,20 @@ Postgres Distributed. ## Bug fixes -| Component | Version | Description | Addresses | -|-----------|---------|---------------------------------------------------------------------------------------------------------|----------------| -| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956/31896 | -| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290/34051 | -| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | -| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | -| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966/35447 | -| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | -| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | -| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | -| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | -| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | -| BDR | 5.5.0 | Fixed a bug preventing some trusted extension management commands (CREATE/ALTER) from being replicated. | | +| Component | Version | Description | Addresses | +|-----------|---------|---------------------------------------------------------------------------------------------------------------------------|----------------| +| BDR | 5.5.0 | Improve handling of node group configuration parameter "check_constraints" | RT99956/31896 | +| BDR | 5.5.0 | Fix incorrect parsing of pre-commit message that caused nodes to diverge on commit decision for group commit transactions | | +| BDR | 5.5.0 | Prevent potential segfault in bdr.monitor_group_versions() | RT102290/34051 | +| BDR | 5.5.0 | Correctly elect a new leader when the current leader gets route_writes turned off | | +| BDR | 5.5.0 | Ensure bdr.remove_commit_scope() handles non-existent commit scope | | +| BDR | 5.5.0 | Prevent unexpected writer terminations with improved queue flush process | RT98966/35447 | +| BDR | 5.5.0 | Fix multi-row conflict accidentally deleting the wrong tuple multiple times | | +| BDR | 5.5.0 | Fixed receiver to send status update when writer is blocked, avoiding slot disconnect. | | +| BDR | 5.5.0 | Fix minor memory leak during bdr_join_node_group_sql | | +| BDR | 5.5.0 | Disallow using witness and standby nodes as source nodes for node joins | | +| BDR | 5.5.0 | Use bdr.default_sequence_kind when updating sequence kind of existing sequences upon node creation. | | +| BDR | 5.5.0 | Fixed a bug preventing some trusted extension management commands (CREATE/ALTER) from being replicated. | | From 8ca76600471c2d5622ac8f98f889aa9df7b1d9c6 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Tue, 14 May 2024 17:40:34 +0100 Subject: [PATCH 51/84] Update known issues and bdr.add_commit_scope reference Signed-off-by: Dj Walker-Morgan --- product_docs/docs/pgd/5/known_issues.mdx | 10 +++++++++- product_docs/docs/pgd/5/reference/functions.mdx | 8 ++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/known_issues.mdx b/product_docs/docs/pgd/5/known_issues.mdx index 69e112e20c6..c9c75df33b8 100644 --- a/product_docs/docs/pgd/5/known_issues.mdx +++ b/product_docs/docs/pgd/5/known_issues.mdx @@ -67,4 +67,12 @@ release. settings to ensure only active nodes in the cluster are listed for connection. -Details of other design or implementation [limitations](limitations) are also available. +- When using + [`bdr.add_commit_scope`](/pgd/latest/reference/functions#bdradd_commit_scope) + if a new commit scope is added which has the same name as a commit scope on + any group, then the commit scope silently overwrites the commit scope but + retains the original group the scope was associated with (if any). To modify + a commit scope safely, use + [`bdr.alter_commit_scope`](/pgd/latest/reference/functions#bdralter_commit_scope). + +Details of other design or implementation [limitations](planning/limitations) are also available. diff --git a/product_docs/docs/pgd/5/reference/functions.mdx b/product_docs/docs/pgd/5/reference/functions.mdx index 456ebe3d4d6..c0e7bbd7e74 100644 --- a/product_docs/docs/pgd/5/reference/functions.mdx +++ b/product_docs/docs/pgd/5/reference/functions.mdx @@ -1141,6 +1141,14 @@ bdr.alter_commit_scope( rule TEXT) ``` +#### Note + +When using `bdr.add_commit_scope` if a new commit scope is added which has the +same name as a commit scope on any group, then the commit scope silently +overwrites the commit scope but retains the original group the scope was +associated with (if any). To modify a commit scope safely, use +[`bdr.alter_commit_scope`](#bdralter_commit_scope). + ### `bdr.remove_commit_scope` Drops a single rule in a commit scope. If you define multiple rules for the commit scope, you must invoke this function once per rule to fully remove the entire commit scope. From 0a9f02f0bcd9f3a2b2e640178b692463861f28de Mon Sep 17 00:00:00 2001 From: Betsy Gitelman Date: Tue, 14 May 2024 14:15:55 -0400 Subject: [PATCH 52/84] Edits to PGD 5.5 content --- .../docs/pgd/5/cli/command_ref/index.mdx | 4 +- product_docs/docs/pgd/5/cli/index.mdx | 12 ++--- .../docs/pgd/5/cli/installing/linux.mdx | 2 +- .../docs/pgd/5/cli/installing/macos.mdx | 4 +- .../pgd/5/quickstart/quick_start_docker.mdx | 6 +-- .../pgd/5/reference/catalogs-internal.mdx | 22 ++++---- .../reference/nodes-management-interfaces.mdx | 37 ++++++------- product_docs/docs/pgd/5/reference/routing.mdx | 22 ++++---- .../docs/pgd/5/routing/administering.mdx | 13 ++--- .../docs/pgd/5/routing/configuration.mdx | 24 ++++----- product_docs/docs/pgd/5/routing/index.mdx | 5 +- .../docs/pgd/5/routing/monitoring.mdx | 12 ++--- product_docs/docs/pgd/5/routing/proxy.mdx | 53 +++++++++---------- product_docs/docs/pgd/5/routing/readonly.mdx | 45 ++++++++-------- 14 files changed, 126 insertions(+), 135 deletions(-) diff --git a/product_docs/docs/pgd/5/cli/command_ref/index.mdx b/product_docs/docs/pgd/5/cli/command_ref/index.mdx index 6668a1e0d64..bf6dec3966b 100644 --- a/product_docs/docs/pgd/5/cli/command_ref/index.mdx +++ b/product_docs/docs/pgd/5/cli/command_ref/index.mdx @@ -1,10 +1,10 @@ --- title: Command reference navTitle: Command reference -description: A reference guide to the commands available in the (PGD CLI). +description: A reference guide to the commands available in the PGD CLI. --- -pgd is the command name for the PGD command line interface. +The command name for the PGD command line interface is `pgd`. ## Synopsis diff --git a/product_docs/docs/pgd/5/cli/index.mdx b/product_docs/docs/pgd/5/cli/index.mdx index 61ed5b10ad5..f95b828e8b7 100644 --- a/product_docs/docs/pgd/5/cli/index.mdx +++ b/product_docs/docs/pgd/5/cli/index.mdx @@ -13,16 +13,16 @@ directoryDefaults: description: "The PGD Command Line Interface (CLI) is a tool to manage your EDB Postgres Distributed cluster" --- -The EDB Postgres Distributed Command Line Interface (PGD CLI) is a tool for managing your EDB Postgres Distributed cluster. It is the key tool for inspecting and managing cluster resources. +The EDB Postgres Distributed Command Line Interface (PGD CLI) is a tool for managing your EDB Postgres Distributed cluster. It's the key tool for inspecting and managing cluster resources. -It allows you to run commands against EDB Postgres Distributed clusters - * to determine the health of the cluster, inspect the cluster's configuration, and manage the cluster's resources. - * to inspect and manage the cluster's nodes, groups, and proxies. - * to perform switchover operations on the write leaders of groups. +It allows you to run commands against EDB Postgres Distributed clusters to: + * Determine the health of the cluster, inspect the cluster's configuration, and manage the cluster's resources. + * Inspect and manage the cluster's nodes, groups, and proxies. + * Perform switchover operations on the write leaders of groups. PGD CLI is installed automatically on systems in a TPA-deployed PGD cluster. -It can also be installed manually on Linux and macOS systems that can connect to a PGD cluster, including: +You can also install it manually on Linux and macOS systems that can connect to a PGD cluster, including: * EDB BigAnimal distributed high-availability clusters. * PGD clusters deployed using the EDB PGD for Kubernetes operator. * Manually deployed PGD clusters. diff --git a/product_docs/docs/pgd/5/cli/installing/linux.mdx b/product_docs/docs/pgd/5/cli/installing/linux.mdx index 8189fb21832..83b75a91d59 100644 --- a/product_docs/docs/pgd/5/cli/installing/linux.mdx +++ b/product_docs/docs/pgd/5/cli/installing/linux.mdx @@ -4,7 +4,7 @@ navTitle: Linux description: Installing PGD CLI on Linux --- - PGD CLI is available for most Linux distributions. It is installable from the EDB repositories, which you can access with your EDB account. PGD users and BigAnimal users, including those on a free trial, have an EDB account and access to PGD CLI. + PGD CLI is available for most Linux distributions. You can install it from the EDB repositories, which you can access with your EDB account. PGD users and BigAnimal users, including those on a free trial, have an EDB account and access to PGD CLI. ## Obtain your EDB subscription token diff --git a/product_docs/docs/pgd/5/cli/installing/macos.mdx b/product_docs/docs/pgd/5/cli/installing/macos.mdx index e98cdcca5a6..717a21152b3 100644 --- a/product_docs/docs/pgd/5/cli/installing/macos.mdx +++ b/product_docs/docs/pgd/5/cli/installing/macos.mdx @@ -4,14 +4,14 @@ navTitle: macOS description: Installing PGD CLI on macOS --- -PGD CLI is available for macOS as a [Homebrew](https://brew.sh/) formula. To install it, run the following commands: +PGD CLI is available for macOS as a [Homebrew](https://brew.sh/) formula. To install it, run: ```shell brew tap enterprisedb/tap brew install pgd-cli ``` -Verify the installation by running: +To verify the installation, run: ```shell pgd --version diff --git a/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx b/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx index 98d534043a9..a15cd183f3d 100644 --- a/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx +++ b/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx @@ -176,9 +176,9 @@ By default, TPA commits configuration changes to a Git repository. For this exam You also ask TPA to generate repeatable hostnames for the nodes by passing `--hostnames-unsorted`. Otherwise, it selects hostnames at random from a predefined list of suitable words. -Finally, `--keyring-backend legacy` tells that the keyring backend should be the legacy backend. Secrets are stored with an older keyring backend, as the version of Ubuntu this example is based on doesn't support the newer keyring backend. +Finally, `--keyring-backend legacy` tells that the keyring backend is the legacy backend. Secrets are stored with an older keyring backend, as the version of Ubuntu this example is based on doesn't support the newer keyring backend. -This command creates a subdirectory in the current working directory called `democluster`. It contains the `config.yml` configuration file TPA uses to create the cluster. You can view it using: +This command creates a subdirectory called `democluster` in the current working directory. It contains the `config.yml` configuration file TPA uses to create the cluster. You can view it using: ```shell less democluster/config.yml @@ -197,7 +197,7 @@ less democluster/config.yml ### Deploying the cluster -You can now [deploy](/tpa/latest/tpaexec-deploy/) the distributed cluster. The `tpaexec` command will automatically run its `provision` command to prepare the Docker containers for the deployment: +You can now [deploy](/tpa/latest/tpaexec-deploy/) the distributed cluster. The `tpaexec` command runs its `provision` command to prepare the Docker containers for the deployment: ```shell tpaexec deploy democluster diff --git a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx index 2cf0fbf2548..59850843e8e 100644 --- a/product_docs/docs/pgd/5/reference/catalogs-internal.mdx +++ b/product_docs/docs/pgd/5/reference/catalogs-internal.mdx @@ -25,10 +25,10 @@ Internal catalog table that tracks cluster membership events for a given PGD node. Specifically, it tracks: * Node joins (to the cluster) -* Raft state changes (i.e. whenever the node changes its role in the consensus -protocol - leader, follower or candidate to leader) - see [Monitoring Raft Consensus](../monitoring/sql#monitoring-raft-consensus) +* Raft state changes (that is, whenever the node changes its role in the consensus +protocol - leader, follower or candidate to leader) - see [Monitoring Raft consensus](../monitoring/sql#monitoring-raft-consensus) * Whenever a worker has errored out (see [bdr.workers](/pgd/latest/reference/catalogs-visible/#bdrworkers) -and [Monitoring PGD Replication Workers](../monitoring/sql#monitoring-pgd-replication-workers)) +and [Monitoring PGD replication workers](../monitoring/sql#monitoring-pgd-replication-workers)) #### `bdr.event_history` columns @@ -44,8 +44,8 @@ and [Monitoring PGD Replication Workers](../monitoring/sql#monitoring-pgd-replic ### `bdr.event_summary` -A view of the `bdr.event_history` catalog that display the information in a more -human-friendly format. Specifically, it displays the event types and sub-types +A view of the `bdr.event_history` catalog that displays the information in a more +human-friendly format. Specifically, it displays the event types and subtypes as textual representations, rather than integers. ### `bdr.node_config` @@ -56,7 +56,7 @@ An internal catalog table with per node configuration options. | Name | Type | Description | | ----------------------- | -------- | ---------------------------------------- | -| node_id | oid | The node id | +| node_id | oid | The node ID | | node_route_priority | int | Priority assigned to this node | | node_route_fence | boolean | Switch to fence this node | | node_route_writes | boolean | Switch to allow writes | @@ -71,14 +71,14 @@ An internal catalog table with per node group configuration options. | Name | Type | Description | | ----------------------- | -------- | ---------------------------------------- | -| node_group_id | oid | The node group id | +| node_group_id | oid | The node group ID | | route_writer_max_lag | bigint | Maximum write lag accepted | | route_reader_max_lag | bigint | Maximum read lag accepted | | route_writer_wait_flush | boolean | Switch if we need to wait for the flush | ### `bdr.node_group_routing_config_summary` -Per node group routing configuration options. +Per-node-group routing configuration options. #### `bdr.node_group_routing_config_summary` columns @@ -101,14 +101,14 @@ An internal catalog table holding current routing information for a proxy. | Name | Type | Description | |--------------------|-------|-----------------------------| -| node_group_id | oid | The node group id | +| node_group_id | oid | The node group ID | | write_node_id | oid | Current write node | | prev_write_node_id | oid | Previous write node | | read_node_ids | oid[] | List of read-only nodes IDs | ### `bdr.node_group_routing_summary` -A view of `bdr.node_group_routing_info` catalog which shows the information in more friendly way +A view of `bdr.node_group_routing_info` catalog which shows the information in more friendly way. #### `bdr.node_group_routing_summary` columns | Name | Type | Description | @@ -142,7 +142,7 @@ An internal catalog table holding proxy specific configurations. | Name | Type | Description | |-----------------------------|----------|------------------------------------------------------------------------------| | proxy_name | name | The name of the proxy | -| node_group_id | oid | The node group id that this proxy uses | +| node_group_id | oid | The node group ID that this proxy uses | | listen_port | int | Port that the proxy uses for read-write connections (set to 0 disables port) | | max_client_conn | int | Number of maximum read-write client connections that the proxy accepts | | max_server_conn | int | Number of maximum read-write connections that the server accepts | diff --git a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx index 4278dd840da..26cdb6bd745 100644 --- a/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx +++ b/product_docs/docs/pgd/5/reference/nodes-management-interfaces.mdx @@ -27,23 +27,23 @@ bdr.alter_node_group_option(node_group_name text, - `config_value` — New value to be set for the given key. `config_value` will be parsed into the data type appropriate for the option. -Note that some parameters can only be applied to the top-level node group. These are denoted in the table below with 'Top' appearing in the Groups coluumn. Where the parameter can be applied to the top-level node group and sub-groups, 'All' appears in the Groups column. 'Sub' in the Groups column means the parameter can only be applied to subgroups. +Note that some parameters can be applied only to the top-level node group. These are denoted in the table with Top appearing in the Groups column. Where the parameter can be applied to the top-level node group and subgroups, All appears in the Groups column. Sub in the Groups column means the parameter can be applied only to subgroups. -The group options which can be changed using this function are: +The table shows the group options that can be changed using this function. | Name | Type | Groups | Description | | ---- | ---- | ------ | ----------- | -| `apply_delay` | `interval` | All | How long nodes wait to apply incoming changes. This is useful mainly to setup a special sub-group with delayed subscriber-only nodes. Don't set this on groups which contain data nodes or on the top-level group. Default is `0s`. | -| `check_constraints` | `boolean` | Top | Whether the apply process checks the constraints when writing replicated data. It's recommended to keep this to default value, otherwise you risk data loss. Valid values are either `on` or `off`. Default is `on`. | -| `default_commit_scope` | `text` | All | The commit scope to use by default, initially the `local` commit scope. This applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | +| `apply_delay` | `interval` | All | How long nodes wait to apply incoming changes. This option is useful mainly to set up a special subgroup with delayed subscriber-only nodes. Don't set this on groups that contain data nodes or on the top-level group. Default is `0s`. | +| `check_constraints` | `boolean` | Top | Whether the apply process checks the constraints when writing replicated data. We recommend keeping the default value or you risk data loss. Valid values are `on` or `off`. Default is `on`. | +| `default_commit_scope` | `text` | All | The commit scope to use by default, initially the `local` commit scope. This option applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | | `enable_proxy_routing` | `boolean` | All | Where [`pgd-proxy`](../routing/proxy) through the group leader is enabled for given group. Valid values are `on` or `off`. Default is `off`. | -| `enable_raft` | `boolean` | Sub | Whether group has its own Raft consensus. This is necessary for setting `enable_proxy_routing` to `on`. This is always `on` for the top-level group. Valid values are `on` or `off`. Default is `off` for subgroups. | -| `enable_wal_decoder` | `boolean` | Top | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are either `on` or `off`. Default is `off`. | -| `location` | `text` | All | Information about group location, this is purely metadata for monitoring. Default is `''` (empty string). | -| `num_writers` | `integer` | Top | Number of parallel writers for the subscription backing this node group. Valid values are either `-1` or a positive integer. `-1` means the value specified by the GUC [`bdr.writers_per_subscription`](pgd-settings#bdrwriters_per_subscription) is used. `-1` is the default. | +| `enable_raft` | `boolean` | Sub | Whether group has its own Raft consensus. This option is necessary for setting `enable_proxy_routing` to `on`. This option is always `on` for the top-level group. Valid values are `on` or `off`. Default is `off` for subgroups. | +| `enable_wal_decoder` | `boolean` | Top | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are `on` or `off`. Default is `off`. | +| `location` | `text` | All | Information about group location. This option is purely metadata for monitoring. Default is `''` (empty string). | +| `num_writers` | `integer` | Top | Number of parallel writers for the subscription backing this node group. Valid values are `-1` or a positive integer. `-1` means the value specified by the GUC [`bdr.writers_per_subscription`](pgd-settings#bdrwriters_per_subscription) is used. `-1` is the default. | | `route_reader_max_lag` | `integer` | All | Maximum lag in bytes for a node to be considered a viable read-only node. Currently reserved for future use. | | `route_writer_max_lag` | `integer` | All | Maximum lag in bytes of the new write candidate to be selected as write leader. If no candidate passes this, no writer is selected automatically. Default is `-1`. | -| `route_writer_wait_flush` | `boolean` | All | Whether to switch if we need to wait for the flush. Currently reserved for future use. | +| `route_writer_wait_flush` | `boolean` | All | Whether to switch if PGD needs to wait for the flush. Currently reserved for future use. | | `streaming_mode` | `text` | Top | Enables/disables streaming of large transactions. When set to `off`, streaming is disabled. When set to any other value, large transactions are decoded while they're still in progress, and the changes are sent to the downstream. If the value is set to `file`, then the incoming changes of streaming transactions are stored in a file and applied only after the transaction is committed on upstream. If the value is set to `writer`, then the incoming changes are directly sent to one of the writers, if available.
If [parallel apply](../parallelapply) is disabled or no writer is free to handle streaming transactions, then the changes are written to a file and applied after the transaction is committed. If the value is set to `auto`, PGD tries to intelligently pick between `file` and `writer`, depending on the transaction property and available resources. You can't enable `streaming_mode` if the WAL decoder is already enabled. Default is `auto`.

For more details, see [Transaction streaming](../transaction-streaming). | @@ -497,8 +497,8 @@ This function changes the configuration parameters of an existing PGD group. Options with NULL value (default for all of them) aren't modified. !!! Warning - This function only exists for compatibility with PGD4 and 3.7. - Please use [`bdr.alter_node_group_option`](#bdralter_node_group_option) instead. + This function exists only for compatibility with PGD4 and 3.7. + Use [`bdr.alter_node_group_option`](#bdralter_node_group_option) instead. ### Synopsis @@ -525,12 +525,12 @@ bdr.alter_node_group_config(node_group_name text, | `update_to_insert` | Reserved for backward compatibility. This option is deprecated and will be disabled or removed in future versions of PGD. Use `bdr.alter_node_set_conflict_resolver` instead. | | `ignore_redundant_updates` | Reserved for backward compatibility. | | `check_full_tuple` | Reserved for backward compatibility. | -| `apply_delay` | How long nodes wait to apply incoming changes. This is useful mainly to set up a special sub-group with delayed subscriber-only nodes. Don't set this on groups that contain data nodes or on the top-level group. Default is `0s`. | -| `check_constraints` | Whether the apply process checks the constraints when writing replicated data. It's recommended to keep this to the default value, otherwise you risk data loss. Valid values are either `on` or `off`. Default is `on`. Applies to top-level group only. | -| `num_writers` | Number of parallel writers for the subscription backing this node group. Valid values are either `-1` or a positive integer. `-1` means the value specified by the GUC `bdr.writers_per_subscription` is used. `-1` is the default. Applies to top-level group only.| -| `enable_wal_decoder` | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are either `on` or `off`. Default is `off`. Applies to top-level group only.| +| `apply_delay` | How long nodes wait to apply incoming changes. This parameter is useful mainly to set up a special subgroup with delayed subscriber-only nodes. Don't set this on groups that contain data nodes or on the top-level group. Default is `0s`. | +| `check_constraints` | Whether the apply process checks the constraints when writing replicated data. We recommend keeping this set to the default value or you risk data loss. Valid values are `on` or `off`. Default is `on`. Applies to top-level group only. | +| `num_writers` | Number of parallel writers for the subscription backing this node group. Valid values are `-1` or a positive integer. `-1` means the value specified by the GUC `bdr.writers_per_subscription` is used. `-1` is the default. Applies to top-level group only.| +| `enable_wal_decoder` | Enables/disables the decoding worker process. You can't enable the decoding worker process if `streaming_mode` is already enabled. Valid values are `on` or `off`. Default is `off`. Applies to top-level group only.| | `streaming_mode` | Enables/disables streaming of large transactions. When set to `off`, streaming is disabled. When set to any other value, large transactions are decoded while they're still in progress, and the changes are sent to the downstream. If the value is set to `file`, then the incoming changes of streaming transactions are stored in a file and applied only after the transaction is committed on upstream. If the value is set to `writer`, then the incoming changes are directly sent to one of the writers, if available. If parallel apply is disabled or no writer is free to handle streaming transaction, then the changes are written to a file and applied after the transaction is committed. If the value is set to `auto`, PGD tries to intelligently pick between `file` and `writer`, depending on the transaction property and available resources. You can't enable `streaming_mode` if the WAL decoder is already enabled.

For more details, see [Transaction streaming](../transaction-streaming). Applies to top-level group only.| -| `default_commit_scope` | The commit scope to use by default, initially the `local` commit scope. This applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | +| `default_commit_scope` | The commit scope to use by default, initially the `local` commit scope. This parameter applies only to the top-level node group. You can use individual rules for different origin groups of the same commit scope. See [Origin groups](../durability/commit-scopes/#origin-groups) for more details. | ### Notes @@ -543,6 +543,3 @@ so you can't roll back the function call. Also, the changes might not be immediately visible to the current transaction. This function doesn't hold any locks. - - - diff --git a/product_docs/docs/pgd/5/reference/routing.mdx b/product_docs/docs/pgd/5/reference/routing.mdx index 0a68bb007a5..84129b07afa 100644 --- a/product_docs/docs/pgd/5/reference/routing.mdx +++ b/product_docs/docs/pgd/5/reference/routing.mdx @@ -7,7 +7,7 @@ rootisheading: false ### `bdr.create_proxy` -Create a proxy configuration +Create a proxy configuration. #### Synopsis @@ -21,9 +21,9 @@ bdr.create_proxy(proxy_name text, node_group text, proxy_mode text); |--------------|------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `proxy_name` | text | | Name of the new proxy. | | `node_group` | text | | Name of the group to be used by the proxy. | -| `proxy_mode` | text | 'default' | Mode of the proxy. It can be 'default' (listen_port connections follow write leader, no read_listen_port), 'read-only' (no listen_port, read_listen_port connections follow read-only nodes) or 'any' (listen_port connections follow write_leader, read_listen_port connections follow read-only nodes). Default is 'default'. | +| `proxy_mode` | text | `default` | Mode of the proxy. It can be `default` (listen_port connections follow write leader, no read_listen_port), `read-only` (no listen_port, read_listen_port connections follow read-only nodes), or `any` (listen_port connections follow write_leader, read_listen_port connections follow read-only nodes). Default is `default`. | -When proxy_mode is set to `default` all read options in the proxy config will be set to NULL. When it is set to `read-only` all write options in the proxy config will be set to NULL. When set to `any` all options will be set to their defaults. +When proxy_mode is set to `default`, all read options in the proxy config will be set to NULL. When it's set to `read-only`, all write options in the proxy config will be set to NULL. When set to `any` all options will be set to their defaults. ### `bdr.alter_proxy_option` @@ -45,7 +45,7 @@ bdr.alter_proxy_option(proxy_name text, config_key text, config_value text); | `config_value` | text | | New value to be set for the given key. | -The proxy options (`config_key`) that can be changed using this function are: +The table shows the proxy options (`config_key`) that can be changed using this function. | Option | Description | |-------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -55,20 +55,20 @@ The proxy options (`config_key`) that can be changed using this function are: | `max_server_conn` | Maximum number of connections the proxy can make to the Postgres node. Default is '32767'. | | `server_conn_timeout` | Connection timeout for server connections. Default is '2' (seconds). | | `server_conn_keepalive` | Keepalive interval for server connections. Default is '10' (seconds). | -| `consensus_grace_period` | Duration for which proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overriden if `raft_response_timeout`, `raft_global_election_timeout` or `raft_group_election_timeout` are changed from their defaults. | +| `consensus_grace_period` | Duration for which proxy continues to route even upon loss of a Raft leader. If set to 0s, proxy stops routing immediately. Default is generally '6' (seconds) for local proxies and '12' (seconds) for global proxies. These values will be overridden if `raft_response_timeout`, `raft_global_election_timeout`, or `raft_group_election_timeout` are changed from their defaults. | | `read_listen_address` | Address for the read-only proxy to listen on. Default is '{0.0.0.0}'. | | `read_listen_port` | Port for the read-only proxy to listen on. Default is '6433' in 'read-only' or 'any' mode and '0' in 'default' mode which disables the read-only port . | | `read_max_client_conn` | Maximum number of connections for the read-only proxy to accept. Default is '32767'. | | `read_max_server_conn` | Maximum number of connections the read-only proxy can make to the Postgres node. Default is '32767'. | | `read_server_conn_keepalive` | Keepalive interval for read-only server connections. Default is '10' (seconds). | | `read_server_conn_timeout` | Connection timeout for read-only server connections. Default is '2' (seconds). | -| `read_consensus_grace_period` | Duration for which read-only proxy continues to route even upon loss of a Raft leader.Default is 1 hour. | +| `read_consensus_grace_period` | Duration for which read-only proxy continues to route even upon loss of a Raft leader. Default is 1 hour. | -Changing any of theses values requires a restart of the proxy. +Changing any of these values requires a restart of the proxy. ### `bdr.drop_proxy` -Drop a proxy configuration +Drop a proxy configuration. #### Synopsis @@ -84,7 +84,7 @@ bdr.drop_proxy(proxy_name text); ### `bdr.routing_leadership_transfer` -Changing the routing leader transfers the leadership of the node group to another node +Changing the routing leader transfers the leadership of the node group to another node. #### Synopsis @@ -101,6 +101,6 @@ bdr.routing_leadership_transfer(node_group_name text, |--------------------|----------|----------|---------------------------------------------------------------------------------------------| | `node_group_name` | text | | Name of group where the leadership transfer is requested. | | `leader_name` | text | | Name of node that will become write leader. | -| `transfer_method` | text | 'strict' | Type of the transfer, it can be "fast" or the default "strict" that checks the maximum lag. | -| `transfer_timeout` | interval | '10s' | Timeout of the leadership transfer, default is 10 seconds. | +| `transfer_method` | text | 'strict' | Type of the transfer. It can be 'fast' or the default, 'strict', which checks the maximum lag. | +| `transfer_timeout` | interval | '10s' | Timeout of the leadership transfer. Default is 10 seconds. | diff --git a/product_docs/docs/pgd/5/routing/administering.mdx b/product_docs/docs/pgd/5/routing/administering.mdx index 8a173438b94..652ad7cf4ed 100644 --- a/product_docs/docs/pgd/5/routing/administering.mdx +++ b/product_docs/docs/pgd/5/routing/administering.mdx @@ -9,11 +9,11 @@ Switching the write leader is a manual operation that you can perform to change It can be useful when you want to perform maintenance on the current write leader node or when you want to change the write leader for any other reason. When changing write leader, there are two modes, `strict` and `fast`. In `strict` mode, the lag is checked before switching the write leader. It will wait until the lag is less than `route_writer_max_lag` before starting the switchover. This is the default. -In `fast` mode, the write leader is switched immediatly. -There is also a timeout parameter that can be set to specify the time to wait for the switchover to complete. +In `fast` mode, the write leader is switched immediately. +You can also set timeout parameter to specify the time to wait for the switchover to complete. !!! Note - The switchover operation is not a guaranteed operation. If due to a timeout or for other reasons, the switchover to the given target node fails, PGD may elect another node as write leader in its place. This other node can include the current write leader node. PGD will always try to elect a new write leader if the switchover operation fails. + The switchover operation is not a guaranteed operation. If, due to a timeout or for other reasons, the switchover to the given target node fails, PGD may elect another node as write leader in its place. This other node can include the current write leader node. PGD will always try to elect a new write leader if the switchover operation fails. !!! ### Using SQL @@ -28,7 +28,7 @@ For example, to switch the write leader to node `node1` in group `group1`, use t ```sql SELECT bdr.routing_leadership_transfer('group1', 'node1','strict','10s'); ``` -This command will switch the write leader using `strict` mode and wait for up to 10 seconds for the switchover to complete. Those are default settings and can be omitted. +This command switches the write leader using `strict` mode and waits for up to 10 seconds for the switchover to complete. Those are default settings, so you can omit them. ```sql SELECT bdr.routing_leadership_transfer('group1', 'node1'); @@ -44,11 +44,8 @@ For example, to switch the write leader from node `node1` to node `node2` in gro pgd switchover --node-group group1 --node-name node1 --method strict --timeout 10s ``` -This command will switch the write leader using `strict` mode and wait for up to 10 seconds for the switchover to complete. Those are default settings and can be omitted. +This command switches the write leader using `strict` mode and waits for up to 10 seconds for the switchover to complete. Those are default settings, so you can omit them. ```sh pgd switchover --node-group group1 --node-name node1 ``` - - - diff --git a/product_docs/docs/pgd/5/routing/configuration.mdx b/product_docs/docs/pgd/5/routing/configuration.mdx index 713fce08053..151a46d7bb3 100644 --- a/product_docs/docs/pgd/5/routing/configuration.mdx +++ b/product_docs/docs/pgd/5/routing/configuration.mdx @@ -3,10 +3,10 @@ title: "PGD Proxy configuration" navTitle: "Configuration" --- -## Group level configuration +## Group-level configuration Configuring the routing is done through either SQL interfaces or through -PGD-CLI. +PGD CLI. You can enable routing decisions by calling the [`bdr.alter_node_group_option()`](/pgd/latest/reference/nodes-management-interfaces#bdralter_node_group_option) function. For example: @@ -23,23 +23,23 @@ Additional group-level options affect the routing decisions: selected as write leader. If no candidate passes this, no writer is selected automatically. - `route_reader_max_lag` — Maximum lag in bytes for a node to be considered a viable - read-only node. PGD 5.5.0 and later. + read-only node (PGD 5.5.0 and later). -## Node level configuration +## Node-level configuration -Per-node configuration of routing is set using [`bdr.alter_node_option()`](/pgd/latest/reference/nodes-management-interfaces#bdralter_node_option). The -available options that affect routing are the following: +Set per-node configuration of routing using [`bdr.alter_node_option()`](/pgd/latest/reference/nodes-management-interfaces#bdralter_node_option). The +available options that affect routing are: - `route_dsn` — The dsn used by proxy to connect to this node. - `route_priority` — Relative routing priority of the node against other nodes in - the same node group. Only used when electing a Write Leader. -- `route_fence` — Determines Whether the node is fenced from routing. When fenced the node can't receive connections - from PGD Proxy. It therefore cannot become the Write Leader or be available in read-only node pool. -- `route_writes` — Whether writes can be routed to this node, that is, whether the node + the same node group. Used only when electing a write Leader. +- `route_fence` — Determines whether the node is fenced from routing. When fenced, the node can't receive connections + from PGD Proxy. It therefore can't become the write Leader or be available in the read-only node pool. +- `route_writes` — Determines whether writes can be routed to this node, that is, whether the node can become write leader. -- `route_reads` — Whether read-only connections can be routed to this node. PGD 5.5.0 and later. +- `route_reads` — Determines whether read-only connections can be routed to this node (PGD 5.5.0 and later). -## Proxy level configuration +## Proxy-level configuration You can configure the proxies using SQL interfaces. diff --git a/product_docs/docs/pgd/5/routing/index.mdx b/product_docs/docs/pgd/5/routing/index.mdx index 70aa43b77d5..27308c310ab 100644 --- a/product_docs/docs/pgd/5/routing/index.mdx +++ b/product_docs/docs/pgd/5/routing/index.mdx @@ -15,7 +15,7 @@ navigation: Managing application connections is an important part of high availability. PGD Proxy offers a way to manage connections to the EDB Postgres Distributed cluster. It acts as a proxy layer between the client application and the Postgres database. -* [PGD Proxy overview](/pgd/latest/routing/proxy) provides an overview of the PGD Proxy, its processes and how it interacts with the EDB Postgres Distributed cluster. +* [PGD Proxy overview](/pgd/latest/routing/proxy) provides an overview of the PGD Proxy, its processes, and how it interacts with the EDB Postgres Distributed cluster. * [Installing the PGD Proxy service](/pgd/latest/routing/installing_proxy) covers installation of the PGD Proxy service on a host. @@ -25,7 +25,6 @@ Managing application connections is an important part of high availability. PGD * [Monitoring PGD Proxy](/pgd/latest/routing/monitoring) looks at how to monitor PGD Proxy through the cluster and at a service level. -* [Read-Only Routing](/pgd/latest/routing/readonly) explains how the read-only routing feature in PGD Proxy enables read scalability. +* [Read-only routing](/pgd/latest/routing/readonly) explains how the read-only routing feature in PGD Proxy enables read scalability. * [Raft](/pgd/latest/routing/raft) provides an overview of the Raft consensus mechanism used to coordinate PGD Proxy. - diff --git a/product_docs/docs/pgd/5/routing/monitoring.mdx b/product_docs/docs/pgd/5/routing/monitoring.mdx index 554bb9fffb3..497fddfa9e5 100644 --- a/product_docs/docs/pgd/5/routing/monitoring.mdx +++ b/product_docs/docs/pgd/5/routing/monitoring.mdx @@ -18,26 +18,26 @@ The [`bdr.node_routing_config_summary`](/pgd/latest/reference/catalogs-internal# ### Using PGD CLI You can use the `show-proxies` command to view the current status of all proxies in the PGD cluster. -The `show-proxy` command to view the status of a specific proxy. +Use the `show-proxy` command to view the status of a specific proxy. ## Monitoring at the process level ### Proxy health check -PGD Proxy provides the following HTTP(s) health check API endpoints. The API endpoints respond to `GET` requests. You need to enable and configure the endpoints before using them. See [Configurations](installing_proxy#configuring-health-check). +PGD Proxy provides the following HTTP(s) health check API endpoints. The API endpoints respond to `GET` requests. You need to enable and configure the endpoints before using them. See [Configuration](installing_proxy#configuring-health-check). | Endpoint | Description | | --- | --- | | `/health/is-ready` | Checks if the proxy can successfully route connections to the current write leader. | | `/health/is-live` | Checks if the proxy is running. | -| `/health/is-write-ready` | Checks if the proxy can successfully route connections to the current write leader. PGD 5.5.0 and later | -| `/health/is-read-only-ready` | Checks if the proxy can successfully route read-only connections. PGD 5.5.0 and later | +| `/health/is-write-ready` | Checks if the proxy can successfully route connections to the current write leader (PGD 5.5.0 and later). | +| `/health/is-read-only-ready` | Checks if the proxy can successfully route read-only connections (PGD 5.5.0 and later). | #### Readiness On receiving a valid `GET` request: -* When in default (write mode), the proxy checks if it can successfully route connections to the current write leader. +* When in default (write) mode, the proxy checks if it can successfully route connections to the current write leader. * When in read-only mode, the proxy checks if it can successfully route read-only connections. * When in any mode, the proxy first checks if it can successfully route connections to the current write leader. If it can, the check is successful. If not, it checks if it can route a read-only connection. If it can, the check is successful. If not, the check fails. @@ -56,7 +56,7 @@ Proxies also write logs to system logging where they can be monitored with other - Debian based - `/var/log/syslog` - Red Hat based - `/var/log/messages` -Use the `journalctl` command to filter and view logs for troubleshooting PGD Proxy. The following are few sample commands for quick reference: +Use the `journalctl` command to filter and view logs for troubleshooting PGD Proxy. The following are sample commands for quick reference: ```sh journalctl -u pgd-proxy -n100 -f diff --git a/product_docs/docs/pgd/5/routing/proxy.mdx b/product_docs/docs/pgd/5/routing/proxy.mdx index ae485a2b759..4f802fb3af6 100644 --- a/product_docs/docs/pgd/5/routing/proxy.mdx +++ b/product_docs/docs/pgd/5/routing/proxy.mdx @@ -1,52 +1,52 @@ --- -title: "EDB Postgres Distributed Proxy Overview" -navTitle: "PGD Proxy Overview" +title: "EDB Postgres Distributed Proxy overview" +navTitle: "PGD Proxy overview" indexCards: simple directoryDefaults: - description: "The PGD Proxy service acts as proxy layer between the client application and Postgres for your PGD cluster" + description: "The PGD Proxy service acts as proxy layer between the client application and Postgres for your PGD cluster." --- -Especially with asynchronous replication, having a consistent write lead node is +Especially with asynchronous replication, having a consistent write leader node is important to avoid conflicts and guarantee availability for the application. -There are two parts to EDB Postgres Distributed's proxy layer: +The two parts to EDB Postgres Distributed's proxy layer are: -* Proxy configuration and routing information which is maintained by the PGD consensus mechanism. -* The PGD Proxy service which is installed on a host. It connects to the PGD cluster where it reads its configuration and listens for changes to the routing information. +* Proxy configuration and routing information, which is maintained by the PGD consensus mechanism. +* The PGD Proxy service, which is installed on a host. It connects to the PGD cluster where it reads its configuration and listens for changes to the routing information. This is normally installed in highly available configuration (at least two instances of the proxy service per PGD group). Once configured, the PGD Proxy service monitors routing changes as decided by the EDB Postgres Distributed cluster. It acts on these changes to ensure that connections are consistently routed to the correct nodes. Configuration changes to the PGD Proxy service are made through the PGD cluster. -The PGD Proxy service reads its configuration from the PGD cluster, but the proxy service will need to be restarted to apply those changes. +The PGD Proxy service reads its configuration from the PGD cluster, but the proxy service must be restarted to apply those changes. The information about currently selected write and read nodes is visible in -`bdr.node_group_routing_summary`. Note that this is node-local view, the proxy -always reads from Raft leader to get current and consistent view. +`bdr.node_group_routing_summary`. This is node-local view: the proxy +always reads from Raft leader to get a current and consistent view. ## Leader selection The write leader is selected by the current Raft leader (either subgroup one or top-level group one, -depending on whether the leader for sub-group or the cluster's top-level group is being selected). +depending on whether the leader for the subgroup or the cluster's top-level group is being selected). -Leader is selected from candidate nodes which are reachable and meet the criteria based -on the configuration as described bellow. To be a viable candidate, node has to have +Leader is selected from candidate nodes that are reachable and meet the criteria based +on the configuration as described in [PGD Proxy cluster configuration](#pgd-proxy-cluster-configuration). To be a viable candidate, the node must have `route_writes` enabled and `route_fence` disabled and be within `route_writer_max_lag` -(if enabled) from previous leader. The candidates are ordered by their `route_priority` -in descending order and by the lag from previous leader in ascending order. +(if enabled) from the previous leader. The candidates are ordered by their `route_priority` +in descending order and by the lag from the previous leader in ascending order. -New leader selection process is started when there is either no existing leader currently -(this could be because there were no valid candidates or because Raft was down), or when +The new leader selection process is started either when there's no existing leader currently +(this could be because there were no valid candidates or because Raft was down) or when connectivity is lost to the existing leader. -A Node is considered connected if the last Raft protocol message received from by the leader -it is not older than Raft election timeout -(see [Internal settings - Raft timeouts](/pgd/latest/reference/pgd-settings#internal-settings---raft-timeouts) ). +A node is considered connected if the last Raft protocol message received from the leader +isn't older than Raft election timeout +(see [Internal settings - Raft timeouts](../reference/pgd-settings#internal-settings---raft-timeouts)). Since the Raft leader is sending heartbeat 3 times every election timeout limit, the leader -node needs to miss reply to 3 heartbeats before it's considered disconnected. +node needs to miss the reply to 3 heartbeats before it's considered disconnected. ## PGD Proxy cluster configuration @@ -54,12 +54,12 @@ The PGD cluster always has at least one top-level group and one data group. PGD The cluster also maintains Proxy configurations for each group. Each configuration has a name and is associated with a group. You can attach Proxy to a top-level group or data group. You can attach multiple proxies to each group. -When a PGD Proxy service starts running on a host, it has a name in its local configuration file and it will connect to a node in a group, from where it will use the name to look up its complete configuration as stored on the group. +When a PGD Proxy service starts running on a host, it has a name in its local configuration file and it connects to a node in a group, from where it uses the name to look up its complete configuration as stored on the group. ## PGD Proxy service -The PGD Proxy service (EDB Postgres Distributed Proxy) is a process that acts as an abstraction layer between the client application and Postgres. It interfaces with the PGD consensus mechanism to get the identity of the current write leader node and redirects traffic to that node. It also optionally supports a read-only mode where it can route read-only queries to nodes that are not the write leader, improving the overall performance of the cluster. +The EDB Postgres Distributed Proxy (PGD Proxy) servic e is a process that acts as an abstraction layer between the client application and Postgres. It interfaces with the PGD consensus mechanism to get the identity of the current write leader node and redirects traffic to that node. It also optionally supports a read-only mode where it can route read-only queries to nodes that aren't the write leader, improving the overall performance of the cluster. PGD Proxy is a TCP layer 4 proxy. @@ -73,7 +73,7 @@ Upon starting, PGD Proxy connects to one of the endpoints given in the local con The endpoints given in the config file are used only at startup. After that, actual endpoints are taken from the PGD catalog's `route_dsn` field in [`bdr.node_routing_config_summary`](/pgd/latest/reference/catalogs-internal#bdrnode_routing_config_summary). -PGD manages write leader election. PGD Proxy interacts with PGD to get write leader change events notifications on Postgres notify/listen channels and routes client traffic to the current write leader. PGD Proxy disconnects all existing client connections on write leader change or when write leader is unavailable. Write leader election is a Raft-backed activity and is subject to Raft leader availability. PGD Proxy closes the new client connections if write leader is unavailable. +PGD manages write leader election. PGD Proxy interacts with PGD to get write leader change events notifications on Postgres notify/listen channels and routes client traffic to the current write leader. PGD Proxy disconnects all existing client connections on write leader change or when write leader is unavailable. Write leader election is a Raft-backed activity and is subject to Raft leader availability. PGD Proxy closes the new client connections if the write leader is unavailable. PGD Proxy responds to write leader change events that can be categorized into two modes of operation: *failover* and *switchover*. @@ -87,11 +87,11 @@ If the proxy is configured to support read-only routing, it can route read-only PGD Proxy provides the `consensus_grace_period` proxy option that can be used to configure the routing behavior upon loss of a Raft leader. PGD Proxy continues to route to the current write leader (if it's available) for this duration. If the new Raft leader isn't elected during this period, the proxy stops routing. If set to `0s`, PGD Proxy stops routing immediately. -The main purpose of this option is to allow users to configure the write behavior when the Raft leader is lost. When the Raft leader isn't present in the cluster, it's not always guaranteed that the current write leader seen by the proxy is the correct one. In some cases, like network partition in the following example, it is possible that the two write leaders may be seen by two different proxies attached to the same group increasing the chances of write conflicts. If this isn't the desired behavior, then the previously mentioned `consensus_grace_period` can be set to 0s. This setting configures the proxy to stop routing and closes existing open connections immediately when it detects the Raft leader is lost. +The main purpose of this option is to allow users to configure the write behavior when the Raft leader is lost. When the Raft leader isn't present in the cluster, it's not always guaranteed that the current write leader seen by the proxy is the correct one. In some cases, like network partition in the following example, it's possible that the two write leaders may be seen by two different proxies attached to the same group, increasing the chances of write conflicts. If this isn't the desired behavior, then you can set the previously mentioned `consensus_grace_period` to 0s. This setting configures the proxy to stop routing and closes existing open connections immediately when it detects the Raft leader is lost. #### Network partition example -Consider a 3-data node group with a proxy on each data node. In this case, if the current write leader gets network partitioned or isolated, then the data nodes present in the majority partition elects a new write leader. If `consensus_grace_period` is set to a non-zero value, say `10s`, then the proxy present on the previous write leader continues to route writes for this duration. +Consider a 3-data node group with a proxy on each data node. In this case, if the current write leader gets network partitioned or isolated, then the data nodes present in the majority partition elect a new write leader. If `consensus_grace_period` is set to a non-zero value, say `10s`, then the proxy present on the previous write leader continues to route writes for this duration. In this case, if the grace period is kept too high, then writes continue to happen on the two write leaders. This condition increases the chances of write conflicts. @@ -106,4 +106,3 @@ Similar to the `consensus_grace_period`, a `read_consensus_grace_period` option The PostgreSQL C client library (libpq) allows you to specify multiple host names in a single connection string for simple failover. This is also supported by client libraries (drivers) in some other programming languages. It works well for failing over across PGD Proxy instances that are down or inaccessible. However, if the PGD Proxy instance is accessible but doesn't have access to the write leader, or the write leader for a given instance doesn't exist (that is, because there's no write leader for the given PGD group), the connection simply fails. No other hosts in the multi-host connection string is tried. This behavior is consistent with the behavior of PostgreSQL client libraries with other proxies like HAProxy or pgbouncer. - diff --git a/product_docs/docs/pgd/5/routing/readonly.mdx b/product_docs/docs/pgd/5/routing/readonly.mdx index 76e559a6918..b2da5c6bd27 100644 --- a/product_docs/docs/pgd/5/routing/readonly.mdx +++ b/product_docs/docs/pgd/5/routing/readonly.mdx @@ -7,36 +7,37 @@ navTitle: Read-Only Routing By default, PGD Proxy routes connections to the currently selected write leader in the cluster. This allows the write traffic conflicts to be rapidly and consistently resolved. Just routing everything to a single node, the write leader, is a natural fit for traditional high availability deployments where system throughput is typically limited to the throughput of what a single node can handle. -But for some use cases this behavior also means that clients which are only querying the data are also placing a load on the current write leader which could equally well be served by one of the non-write leader nodes in the cluster. +But for some use cases, this behavior also means that clients that are only querying the data are also placing a load on the current write leader. It's possible this writer leader could be equally well served by one of the non-write leader nodes in the cluster. -If you could move traffic that was read-only queries to the non-write leader nodes, you could, at least in theory, handle a multiple of single node throughput. An approach like this would typically require changes to applications knowing details of cluster topology and current node status to detect write lead. +If you could move traffic that was read-only queries to the non-write leader nodes, you could, at least in theory, handle a multiple of single node throughput. An approach like this would typically require changes to applications knowing details of cluster topology and current node status to detect write lead. +

{{ thisass.vulnerability_details.cve_id }}

-  Read Assesment +  Read Assessment   Updated: {{ thisass.open.last_updated }}

{{ thisass.frontmatter.title }}

{{ thisass.frontmatter.affectedProducts }}
diff --git a/product_docs/docs/epas/12/epas_rel_notes/epas12_15_19_rel_notes.mdx b/product_docs/docs/epas/12/epas_rel_notes/epas12_15_19_rel_notes.mdx index f26122eb6ae..01efd75c309 100644 --- a/product_docs/docs/epas/12/epas_rel_notes/epas12_15_19_rel_notes.mdx +++ b/product_docs/docs/epas/12/epas_rel_notes/epas12_15_19_rel_notes.mdx @@ -4,16 +4,16 @@ title: "Version 12.15.19" EDB Postgres Advanced Server 12.15.19 includes the following enhancements: -| Type | Description | Category | -| -------------- | -------------------------------------------------------------------------------------------------------------------------------------| --------------------- | -| Upstream merge | Merged with community PostgreSQL 12.15.19. See the community [Release Notes](https://www.postgresql.org/docs/release/12.15/) for details. | | -| Enhancement | SQL Profiler and Index Advisor are now extensions and can be downloaded from [EDB Repos](https://repos.enterprisedb.com/). | | +| Type | Description | Category | +|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| +| Upstream merge | Merged with community PostgreSQL 12.15.19. See the community [Release Notes](https://www.postgresql.org/docs/release/12.15/) for details. | | +| Enhancement | SQL Profiler and Index Advisor are now extensions and can be downloaded from [EDB Repos](https://repos.enterprisedb.com/). | | | Bug fix | Fixed an issue in which "PASSWORD EXPIRE AT" was dumped when the password status wasn't expired. This fix prevents marking the user account as expired after an upgrade. | Profile | -| Bug fix | Fixed the password profile behavior after the password grace time has changed. | | -| Bug fix | Fixed unexpected error for `edb_enable_pruning` parameter. [Support ticket: #89863] | | -| Bug fix | Fixed an issue when a user enters `Ctrl-c`(SIGINT) to cancel the load in EDB\*Loader. [Support ticket: #88734] | | -| Bug fix | Set correct object descriptions for redaction policy to make pg_dump work cleanly with `--clean` and `--if-exists` options. | | -| Bug fix | Fixed pg_dump to dump password verify function for the user profile. | | -| Bug fix | Fixed assertion failure while terminating the process within the autonomous transaction. | | -| Bug fix | Fixed corner-case uninitialized-variable issues in SPL. | | -| Bug fix | Fixed memory leakage in anonymous blocks that use cast expressions. [Support ticket: #88816] | | +| Bug fix | Fixed the password profile behavior after the password grace time has changed. | | +| Bug fix | Fixed unexpected error for `edb_enable_pruning` parameter. [Support ticket: #89863] | | +| Bug fix | Fixed an issue when a user enters `Ctrl-c`(SIGINT) to cancel the load in EDB\*Loader. [Support ticket: #88734] | | +| Bug fix | Set correct object descriptions for redaction policy to make pg_dump work cleanly with `--clean` and `--if-exists` options. | | +| Bug fix | Fixed pg_dump to dump password verify function for the user profile. | | +| Bug fix | Fixed assertion failure while terminating the process within the autonomous transaction. | | +| Bug fix | Fixed corner-case uninitialized-variable issues in SPL. | | +| Bug fix | Fixed memory leakage in anonymous blocks that use cast expressions. [Support ticket: #88816] | | diff --git a/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx b/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx new file mode 100644 index 00000000000..da22247b828 --- /dev/null +++ b/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx @@ -0,0 +1,17 @@ +--- +title: EDB Postgres Advanced Server 12.19.24 release notes +navTitle: "Version 12.19.24" +--- + +Released: 9 May 2024 + +EDB Postgres Advanced Server 12.19.24 includes the following enhancements and bug fixes: + +| Type | Description | Category | +|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| +| Upstream merge | Merged with community PostgreSQL 12.19. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 12.19 Release Notes](https://www.postgresql.org/docs/release/12.19/) for more information. | | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. [Support ticket: #36220] | | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. [Support ticket: #99282]| | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. [Support ticket: 102746] | | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs. [Support ticket: #86497] | | +| Bug fix | Fixed a crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | \ No newline at end of file diff --git a/product_docs/docs/epas/12/epas_rel_notes/index.mdx b/product_docs/docs/epas/12/epas_rel_notes/index.mdx index 7456d3d422b..84152502872 100644 --- a/product_docs/docs/epas/12/epas_rel_notes/index.mdx +++ b/product_docs/docs/epas/12/epas_rel_notes/index.mdx @@ -2,6 +2,7 @@ navTitle: Release Notes title: "EDB Postgres Advanced Server Release Notes" navigation: +- epas12_19_24_rel_notes - epas12_18_23_rel_notes - epas12_17_22_rel_notes - epas12_16_21_rel_notes @@ -30,6 +31,7 @@ The EDB Postgres Advanced Server (Advanced Server) documentation describes the l | Version | Release Date | Upstream Merges | | ----------------------------------------- | ------------ | -------------------------------------------------------------- | +| [12.19.24](epas12_19_24_rel_notes.mdx) | 09 May 2024 | [12.19](https://www.postgresql.org/docs/12/release-12-19.html) | | [12.18.23](epas12_18_23_rel_notes.mdx) | 08 Feb 2023 | [12.18](https://www.postgresql.org/docs/12/release-12-18.html) | | [12.17.22](epas12_17_22_rel_notes.mdx) | 09 Nov 2023 | [12.17](https://www.postgresql.org/docs/12/release-12-17.html) | | [12.16.21](epas12_16_21_rel_notes.mdx) | 25 Sep 2023 | | diff --git a/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx b/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx new file mode 100644 index 00000000000..df27f2a9203 --- /dev/null +++ b/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx @@ -0,0 +1,17 @@ +--- +title: EDB Postgres Advanced Server 13.15.21 release notes +navTitle: "Version 13.15.21" +--- + +Released: 9 May 2024 + +EDB Postgres Advanced Server 13.15.21 includes the following enhancements and bug fixes: + +| Type | Description | Addresses                | +|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| +| Upstream merge | Merged with community PostgreSQL 13.15. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 13.15 Release Notes](https://www.postgresql.org/docs/release/14.15/) for more information. | | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs. | #86497 | +| Bug fix | Fixed a crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | diff --git a/product_docs/docs/epas/13/epas_rel_notes/index.mdx b/product_docs/docs/epas/13/epas_rel_notes/index.mdx index d011ce584a7..4ee32a0b812 100644 --- a/product_docs/docs/epas/13/epas_rel_notes/index.mdx +++ b/product_docs/docs/epas/13/epas_rel_notes/index.mdx @@ -2,6 +2,7 @@ navTitle: Release Notes title: "EDB Postgres Advanced Server Release Notes" navigation: +- epas13_15_21_rel_notes - epas13_14_20_rel_notes - epas13_13_19_rel_notes - epas13_12_18_rel_notes @@ -21,8 +22,9 @@ EDB Postgres Advanced Server 13 is built on open-source PostgreSQL 13, which int The EDB Postgres Advanced Server (Advanced Server) documentation describes the latest version of Advanced Server 13 including minor releases and patches. The release notes in this section provide information on what was new in each release. -| Version | Release Date | Upstream Merges | -| ----------------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------- | +| Version | Release Date | Upstream Merges | +|-------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------| +| [13.15.21](epas13_15_21_rel_notes) | 09 May 2024 | [13.15](https://www.postgresql.org/docs/release/13.15/) | | [13.14.20](epas13_14_20_rel_notes) | 08 Feb 2024 | [13.14](https://www.postgresql.org/docs/release/13.14/) | | [13.13.19](epas13_13_19_rel_notes) | 09 Nov 2023 | [13.13](https://www.postgresql.org/docs/release/13.13/) | | [13.12.18](epas13_12_17_rel_notes) | 25 Sep 2023 | | diff --git a/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx b/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx new file mode 100644 index 00000000000..e1368f2829a --- /dev/null +++ b/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx @@ -0,0 +1,17 @@ +--- +title: EDB Postgres Advanced Server 14.12.0 release notes +navTitle: "Version 14.12.0" +--- + +Released: 9 May 2024 + +EDB Postgres Advanced Server 14.12.0 includes the following enhancements and bug fixes: + +| Type | Description | Addresses                | +|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| +| Upstream merge | Merged with community PostgreSQL 14.12. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 14.12 Release Notes](https://www.postgresql.org/docs/release/14.12/) for more information. | | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs. | #86497 | +| Bug fix | Fixed a crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | \ No newline at end of file diff --git a/product_docs/docs/epas/14/epas_rel_notes/index.mdx b/product_docs/docs/epas/14/epas_rel_notes/index.mdx index 0a27a139ad0..9f38fd51efa 100644 --- a/product_docs/docs/epas/14/epas_rel_notes/index.mdx +++ b/product_docs/docs/epas/14/epas_rel_notes/index.mdx @@ -2,6 +2,7 @@ navTitle: Release notes title: "EDB Postgres Advanced Server release notes" navigation: +- epas14_12_0_rel_notes - epas14_11_0_rel_notes - epas14_10_0_rel_notes - epas14_9_1_rel_notes @@ -20,10 +21,11 @@ EDB Postgres Advanced Server 14 is built on open-source PostgreSQL 14, which int The EDB Postgres Advanced Server (EDB Postgres Advanced Server) documentation describes the latest version of EDB Postgres Advanced Server 14 including minor releases and patches. The release notes in this section provide information on what was new in each release. -| Version | Release date | Upstream merges | -| --------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------ | -| [14.11.0](epas14_11_0_rel_notes) | 08 Feb 2024 | [14.11](https://www.postgresql.org/docs/14/release-14-11.html) | -| [14.10.0](epas14_10_0_rel_notes) | 09 Nov 2023 | [14.10](https://www.postgresql.org/docs/14/release-14-10.html) | +| Version | Release date | Upstream merges | +|-----------------------------------|--------------|--------------------------------------------------------------------------------------------------------------------------| +| [14.12.0](epas14_12_0_rel_notes) | 09 May 2024 | [14.12](https://www.postgresql.org/docs/14/release-14-12.html) | +| [14.11.0](epas14_11_0_rel_notes) | 08 Feb 2024 | [14.11](https://www.postgresql.org/docs/14/release-14-11.html) | +| [14.10.0](epas14_10_0_rel_notes) | 09 Nov 2023 | [14.10](https://www.postgresql.org/docs/14/release-14-10.html) | | [14.9.1](epas14_9_1_rel_notes) | 25 Sep 2023 | | | [14.9.0](epas14_9_0_rel_notes) | 21 Aug 2023 | [14.9](https://www.postgresql.org/docs/14/release-14-9.html) | | [14.8.0](epas14_8_0_rel_notes) | 11 May 2023 | [14.8](https://www.postgresql.org/docs/14/release-14-8.html) | diff --git a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx new file mode 100644 index 00000000000..6fadd698179 --- /dev/null +++ b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx @@ -0,0 +1,19 @@ +--- +title: EDB Postgres Advanced Server 15.7.0 release notes +navTitle: "Version 15.7.0 " +--- + +Released: 9 May 2024 + +EDB Postgres Advanced Server 15.7.0 includes the following enhancements and bug fixes: + +| Type | Description | Addresses                | +|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| +| Upstream merge | Merged with community PostgreSQL 15.7. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15.7 Release Notes](https://www.postgresql.org/docs/release/15.7/) for more information. | | +| Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sublink in `CONNECT BY` processing to avoid the server crash. | #102746 | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the freed path is still in use, specially in FDWs. | #86497 | +| Bug fix | Fixed an crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | +| Bug fix | Fixed an issue with possible data loss and `pg_dump` failures when using rowids. | #35901 | diff --git a/product_docs/docs/epas/15/epas_rel_notes/index.mdx b/product_docs/docs/epas/15/epas_rel_notes/index.mdx index bbef2bebd7f..e413b74f2fc 100644 --- a/product_docs/docs/epas/15/epas_rel_notes/index.mdx +++ b/product_docs/docs/epas/15/epas_rel_notes/index.mdx @@ -2,6 +2,7 @@ navTitle: Release notes title: "EDB Postgres Advanced Server release notes" navigation: +- epas15_7_0_rel_notes - epas15_6_0_rel_notes - epas15_5_0_rel_notes - epas15_4_1_rel_notes @@ -14,9 +15,10 @@ EDB Postgres Advanced Server 15 is built on open-source PostgreSQL 15, which int The EDB Postgres Advanced Server documentation describes the latest version of EDB Postgres Advanced Server 15 including minor releases and patches. These release notes provide information on what was new in each release. -| Version | Release date | Upstream merges | -| ------------------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [15.6.0](epas15_6_0_rel_notes) | 08 Feb 2024 | [15.6](https://www.postgresql.org/docs/release/15.6/) +| Version | Release date | Upstream merges | +|--------------------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [15.7.0](epas15_7_0_rel_notes) | 09 May 2024 | [15.7](https://www.postgresql.org/docs/release/15.7/) | +| [15.6.0](epas15_6_0_rel_notes) | 08 Feb 2024 | [15.6](https://www.postgresql.org/docs/release/15.6/) | | [15.5.0](epas15_5_0_rel_notes) | 09 Nov 2023 | [15.5](https://www.postgresql.org/docs/release/15.5/) | | [15.4.1](epas15_4_1_rel_notes) | 25 Sep 2023 | | | [15.4.0](epas15_4_0_rel_notes) | 21 Aug 2023 | [15.4](https://www.postgresql.org/docs/release/15.4/) | diff --git a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx index ee006e4afc9..055a4d019c1 100644 --- a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx +++ b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx @@ -1,17 +1,21 @@ --- -title: EDB Postgres Advanced Server 16.3 release notes -navTitle: "Version 16.3" +title: EDB Postgres Advanced Server 16.3.0 release notes +navTitle: "Version 16.3.0" --- Released: 9 May 2024 -EDB Postgres Advanced Server 16.3 includes the following enhancements and bug fixes: +EDB Postgres Advanced Server 16.3.0 includes the following enhancements and bug fixes: -| Type | Description | Category | -|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| -| Upstream merge | Merged with community PostgreSQL 16.3. Addresses CVE-2024-4317. See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. | -| Security |edbldr: check pg_read_server_files privilege before data file access. (#35906) -Permission to read data from the server File System should be restricted to superusers or users who possess the pg_read_server_files role. However, in affected versions of EPAS, any user can use edbldr to read data from any server file which is accessible to the OS user account under which EPAS is running. With this fix, a non-superuser using edbldr must either possess the pg_read_server_files role or must load data from standard input, rather than the server filesystem. Customer Advisory: CVE-2024-4545-for-db-2681 | -| Bug fix | -Fix assertion in DROP ROLE statement having duplicate names. | Bug | -| Security fix | Fixed a security vulnerability that could allow users using edbldr to bypass role permissions from pg_read_server_files. This could allow low privilege users to read files to which they would not otherwise have access. This issue is tracked as [CVE-2024-4545](linktoadvisory). | Security | \ No newline at end of file +| Type | Description | Addresses                | +|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| +| Upstream merge | Merged with community PostgreSQL 16.3. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/release/16.3/) for more information. | | +| Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sublink in `CONNECT BY` processing to avoid the server crash. | #102746 | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the freed path is still in use, specially in FDWs. | #86497 | +| Bug fix | Fixed an crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | +| Bug fix | Fixed an issue with possible data loss and `pg_dump` failures when using rowids. | #35901 | +| Bug fix | Fixed an issue with assertion in `DROP ROLE` statement having duplicate names. | | +| Buf fix | Fixed server crash with `edb_dblink_oci` in left join with whole row reference. | | diff --git a/product_docs/docs/epas/16/epas_rel_notes/index.mdx b/product_docs/docs/epas/16/epas_rel_notes/index.mdx index 11057b59f9d..f2dfda4fec7 100644 --- a/product_docs/docs/epas/16/epas_rel_notes/index.mdx +++ b/product_docs/docs/epas/16/epas_rel_notes/index.mdx @@ -2,7 +2,8 @@ navTitle: Release notes title: "EDB Postgres Advanced Server release notes" navigation: -- epas16_02_0_rel_notes +- epas16_3_0_rel_notes +- epas16_2_0_rel_notes - epas16_rel_notes --- @@ -13,6 +14,7 @@ The EDB Postgres Advanced Server documentation describes the latest version of E | Version | Release date | Upstream merges | | ------------------------ | ------------ | ---------------------------------------------------------- | +| [16.3.0](epas16_3_0_rel_notes) | 09 May 2024 | [16.3](https://www.postgresql.org/docs/16/release-16-3.html) | | [16.2](epas16_2_0_rel_notes)| 08 Feb 2024 | [16.2](https://www.postgresql.org/docs/16/release-16-2.html) | [16.1](epas16_rel_notes) | 09 Nov 2023 | [16.0](https://www.postgresql.org/docs/16/release-16.html),[16.1](https://www.postgresql.org/docs/release/16.1/) | diff --git a/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx b/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx index 23c767dc7f3..ce6b83e9182 100644 --- a/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx +++ b/product_docs/docs/pge/15/release_notes/rel_notes15.7.mdx @@ -9,7 +9,7 @@ New features, enhancements, bug fixes, and other changes in EDB Postgres Extende | Type | Description | | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Upstream merge | Merged with community PostgreSQL 15.7. Addresses CVE-2024-4137. See the [PostgreSQL 15 Release Notes](https://www.postgresql.org/docs/15/release-15-6.html) for more information. | +| Upstream merge | Merged with community PostgreSQL 15.7. Includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15 Release Notes](https://www.postgresql.org/docs/15/release-15-6.html) for more information. | diff --git a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx index 37504405e28..eb1f6406528 100644 --- a/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx +++ b/product_docs/docs/pge/16/release_notes/rel_notes16.3.mdx @@ -9,4 +9,4 @@ EDB Postgres Extended Server 16.3 includes the following enhancements and bug fi | Type | Description | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -| Upstream merge | Merged with community PostgreSQL 16.3. Addresses CVE-2024-4317. See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. +| Upstream merge | Merged with community PostgreSQL 16.3. Includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/16/release-16-3.html) for more information. From 78c7a834eaf3143c838216dcab541c250c49019f Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 08:55:26 +0100 Subject: [PATCH 44/84] Small tweaks to Upstream Merge lines Signed-off-by: Dj Walker-Morgan --- .../12/epas_rel_notes/epas12_19_24_rel_notes.mdx | 12 ++++++------ .../13/epas_rel_notes/epas13_15_21_rel_notes.mdx | 16 ++++++++-------- .../14/epas_rel_notes/epas14_12_0_rel_notes.mdx | 2 +- .../15/epas_rel_notes/epas15_7_0_rel_notes.mdx | 2 +- .../16/epas_rel_notes/epas16_3_0_rel_notes.mdx | 2 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --git a/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx b/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx index da22247b828..c48e0cebb7c 100644 --- a/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx +++ b/product_docs/docs/epas/12/epas_rel_notes/epas12_19_24_rel_notes.mdx @@ -7,11 +7,11 @@ Released: 9 May 2024 EDB Postgres Advanced Server 12.19.24 includes the following enhancements and bug fixes: -| Type | Description | Category | +| Type | Description | Addresses            | |----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| -| Upstream merge | Merged with community PostgreSQL 12.19. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 12.19 Release Notes](https://www.postgresql.org/docs/release/12.19/) for more information. | | -| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. [Support ticket: #36220] | | -| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. [Support ticket: #99282]| | -| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. [Support ticket: 102746] | | -| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs. [Support ticket: #86497] | | +| Upstream merge | Merged with community PostgreSQL 12.19. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 12.19 Release Notes](https://www.postgresql.org/docs/release/12.19/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs.| #86497 | | Bug fix | Fixed a crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | \ No newline at end of file diff --git a/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx b/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx index df27f2a9203..64a8db33c42 100644 --- a/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx +++ b/product_docs/docs/epas/13/epas_rel_notes/epas13_15_21_rel_notes.mdx @@ -7,11 +7,11 @@ Released: 9 May 2024 EDB Postgres Advanced Server 13.15.21 includes the following enhancements and bug fixes: -| Type | Description | Addresses                | -|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 13.15. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 13.15 Release Notes](https://www.postgresql.org/docs/release/14.15/) for more information. | | -| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | -| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | -| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | -| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs. | #86497 | -| Bug fix | Fixed a crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | +| Type | Description | Addresses                | +|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| +| Upstream merge | Merged with community PostgreSQL 13.15. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 13.15 Release Notes](https://www.postgresql.org/docs/release/14.15/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | +| Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | +| Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | +| Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | +| Bug fix | Added conditional free path in `add_path()` to avoid the rare possible server crashes when the free path is still in use, specially in the FDWs. | #86497 | +| Bug fix | Fixed a crash issue for `edbldr`. Now `edbldr` loads data into multiple tables with different encodings from the target database. | | diff --git a/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx b/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx index e1368f2829a..6017ec579d0 100644 --- a/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx +++ b/product_docs/docs/epas/14/epas_rel_notes/epas14_12_0_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 14.12.0 includes the following enhancements and bug | Type | Description | Addresses                | |----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 14.12. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 14.12 Release Notes](https://www.postgresql.org/docs/release/14.12/) for more information. | | +| Upstream merge | Merged with community PostgreSQL 14.12. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 14.12 Release Notes](https://www.postgresql.org/docs/release/14.12/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | | Bug fix | Fixed an issue to fetch all the attributes correctly from the sub link in `CONNECT BY` processing to avoid the server crash. | #102746 | diff --git a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx index 6fadd698179..9e354230428 100644 --- a/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx +++ b/product_docs/docs/epas/15/epas_rel_notes/epas15_7_0_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 15.7.0 includes the following enhancements and bug | Type | Description | Addresses                | |-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 15.7. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15.7 Release Notes](https://www.postgresql.org/docs/release/15.7/) for more information. | | +| Upstream merge | Merged with community PostgreSQL 15.7. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 15.7 Release Notes](https://www.postgresql.org/docs/release/15.7/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317)| | Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | diff --git a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx index 055a4d019c1..a84b5607245 100644 --- a/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx +++ b/product_docs/docs/epas/16/epas_rel_notes/epas16_3_0_rel_notes.mdx @@ -9,7 +9,7 @@ EDB Postgres Advanced Server 16.3.0 includes the following enhancements and bug | Type | Description | Addresses                | |-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| Upstream merge | Merged with community PostgreSQL 16.3. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/release/16.3/) for more information. | | +| Upstream merge | Merged with community PostgreSQL 16.3. Important: this release includes a fix for [CVE-2024-4317](/security/assessments/cve-2024-4317). See the [PostgreSQL 16.3 Release Notes](https://www.postgresql.org/docs/release/16.3/) for more information. | [CVE-2024-4317](/security/assessments/cve-2024-4317) | | Security fix | Fixed an issue for `edbldr`. Now `edbldr` checks the `pg_read_server_files` privilege before accessing the data files. | #35906, [CVE-2024-4545](/security/advisories/cve2024545/) | | Bug fix | Fixed an issue for `edb_filter_log`. Now it correctly redacts the password when the tab is used before the keyword. | #36220 | | Bug fix | Fixed an issue for `edb_audit` on Windows. Now it correctly rotates the log files based on days configured in `edb_audit_rotation_day`. | #99282 | From 41da019b5970e54b4e7b10230041ba7de6fe238d Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 9 May 2024 15:33:07 +0100 Subject: [PATCH 45/84] Various fixes Signed-off-by: Dj Walker-Morgan --- .../security/assessments/cve-2024-4317.mdx | 65 +++++++++++-------- advocacy_docs/security/assessments/index.mdx | 4 +- advocacy_docs/security/index.mdx | 4 +- .../epas_rel_notes/epas12_19_24_rel_notes.mdx | 2 +- .../epas_rel_notes/epas13_15_21_rel_notes.mdx | 2 +- .../epas_rel_notes/epas14_12_0_rel_notes.mdx | 2 +- .../epas_rel_notes/epas15_7_0_rel_notes.mdx | 2 +- .../epas_rel_notes/epas16_3_0_rel_notes.mdx | 2 +- .../pge/15/release_notes/rel_notes15.7.mdx | 2 +- .../pge/16/release_notes/rel_notes16.2.mdx | 2 +- .../pge/16/release_notes/rel_notes16.3.mdx | 2 +- 11 files changed, 49 insertions(+), 40 deletions(-) diff --git a/advocacy_docs/security/assessments/cve-2024-4317.mdx b/advocacy_docs/security/assessments/cve-2024-4317.mdx index f1441fcdbc4..d553264aa94 100644 --- a/advocacy_docs/security/assessments/cve-2024-4317.mdx +++ b/advocacy_docs/security/assessments/cve-2024-4317.mdx @@ -1,5 +1,5 @@ --- -title: CVE-2024-4317 - TBD +title: CVE-2024-4317 - Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner navTitle: CVE-2024-4317 affectedProducts: TBD --- @@ -12,17 +12,18 @@ Important: This is an assessment of the impact of CVE-2024-4317 on EDB products ## Summary -TBC +Missing authorization in PostgreSQL built-in views pg_stats_ext and pg_stats_ext_exprs allows an unprivileged database user to read most common values and other statistics from CREATE STATISTICS commands of other users. The most common values may reveal column values the eavesdropper could not otherwise read or results of functions they cannot execute. Installing an unaffected version only fixes fresh PostgreSQL installations, namely those that are created with the initdb utility after installing that version. Current PostgreSQL installations will remain vulnerable until they follow the instructions in the release notes, which are provided as a convenience in the below section. Within major versions 14-16, minor versions before PostgreSQL 16.3, 15.7, and 14.12 are affected. Versions before PostgreSQL 14 are unaffected. + ## Vulnerability details -CVE-ID: [CVE-2024-4317](https://nvd.nist.gov/vuln/detail/CVE-2024-4317) +CVE-ID: [CVE-2024-4317](https://www.postgresql.org/support/security/CVE-2024-4317/) -CVSS Base Score: TBC +CVSS Base Score: 3.1 -CVSS Temporal Score: TBC +CVSS Temporal Score: Undefined -CVSS Environmental Score: TBC +CVSS Environmental Score: Undefined CVSS Vector: TBC @@ -30,49 +31,56 @@ CVSS Vector: TBC ### PostgreSQL -TBC +* All versions of PostgreSQL prior to 16.3 +* All versions of PostgreSQL prior to 15.7 +* All versions of PostgreSQL prior to 14.12 ### EnterpriseDB Postgres Advanced Server (EPAS) -TBC + +* All versions of EPAS prior to 16.3 +* All versions of EPAS prior to 15.7 +* All versions of EPAS prior to 14.12 ### EnterpriseDB Postgres Extended -TBC + +* All versions of PGE prior to 16.3 +* All versions of PGE prior to 15.7 +* All versions of PGE prior to 14.12 ## Remediation/fixes +The fix is included in the following versions: 16.3, 15.7, and 14.12. + +Installing the fix will not remove the vulnerability from existing installations. To remove the vulnerability, follow the instructions in the [CVE-2024-4317](https://www.postgresql.org/support/security/CVE-2024-4317/) advisory. + ### PostgreSQL Version Information -| Affected Version | Fixed In | Fix Published | -|-------------------|----------|---------------| -| 15 | TBC | 2024-05-09 | -| 14 | TBC | 2024-05-09 | -| 13 | TBC | 2024-05-09 | -| 12 | TBC | 2024-05-09 | +| Affected Version | Fixed In | Fix Published | +|-----------------------------|----------|---------------| +| All versions prior to 16.3 | 16.3 | 2024-05-09 | +| All versions prior to 15.7 | 15.7 | 2024-05-09 | +| All versions prior to 14.12 | 14.12 | 2024-05-09 | ### EPAS Version Information -TBC - | Product | VRMF | Remediation/First Fix | |---------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------| -| EPAS | TBC | TBC | +| EPAS | All versions prior to 16.3 | Update to version 16.3 or later | +| EPAS | All versions prior to 15.7 | Update to version 15.7 or later | +| EPAS | All versions prior to 14.12 | Update to version 14.12 or later | + ### PGE Version Information | Product | VRMF | Remediation/First Fix | |---------|--------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| -| PGE | TBC | TBC | - -!!! Note -The exploit referred to in this CVE did not work on PostgreSQL 16. The -same defensive code as other releases has been added in PostgreSQL 16.2, EPAS -16.2 and PGE 16.2 to ensure strength in depth. We strongly recommend upgrading -your PostgreSQL 16, EPAS 16 and PGE 16 deployments to these versions. -!!! +| PGE | All versions prior to 16.3 | Update to version 16.3 or later | +| PGE | All versions prior to 15.7 | Update to version 15.7 or later | +| PGE | All versions prior to 14.12 | Update to version 14.12 or later | ## References * [CVSS Calculator v3.1](https://www.first.org/cvss/calculator/3.1) - +* [CWE-284 Improper Access Control](http://cwe.mitre.org/data/definitions/284.html) ## Related information @@ -85,8 +93,9 @@ Source: PostgreSQL.org ## Change history -## Disclaimer +9 May 2024: Original Copy Published +## Disclaimer This document is provided on an "as is" basis and does not imply any kind of guarantee or warranty, including the warranties of merchantability or fitness diff --git a/advocacy_docs/security/assessments/index.mdx b/advocacy_docs/security/assessments/index.mdx index 16450c0a7b7..9202d30d36e 100644 --- a/advocacy_docs/security/assessments/index.mdx +++ b/advocacy_docs/security/assessments/index.mdx @@ -31,12 +31,12 @@ The CVEs listed in this section are from PostgreSQL and other parties who have r   Read Assessment   Updated: 2024/05/09 -

TBD

+

Restrict visibility of "pg_stats_ext" and "pg_stats_ext_exprs" entries to the table owner

TBD
Summary:  -TBC +Missing authorization in PostgreSQL built-in views pg_stats_ext and pg_stats_ext_exprs allows an unprivileged database user to read most common values and other statistics from CREATE STATISTICS commands of other users. The most common values may reveal column values the eavesdropper could not otherwise read or results of functions they cannot execute. Installing an unaffected version only fixes fresh PostgreSQL installations, namely those that are created with the initdb utility after installing that version. Current PostgreSQL installations will remain vulnerable until they follow the instructions in the release notes, which are provided as a convenience in the below section. Within major versions 14-16, minor versions before PostgreSQL 16.3, 15.7, and 14.12 are affected. Versions before PostgreSQL 14 are unaffected.
Read More...