Merge pull request #6054 from EnterpriseDB/release/2024-09-12a

Release: 2024-09-12a

advocacy_docs/edb-postgres-ai/analytics/quick_start.mdx

@@ -110,7 +110,7 @@ flavors in the installation when you connect.
 as Delta Tables. Every cluster comes pre-loaded to point to a storage bucket
 with benchmarking data inside (TPC-H, TPC-DS, Clickbench) at
 scale factors 1 and 10.
-* Only AWS is supported at the moment. Bring Your OWn Account (BYOA) is not supported.
+* Only AWS is supported at the moment. Bring Your Own Account (BYOA) is not supported.
 * You can deploy a cluster in any region that is activated in
 your EDB Postgres AI Account. Each region has a bucket with a copy of the
 benchmarking data, and so when you launch a cluster, it will use the

advocacy_docs/pg_extensions/query_advisor/index.mdx

@@ -10,7 +10,11 @@ navigation:
 - using
 ---
 
-EDB Query Advisor is a Postgres extension that provides index recommendations by keeping statistics on predicates found in WHERE statements, JOIN clauses, and workload queries. It analyzes the cost-benefit to actual workload queries by replanning them with the hypothetical index. Hypothetical indexes are created based on the predicates collected from the workload queries.
+EDB Query Advisor is a Postgres extension that includes the [Index Advisor](#query-advisor) and the [Statistics Advisor](#statistics-advisor). 
+
+## Index Advisor
+
+The Index Advisor provides index recommendations by keeping statistics on predicates found in WHERE statements, JOIN clauses, and workload queries. It analyzes the cost-benefit to actual workload queries by replanning them with the hypothetical index. Hypothetical indexes are created based on the predicates collected from the workload queries.
 
 This extension is useful if you want to analyze the most useful indexes that can benefit the workload queries without creating all possible indexes.
 
@@ -26,10 +30,22 @@ The extension saves the first query text, as is, for each distinct queryid execu
 
 The gathered data isn't saved when the Postgres server is restarted.
 
-See [Using EDB Query Advisor](./using) for a description of the `query_advisor_index_recommendations` function you use to generate the index recommendation.
+See [Using EDB Query Advisor](using/#query_advisor_index_recommendationsmin_filter-min_selectivity) for a description of the `query_advisor_index_recommendations` function you use to generate the index recommendation.
+
+## Statistics Advisor
+
+Statistics play a crucial role in determining the quality of query execution plans chosen by the database query optimizer. Sometimes there are dependencies between columns, and unless we create combined statistics using extended statistics, the optimizer assumes these columns are independent. This assumption often leads to incorrect cardinality estimates, which in turn, results in a poor execution plan, negatively impacting query performance.
+
+To address this, we monitor queries where multiple-column filters are applied and check for any selectivity estimation errors. If such errors are detected, we treat those multi-column filters as potential candidates for extended statistics.
+
+Whenever extended statistics recommendations are generated, we process these candidates by exploring various combinations of columns, especially when more than two columns are involved in the filters. Since estimation errors alone don’t definitively indicate which columns are dependent, we try all possible combinations. Currently, to limit the candidate pool, we focus on statistics for no more than two columns at a time.
+
+We also assign weights to each candidate, prioritising them based on how many queries would benefit from those extended statistics and what is the execution cost of the queries are.
+
+See [Using EDB Query Advisor](using/#query_advisor_statistics_recommendationsmin_err_estimate_num-min_err_estimate_ratio) for a description of the `query_advisor_statistics_recommendations` function you can use to generate the statistics recommendation.
 
 ## Limitations
 
-- Only single and two-column indexes are considered.
+- Only single and two-column indexes and statistics are considered.
 - Statistics aren't collected for utility commands.
 - The number of predicates and workload entries are static. A change requires a restart.

advocacy_docs/pg_extensions/query_advisor/rel_notes/index.mdx

@@ -2,16 +2,20 @@
 title: 'EDB Query Advisor release notes'
 navTitle: "Release notes"
 indexCards: none
+navigation:
+- query_advisor_1.1.1_rel_notes
+- query_advisor_1.0.0_rel_notes
 ---
 The EDB Query Advisor documentation describes the latest version of EDB Query Advisor,
 including minor releases and patches. These release notes
 cover what was new in each release. For new functionality introduced
 in a minor or patch release, there are also indicators in the content
 about the release that introduced the feature.
 
-|           Version           | Release Date |
-| --------------------------- | ------------ |
-| [1.0.0](query_advisor_1.0.0_rel_notes) | 10 May 2023 |
+| Version                                | Release Date |
+|----------------------------------------|--------------|
+| [1.1.1](query_advisor_1.1.1_rel_notes) | 12 Sep 2024  |
+| [1.0.0](query_advisor_1.0.0_rel_notes) | 10 May 2023  |
 
 
 

advocacy_docs/pg_extensions/query_advisor/rel_notes/query_advisor_1.1.1_rel_notes.mdx

@@ -0,0 +1,10 @@
+---
+title: Release notes for Query Advisor version 1.1.1
+navTitle: "Version 1.1.1"
+---
+
+This release of Query Advisor includes:
+
+|  Type   |                                Description                                 |
+| ------- | -------------------------------------------------------------------------- |
+| Feature | Added the Statistics Advisor.                                              |

advocacy_docs/pg_extensions/query_advisor/using.mdx

@@ -27,6 +27,29 @@ __OUTPUT__
 | CREATE INDEX ON partsupp USING btree (ps_suppkey);            | 50159616                | 13.254544                    |
 (3 rows)
 ```
+### query_advisor_statistics_recommendations(min_err_estimate_num, min_err_estimate_ratio)
+
+This function recommends potentially useful extended statistics by analyzing the statistics collected from the quals of user queries. 
+
+By default, `min_err_estimate_num` and `min_err_estimate_ratio` are set to `0`. You can use the `min_err_estimate_num` and `min_err_estimate_ratio` parameters to override the default.
+
+The function generates potential candidates from the multi-column filters of your queries. Then, these candidates are processed by exploring different possible combinations. Currently the focus is on statistics for two columns at a time. 
+
+It also shows the weights to each candidate. Weights are based on how many queries would benefit from those extended statistics and what the execution cost of the queries would be.
+
+For example:
+
+```sql
+# running as postgres user
+select * from query_advisor_statistics_recommendations();
+__OUTPUT__
+|                             statistics                                                 |       weight       |
++----------------------------------------------------------------------------------------+--------------------+
+| CREATE STATISTICS part_p_brand_p_container ON p_brand, p_container FROM public.part;   | 4940012.436935346  |
+| CREATE STATISTICS part_p_brand_p_type ON p_brand, p_type FROM public.part;             | 306202.2549564565  |
+| CREATE STATISTICS part_p_brand_p_size ON p_brand, p_size FROM public.part;             | 2879764.4054564573 |
+(3 rows)
+```
 
 ### query_advisor_qualstat
 This function returns the counts for every qualifier identified by the expression hash. This hash identifies each expression.
@@ -92,3 +115,21 @@ __OUTPUT__
  public     | pgbench_branches | bid         | pg_catalog.= |              |             |              |         10 |         2000000 |    1999990
  public     | t1               | id          | pg_catalog.= | public       | t2          | id_t1        |          1 |           10000 |       9999
 ```
+
+### query_advisor_statistics_recommendations
+
+It skips the weight column from `query_advisor_statistics_recommendations`. 
+
+For example:
+
+```sql
+# running as postgres user
+select * from query_advisor_statistics_recommendations;
+__OUTPUT__
+|                             statistics                                                 |
++----------------------------------------------------------------------------------------+
+| CREATE STATISTICS part_p_brand_p_container ON p_brand, p_container FROM public.part;          |
+| CREATE STATISTICS part_p_brand_p_type ON p_brand, p_type FROM public.part;                    |
+| CREATE STATISTICS part_p_brand_p_size ON p_brand, p_size FROM public.part;                    |
+(3 rows)
+```

advocacy_docs/supported-open-source/postgresql/installing/windows.mdx

@@ -27,10 +27,12 @@ Rather than use the EDB installer, you can also obtain a prebuilt installation p
 
 ## Installing PostgreSQL
 
-To perform an installation using the graphical installation wizard, you need superuser or administrator privileges. 
+To perform an installation using the graphical installation wizard, you need superuser or administrator privileges.
 
-!!! Note
-    If you're using the graphical installation wizard to perform a system upgrade, the installer preserves the configuration options specified during the previous installation.
+!!!note Notes
+   - EDB doesn't support all non-ASCII, multi-byte characters in user or machine names. Use ASCII characters only to avoid installation errors related to unsupported path names.
+   - If you're using the graphical installation wizard to perform a system ***upgrade***, the installer preserves the configuration options specified during the previous installation.
+!!!
 
 1.  To start the installation wizard, assume sufficient privileges, and double-click the installer icon. If prompted, provide a password. (In some versions of Windows, to invoke the installer with administrator privileges, you must select **Run as Administrator** from the installer icon's context menu.)
 

product_docs/docs/pgd/5/data_migration/edbloader.mdx

@@ -2,6 +2,7 @@
 title: EDB*Loader and PGD
 navTitle: EDB*Loader
 description: EDB*Loader is a high-speed data loading utility for EDB Postgres Advanced Server.
+deepToC: true
 ---
 
 [EDB\*Loader](/epas/latest/database_administration/02_edb_loader/) is a high-speed data loading utility for EDB Postgres Advanced Server. It provides an interface compatible with Oracle databases, allowing you to load data into EDB Postgres Advanced Server. It's designed to load large volumes of data into EDB Postgres Advanced Server quickly and efficiently.
@@ -14,6 +15,9 @@ As EDB\*Loader is a utility for EDB Postgres Advanced Server, it's available for
 
 ### Replication and EDB\*Loader
 
-As with EDB Postgres Advanced Server, EDB\*Loader works with PGD in a replication environment. But, unlike EDB Postgres Advanced Server with physical replication, it isn't possible to use the [direct path load method](/epas/latest/database_administration/02_edb_loader/invoking_edb_loader/direct_path_load/) to load data into the replica nodes. Only the node connected to by EDB\*Loader gets the data that EDB\*Loader is loading because the direct path load method skips use of the WAL, upon which logical replication relies. 
+As with EDB Postgres Advanced Server, EDB\*Loader works with PGD in a replication environment. You cannot use the direct load path method because the [direct path load method](/epas/latest/database_administration/02_edb_loader/invoking_edb_loader/direct_path_load/) skips use of the WAL, upon which all replication relies. That means that only the node connected to by EDB\*Loader gets the data that EDB\*Loader is loading and no data replicates to the other nodes.
+
+With PGD, you can make use of EDB\*loader's direct load path method by running it independently on each node. You can perform this either on one node at a time or in parallel to all nodes, depending on the use case. When using the direct path load method on multiple nodes, it's important to ensure there are no other writes happening to the table concurrently as this can result in inconsistencies. 
+
+
 
-With PGD it's possible to run the direct load path method to each node. This can be performed on one node at a time or in parallel to all nodes, depending on the use case. When doing this, it's important to ensure there are no other writes happening to the table concurrently as this can result in inconsistencies.

product_docs/docs/pgd/5/monitoring/sql.mdx

@@ -573,10 +573,10 @@ From time to time, Raft consensus starts a new election to define a
 new `RAFT_LEADER`. During an election, there might be an intermediary
 situation where there's no `RAFT_LEADER`, and some of the nodes consider
 themselves as `RAFT_CANDIDATE`. The whole election can't take longer
-than `bdr.raft_election_timeout` (by default it's set to 6 seconds). If
+than `bdr.raft_global_election_timeout` (by default it's set to 6 seconds). If
 the query above returns an in-election situation, then wait for
-`bdr.raft_election_timeout`, and run the query again. If after
-`bdr.raft_election_timeout` has passed and some the listed conditions are
+`bdr.raft_global_election_timeout`, and run the query again. If after
+`bdr.raft_global_election_timeout` has passed and some the listed conditions are
 still not met, then Raft consensus isn't working.
 
 Raft consensus might not be working correctly on only a single node.
@@ -587,15 +587,15 @@ make sure that:
 -   All PGD nodes are accessible to each other through both regular and
     replication connections (check file `pg_hba.conf`).
 -   PGD versions are the same on all nodes.
--   `bdr.raft_election_timeout` is the same on all nodes.
+-   `bdr.raft_global_election_timeout` is the same on all nodes.
 
 In some cases, especially if nodes are geographically distant from each
 other or network latency is high, the default value of
-`bdr.raft_election_timeout` (6 seconds) might not be enough. If Raft
+`bdr.raft_global_election_timeout` (6 seconds) might not be enough. If Raft
 consensus is still not working even after making sure everything is
-correct, consider increasing `bdr.raft_election_timeout` to 30
+correct, consider increasing `bdr.raft_global_election_timeout` to 30
 seconds on all nodes. For PGD 3.6.11 and later, setting
-`bdr.raft_election_timeout` requires only a server reload.
+`bdr.raft_global_election_timeout` requires only a server reload.
 
 Given how Raft consensus affects cluster operational tasks, and also as
 Raft consensus is directly responsible for advancing the group slot,

product_docs/docs/pgd/5/reference/functions.mdx

@@ -46,9 +46,9 @@ Returns the current subscription statistics.
 
 ## System and progress information parameters
 
-PGD exposes some parameters that you can query using `SHOW` in psql
-or using `PQparameterStatus` (or equivalent) from a client
-application.
+PGD exposes some parameters that you can query directly in SQL using e.g.
+`SHOW` or the `current_setting()` function, and/or using `PQparameterStatus`
+(or equivalent) from a client application.
 
 ### `bdr.local_node_id`
 
@@ -68,8 +68,10 @@ becomes remotely visible.
 
 ### `transaction_id`
 
-As soon as Postgres assigns a transaction id, if CAMO is enabled, this parameter is
-updated to show the transaction id just assigned.
+If a CAMO transaction is in progress, `transaction_id` will be updated to show
+the assigned transaction id. Note that this parameter can only be queried
+using `PQparameterStatus` or equivalent. See section [Application use](../durability/camo#application-use)
+for a usage example.
 
 ### `bdr.is_node_connected`
 
@@ -245,7 +247,7 @@ with full voting rights.
 
 If `wait_for_completion` is false, the request is served on
 a best-effort basis. If the node can't become a leader in the
-`bdr.raft_election_timeout` period, then some other capable node
+`bdr.raft_global_lection_timeout` period, then some other capable node
 becomes the leader again. Also, the leadership can change over the
 period of time per Raft protocol. A `true` return result indicates
 only that the request was submitted successfully.

product_docs/docs/pgd/5/reference/index.json

@@ -160,7 +160,7 @@
   "bdrglobal_keepalives_count": "/pgd/latest/reference/pgd-settings#bdrglobal_keepalives_count",
   "bdrglobal_tcp_user_timeout": "/pgd/latest/reference/pgd-settings#bdrglobal_tcp_user_timeout",
   "bdrraft_global_election_timeout": "/pgd/latest/reference/pgd-settings#bdrraft_global_election_timeout",
-  "bdrraft_local_election_timeout": "/pgd/latest/reference/pgd-settings#bdrraft_local_election_timeout",
+  "bdrraft_group_election_timeout": "/pgd/latest/reference/pgd-settings#bdrraft_group_election_timeout",
   "bdrraft_response_timeout": "/pgd/latest/reference/pgd-settings#bdrraft_response_timeout",
   "bdrraft_keep_min_entries": "/pgd/latest/reference/pgd-settings#bdrraft_keep_min_entries",
   "bdrraft_log_min_apply_duration": "/pgd/latest/reference/pgd-settings#bdrraft_log_min_apply_duration",

product_docs/docs/pgd/5/reference/index.mdx

@@ -228,7 +228,7 @@ The reference section is a definitive listing of all functions, views, and comma
  * [`bdr.global_tcp_user_timeout`](pgd-settings#bdrglobal_tcp_user_timeout)
 ### [Internal settings - Raft timeouts](pgd-settings#internal-settings---raft-timeouts)
  * [`bdr.raft_global_election_timeout`](pgd-settings#bdrraft_global_election_timeout)
- * [`bdr.raft_local_election_timeout`](pgd-settings#bdrraft_local_election_timeout)
+ * [`bdr.raft_group_election_timeout`](pgd-settings#bdrraft_group_election_timeout)
  * [`bdr.raft_response_timeout`](pgd-settings#bdrraft_response_timeout)
 ### [Internal settings - Other Raft values](pgd-settings#internal-settings---other-raft-values)
  * [`bdr.raft_keep_min_entries`](pgd-settings#bdrraft_keep_min_entries)

product_docs/docs/pgd/5/reference/pgd-settings.mdx

@@ -579,7 +579,7 @@ To account for network failures, the Raft consensus protocol implements timeouts
 for elections and requests. This value is used when a request is
 being sent to the global (top-level) group. The default is 6 seconds (6s).
 
-### `bdr.raft_local_election_timeout`
+### `bdr.raft_group_election_timeout`
 
 To account for network failures, the Raft consensus protocol implements timeouts
 for elections and requests. This value is used when a request is
@@ -589,7 +589,7 @@ being sent to the sub-group. The default is 3 seconds (3s).
 
 For responses, the settings of
 [`bdr.raft_global_election_timeout`](#bdrraft_global_election_timeout) and
-[`bdr.raft_local_election_timeout`](#bdrraft_local_election_timeout) are used
+[`bdr.raft_group_election_timeout`](#bdrraft_group_election_timeout) are used
 as appropriate. You can override this behavior by setting
 this variable. The setting of `bdr.raft_response_timeout` must be less than
 either of the election timeout values. Set this variable to -1 to disable the override.

product_docs/docs/tde/15/index.mdx

@@ -75,7 +75,9 @@ Data encryption and decryption is managed by the database and doesn't require ap
 
 EDB Postgres Advanced Server and EDB Postgres Extended Server provide hooks to key management that's external to the database. These hooks allow for simple passphrase encrypt/decrypt or integration with enterprise key management solutions. See [Securing the data encryption key](./key_stores) for more information.
 
-### How does TDE encrypt data? 
+### How does TDE encrypt data?
+
+EDB TDE uses [OpenSSL](https://openssl-library.org/) to encrypt data files with the AES encryption algorithm. In Windows systems, TDE uses [OpenSSL 3](https://docs.openssl.org/3.0/). In Linux systems, TDE uses the OpenSSL version installed in the host operating system. To check the installed version, run `openssl version`. For more information, see the [OpenSSL documentation](https://docs.openssl.org/master/). If you're using a custom build not provided by the OpenSSL community, consult your vendor's documentation.
 
 Starting with version 16, EDB TDE introduces the option to choose between AES-128 and AES-256 encryption algorithms during the initialization of the Postgres cluster. The choice between AES-128 and AES-256 hinges on balancing performance and security requirements. AES-128 is commonly advised for environments where performance efficiency and lower power consumption are pivotal, making it suitable for most applications. Conversely, AES-256 is recommended for scenarios demanding the highest level of security, often driven by regulatory mandates.