PEM with PgBouncer - customer issue

product_docs/docs/pem/9/considerations/pem_pgbouncer/configuring_pgBouncer.mdx

@@ -3,76 +3,92 @@ title: "Configuring PgBouncer"
 legacyRedirectsGenerated:
   # This list is generated by a script. If you need add entries, use the `legacyRedirects` key.
   - "/edb-docs/d/edb-postgres-enterprise-manager/installation-getting-started/pgbouncer-configuration-guide/8.0/configuring_pgBouncer.html"
+deepToC: true
 redirects:
     - /pem/latest/pem_pgbouncer/03_configuring_pgBouncer/
     - /pem/latest/pem_online_help/09_toc_pem_configure_pgbouncer/03_pem_pgbouncer_configuring_pgbouncer/
 ---
 
-You must configure PgBouncer to work with the PEM database server. 
+You must configure PgBouncer to work with the PEM database server.
 
-The name and location of the directories and files in the configuration steps depend on whether you installed the community version of PgBouncer or EDB PgBouncer. The instructions assume you are using EDB PgBouncer. If you have installed PgBouncer from the community repo, replace the names of the files and directories in the example with the PgBouncer values.
+## Prerequisites
 
+- If you are running EDB Postgres Advanced Server, you have installed [EDB PgBouncer](/pgbouncer/latest/installing). 
 
-| Name                | PgBouncer                          | EDB PgBouncer                          |
-|---------------------|------------------------------------|----------------------------------------|
-| PgBouncer directory | `/etc/pgbouncer<1.x>`              | `/etc/edb/pgbouncer<1.x>`               |
-| ini file            | `pgbouncer.ini`                    | `edb-pgbouncer.ini`                    |
-| userlist file       | `/etc/pgbouncer<1.x>/userlist.txt` | `/etc/edb/pgbouncer<1.x>/userlist.txt` |
-| HBA file            | `(/etc/pgbouncer<1.x>/hba_file)`   | `(/etc/edb/pgbouncer<1.x>/hba_file)`   |
-| Service file        | `pgbouncer-<1.x>`                  | `edb-pgbouncer-<1.x>`                  |
+  Or
 
-This example runs PgBouncer as the enterprisedb system user and outlines the process of configuring PgBouncer.
+- If you are running EDB Postgres Extended Server or PostgreSQL, you have installed community [PgBouncer](https://www.pgbouncer.org/install.html). 
+
+### EDB PgBouncer and PgBouncer installation considerations
+
+The name and location of the directories and files in the configuration steps, as well as the user, depend on whether you installed the community version of PgBouncer or EDB PgBouncer. If you have installed community PgBouncer (whether you install it from the community repo or the EDB repo), replace the names of the files and directories in the worked example with the values for PgBouncer.
+
+| Name                | PgBouncer                 | EDB PgBouncer                      |
+|---------------------|---------------------------|------------------------------------|
+| PgBouncer directory | `/etc/pgbouncer`          | `/etc/edb/pgbouncer<1.x>`          |
+| ini file            | `pgbouncer.ini`           | `edb-pgbouncer.ini`                |
+| HBA file            | `/etc/pgbouncer/hba_file` | `/etc/edb/pgbouncer<1.x>/hba_file` |
+| Service file        | `pgbouncer`               | `edb-pgbouncer-<1.x>`              |
+| User                | `postgres`                | `enterprisedb`                     |
+
+## Configuring PgBouncer
+
+This example configures EDB PgBouncer with the `enterprisedb` system user. 
+
+If you are running community PgBouncer, replace the names of the directories, files and user as explained in [Location of PgBouncer directories](#edb-pgbouncer-and-pgbouncer-installation-considerations) .
 
 1.  Open a terminal window and navigate to the PgBouncer directory.
 
-2.  Change the owner of the `etc` directory for PgBouncer (where `pgbouncer.ini` resides) to `enterprisedb`, and change the directory permissions to `0700`:
+1.  Change the owner of the `etc` directory for PgBouncer (where `edb-pgbouncer.ini` resides) to `enterprisedb`, and change the directory permissions to `0700`:
 
     ```shell
-    $ chown -R enterprisedb:enterprisedb /etc/edb/pgbouncer<1.x>
-    $ chmod 0700 /etc/edb/pgbouncer<1.x>
+    chown -R enterprisedb:enterprisedb /etc/edb/pgbouncer<1.x>
+    chmod 0700 /etc/edb/pgbouncer<1.x>
     ```
 
-3.  Change the contents of the `pgbouncer.ini` or `edb-pgbouncer.ini` file:
+1.  Change the contents of the `edb-pgbouncer.ini` file:
 
     ```ini
     [databases]
     ;; Change the pool_size according to maximum connections allowed
     ;; to the PEM database server as required.
     ;; 'auth_user' will be used for authenticate the db user (proxy
     ;; agent user in our case)
-
-    pem = port=5444 host=localhost dbname=pem auth_user=pgbouncer
-    pool_size=80 pool_mode=transaction
-    * = port=5444 host=localhost dbname=pem auth_user=pgbouncer
-    pool_size=10
+    pem = port=5444 host=127.0.0.1 dbname=pem auth_user=pgbouncer pool_size=80 pool_mode=transaction
+    * = port=5444 host=127.0.0.1 dbname=pem auth_user=pgbouncer pool_size=10
 
     [pgbouncer]
     logfile = /var/log/edb/pgbouncer<1.x>/edb-pgbouncer-<1.x>.log
     pidfile = /var/run/edb/pgbouncer<1.x>/edb-pgbouncer-<1.x>.pid
     listen_addr = *
     ;; Agent needs to use this port to connect the pem database now
     listen_port = 6432
-    ;; Require to support for the SSL Certificate authentications
+    ;; Set to require to ensure SSL certificates are used for connections
     ;; for PEM Agents
     client_tls_sslmode = require
     ;; These are the root.crt, server.key, server.crt files present
     ;; in the present under the data directory of the PEM database
     ;; server, used by the PEM Agents for connections.
-    client_tls_ca_file = /var/lib/edb/as11/data/root.crt
-    client_tls_key_file = /var/lib/edb/as11/data/server.key
-    client_tls_cert_file = /var/lib/edb/as11/data/server.crt
+    client_tls_ca_file = /var/lib/edb/as16/data/root.crt
+    client_tls_key_file = /var/lib/edb/as16/data/server.key
+    client_tls_cert_file = /var/lib/edb/as16/data/server.crt
+    ;; Allow pgBouncer to use pem_agent_pool certificate
+    ;; and key for connections to the server.
+    server_tls_key_file = /var/lib/edb/.postgresql/pem_agent_pool.key
+    server_tls_cert_file = /var/lib/edb/.postgresql/pem_agent_pool.crt
     ;; Use hba file for client connections
     auth_type = hba
-    ;; Authentication file, Reference:
-    ;; https://pgbouncer.github.io/config.html#auth_file
-    auth_file = /etc/edb/pgbouncer<1.x>/userlist.txt
     ;; HBA file
     auth_hba_file = /etc/edb/pgbouncer<1.x>/hba_file
     ;; Use pem.get_agent_pool_auth(TEXT) function to authenticate
     ;; the db user (used as a proxy agent user).
     auth_query = SELECT * FROM pem.get_agent_pool_auth($1)
     ;; DB User for administration of the pgbouncer
     admin_users = pem_admin1
+    ;; auth_dbname and auth_user allow
+    ;; admin console login by admin_users and stats_users
+    auth_dbname = pem
+    auth_user = pgbouncer
     ;; DB User for collecting the statistics of pgbouncer
     stats_users = pem_admin1
     server_reset_query = DISCARD ALL
@@ -83,70 +99,41 @@ This example runs PgBouncer as the enterprisedb system user and outlines the pro
     server_idle_timeout = 60
     ```
 
-!!! Note
-    For more information on `auth_user` see [Authentication settings](https://www.pgbouncer.org/config.html#authentication-settings).
-
-4.  Create and update the `/etc/edb/pgbouncer<1.x>/userlist.txt` authentication file for PgBouncer:
-
-    ```sql
-    ## Connect to pem database as a superuser, 
-    ## create the userslist.txt file and add 
-    ## username and their password list in CSV format
-    $ psql -p 5444 -d pem -U enterprisedb
-
-    pem=# 
-    COPY (
-    SELECT 'pgbouncer'::TEXT, 'pgbouncer_password'
-    UNION ALL
-    SELECT 'pem_admin1'::TEXT, 'pem_admin1_password')
-    TO '/etc/edb/pgbouncer<1.x>/userlist.txt'
-    WITH (FORMAT CSV, DELIMITER ' ', FORCE_QUOTE *);
-    __OUTPUT__
-    COPY 2
-    ```
-
-    This creates `/etc/edb/pgbouncer<1.x>/userlist.txt` file and adds the username and password list.
+    !!!note
+       For more information on `auth_user` see [Authentication settings](https://www.pgbouncer.org/config.html#authentication-settings).
+    !!!
 
-    !!! Note
-        If the pem_admin user is a superuser, you must add the password to the authentication file (`enterprisedb` in the example). This allows the pem_admin user to invoke the PEM authentication query function `pem.get_proxy_auth(text)`.
-
-5.  Create an HBA file `(/etc/edb/pgbouncer<1.x>/hba_file)` for PgBouncer that contains the following content:
+1.  Create an HBA file `(/etc/edb/pgbouncer<1.x>/hba_file)` for PgBouncer that contains the following content:
 
     ```ini
-    # Use authentication method md5 for the local connections to
-    # connect pem database & pgbouncer (virtual) database.
-    local pgbouncer all md5
-    # Use authentication method md5 for the remote connections to
-    # connect to pgbouncer (virtual database) using enterprisedb
-    # user.
-
-    host pgbouncer,pem pem_admin1 0.0.0.0/0 md5
-    ```
-
-    ```ini
-    # Use authentication method cert for the TCP/IP connections to
-    # connect the pem database using pem_agent_user1
-
+    # Use the authentication method scram-sha-256 for local connections
+    # between the pem database & the pgbouncer (virtual) database.
+    local pgbouncer all scram-sha-256
+    # Use the authentication method scram-sha-256 for remote connections
+    # to pgbouncer (virtual database) using the enterprisedb user.
+    host pgbouncer,pem pem_admin1 0.0.0.0/0 scram-sha-256
+    # Use the authentication method cert for TCP/IP connections
+    # to the pem database using pem_agent_user1
     hostssl pem pem_agent_user1 0.0.0.0/0 cert
     ```
 
-6.  Change the owner of the HBA file `(/etc/edb/pgbouncer<1.x>/hba_file)` to `enterprisedb`, and change the directory permissions to `0600`:
+1.  Change the owner of the HBA file `(/etc/edb/pgbouncer<1.x>/hba_file)` to `enterprisedb`, and change the directory permissions to `0600`:
 
     ```shell
-    $ chown enterprisedb:enterprisedb /etc/edb/pgbouncer<1.x>/hba_file
-    $ chmod 0600 /etc/edb/pgbouncer<1.x>/hba_file
+    chown enterprisedb:enterprisedb /etc/edb/pgbouncer<1.x>/hba_file
+    chmod 0600 /etc/edb/pgbouncer<1.x>/hba_file
     ```
 
-7.  Enable the PgBouncer service, and start the service:
+1.  Enable the PgBouncer service, and start the service:
 
     ```shell
-    $ systemctl enable edb-pgbouncer-<1.x>
+    systemctl enable edb-pgbouncer-<1.x>
     __OUTPUT__
     Created symlink from
     /etc/systemd/system/multi-user.target.wants/edb-pgbouncer-<1.x>.service
     to /usr/lib/systemd/system/edb-pgbouncer-<1.x>.service.
     ```
 
     ```shell
-    $ systemctl start edb-pgbouncer-<1.x>
+    systemctl start edb-pgbouncer-<1.x>
     ```

product_docs/docs/pem/9/considerations/pem_pgbouncer/configuring_the_pem_agent.mdx

@@ -8,16 +8,30 @@ redirects:
   - /pem/latest/pem_online_help/09_toc_pem_configure_pgbouncer/04_pem_pgbouncer_configuring_pem_agent/
 ---
 
-You can use an RPM package to install a PEM agent. For detailed installation information, see [Installating the PEM agent](../../installing_pem_agent/).
+## Prerequisites 
 
-Don't configure the PEM agent responsible for sending SNMP notifications with pgBouncer. For example, if the default PEM agent installed with PEM server is used for SNMP notifications, don't configure it with pgBouncer.
+You have [installed the PEM agent](../../installing_pem_agent/).
 
-## Configuring a new PEM agent (installed via RPM)
+!!!note 
+   Do not configure PEM agents with `enable_smtp`, `enable_snmp`, or `enable_webhook` set to `true` in the `agent.cfg` file to connect through PgBouncer. SNMP, SMTP, and Webhook spoolers use the LISTEN/NOTIFY mechanism provided by Postgres to send notifications asynchronously. Since PgBouncer doesn’t support the LISTEN/NOTIFY mechanism in transaction mode, connecting the agent to PgBouncer can cause notifications to be delayed or not delivered at all. Instead, connect the PEM agent directly to the PEM backend database.
+!!!
 
-After using an RPM package to install the PEM agent, you must configure it to work against a particular PEM database server. Use the following command:
+Now you can choose to [configure a new PEM agent](#configuring-a-new-pem-agent) or [use an existing PEM agent](#configuring-an-existing-pem-agent) for PgBouncer. 
+
+## Configuring a new PEM agent
+
+After installing the PEM agent, configure it to work with a particular PEM database server. Use the following command:
 
 ```shell
-$ PGSSLMODE=require PEM_SERVER_PASSWORD=pem_admin1_password /usr/edb/pem/agent/bin/pemworker --register-agent --pem-server 172.16.254.22 --pem-port 6432 --pem-user pem_admin1 --pem-agent-user pem_agent_user1 --display-name *Agent_Name*
+PGSSLMODE=require PEM_SERVER_PASSWORD=pem_admin1_password \
+      /usr/edb/pem/agent/bin/pemworker \
+      --register-agent \
+      --pem-server 172.16.254.22 \
+      --pem-port 6432 \
+      --pem-user pem_admin1 \
+      --pem-agent-user pem_agent_user1 \
+      --display-name *Agent_Name* \
+__OUTPUT__
 Postgres Enterprise Manager Agent registered successfully!
 ```
 
@@ -36,7 +50,7 @@ The PEM agent uses the keys to connect to the PEM database server as pem_agent_u
 A line mentioning the agent-user to use appears in the `agent.cfg` configuration file. For example:
 
 ```ini
-$ cat /usr/edb/pem/agent/etc/agent.cfg
+cat /usr/edb/pem/agent/etc/agent.cfg
 [PEM/agent]
 pem_host=172.16.254.22
 pem_port=6432
@@ -64,30 +78,30 @@ allow_batch_probes=false
 heartbeat_connection=false
 ```
 
-## Configuring an existing PEM agent (installed via RPM)
+## Configuring an existing PEM agent
 
 If you're using an existing PEM agent, you can copy the SSL certificate and key files to the target machine and reuse the files. You must modify the files, adding a new parameter and replacing some parameters in the existing `agent.cfg` file.
 
-Add a line to use agent_user as the agent:
+1.  Add a line to use agent_user as the agent:
 
-```ini
-agent_user=pem_agent_user1
-```
+    ```ini
+    agent_user=pem_agent_user1
+    ```
 
-Update the port to specify the pgBouncer port:
+1.  Update the port to specify the PgBouncer port:
 
-```ini
-pem_port=6432
-```
+    ```ini
+    pem_port=6432
+    ```
 
-Update the certificate and key path locations:
+1.  Update the certificate and key path locations:
 
-```ini
-agent_ssl_key=/root/.pem/pem_agent_user1.key
-agent_ssl_crt=/root/.pem/pem_agent_user1.crt
-```
+    ```ini
+    agent_ssl_key=/root/.pem/pem_agent_user1.key
+    agent_ssl_crt=/root/.pem/pem_agent_user1.crt
+    ```
 
-As an alternative, you can run the agent self-registration script. However, that process creates a new agent id. If run the agent self-registration script, you must replace the new agent id with the existing id and disable the entry for the new agent id in the `pem.agent` table. For example:
+As an alternative, you can run the agent self-registration script. However, that process creates a new agent id. If you run the agent self-registration script, you must replace the new agent id with the existing id and disable the entry for the new agent id in the `pem.agent` table. For example:
 
 ```sql
 pem=# UPDATE pem.agent SET active = false WHERE id = <new_agent_id>;

product_docs/docs/pem/9/considerations/pem_pgbouncer/index.mdx

@@ -1,5 +1,5 @@
 ---
-title: "Connection pooling using pgBouncer"
+title: "Connection pooling using PgBouncer"
 navTitle: "Deploying connection pooling"
 legacyRedirectsGenerated:
   # This list is generated by a script. If you need add entries, use the `legacyRedirects` key.
@@ -17,10 +17,11 @@ navigation:
 - configuring_the_pem_agent
 ---
 
-You can use pgBouncer as a connection pooler for limiting the number of connections from the PEM agent to the Postgres Enterprise Manager (PEM) server on non-Windows machine:
+You can use PgBouncer as a connection pooler for limiting the number of connections from the PEM agent to the Postgres Enterprise Manager (PEM) server on non-Windows machines:
 
--   Preparing the PEM Database Server provides information about preparing the PEM database server to be used with pgBouncer.
--   Configuring pgBouncer provides detailed information about configuring pgBouncer to make it work with the PEM database server.
--   Configuring the PEM agent provides detailed information about configuring a PEM agent to connect to pgBouncer.
+-   [PEM server and agent connection management mechanism](pem_server_pem_agent_connection_management_mechanism) provides an introduction of the PgBouncer-PEM infrastructure.
+-   [Preparing the PEM Database Server](preparing_the_pem_database_server) provides information about preparing the PEM database server to be used with PgBouncer.
+-   [Configuring PgBouncer](configuring_pgBouncer) provides detailed information about configuring PgBouncer to allow it to work with the PEM database server.
+-   [Configuring the PEM agent](configuring_the_pem_agent) provides detailed information about configuring a PEM agent to connect to PgBouncer.
 
-For detailed information about using the PEM web interface, see the [Accessing the web interface ](../../pem_web_interface).
+For detailed information about using the PEM web interface, see the [Accessing the web interface](../../pem_web_interface).

product_docs/docs/pem/9/considerations/pem_pgbouncer/pem_server_pem_agent_connection_management_mechanism.mdx

@@ -8,17 +8,23 @@ redirects:
   - /pem/latest/pem_online_help/09_toc_pem_configure_pgbouncer/01_pem_pgbouncer_server_agent_connection/
 ---
 
-Each PEM agent connects to the PEM database server using the SSL certificates for each user. For example, an agent with `ID#1` connects to the PEM database server using the agent1 user.
+## Without PgBouncer
 
-![Connecting to the PEM database without pgBouncer](../../images/pem_database_without_pgbouncer.png)
+In the default configuration, each PEM agent connects to the PEM database server directly using SSL for encryption. Each PEM agent uses its own dedicated user for the connection. 
 
-Prior to PEM version 7.5, the following limitations disallowed the use of the connection pooler between the PEM server and PEM agent:
+For example, a user agent with ID 1 connects to the PEM database server as agent1.
 
--   The PEM agent uses an SSL certificate to connect to the PEM database server.
--   It uses an individual user identifier when connecting to the PEM database server.
+![Connecting to the PEM database without PgBouncer](../../images/pem_without_pgbouncer.png)
 
-EDB modified the PEM agent to allow the agent to use a common database user (instead of the dedicated agent users) to connect to the PEM database server.
+## With PgBouncer
 
-![Connecting to pgBouncer.](../../images/pem_database_with_pgbouncer.png)
+In a PgBouncer-enabled environment, PEM agents cannot connect to the PEM database server directly. PEM agents must use a proxy user that you configure specifically for the connection to PgBouncer. In the example, the proxy user handling all PEM agent connections is called `pem_agent_user1`.
+
+Once the PEM agents connect to PgBouncer using SSL, PgBouncer is responsible for managing connection requests to the PEM database server. PgBouncer uses the rules you have configured for connection pooling to manage the incoming connection requests, for example, by respecting the established maximum number of active connections.
+
+![Connecting to pgBouncer.](../../images/pem_with_pgbouncer.png)
+
+!!!note 
+   Use PgBouncer version 1.9.0 or later as the connection pooler. Versions 1.9.0 or later support cert authentication.
+!!!
 
-We recommend using PgBouncer version 1.9.0 or later as the connection pooler. Versions 1.9.0 or later support cert authentication. PEM agents can connect to pgBouncer using SSL certificates.