diff --git a/product_docs/docs/pgd/4/deployments/tpaexec/index.mdx b/product_docs/docs/pgd/4/deployments/tpaexec/index.mdx index 52a77648f65..a222351a13a 100644 --- a/product_docs/docs/pgd/4/deployments/tpaexec/index.mdx +++ b/product_docs/docs/pgd/4/deployments/tpaexec/index.mdx @@ -1,78 +1,19 @@ --- title: TPAexec navigation: -- overview -- configuration_syntax +- installing_tpaexec +- using_tpaexec - quick_start - --- The standard way of deploying EDB Distributed Postgres in a self managed setting, including physical and virtual machines, both self-hosted and in the cloud -(EC2) is to use EDB's deployment tool called TPAexec. +(EC2), is to use EDB's deployment tool called TPAexec. -TPAexec is an orchestration tool that uses Ansible to build Postgres clusters as specified by TPA (Trusted Postgres Architecture), a set of reference architectures that document how to set up and operate Postgres in various scenarios. TPA represents the best practices followed by EDB, and its recommendations are as applicable to quick testbed setups as to production environments. +TPAexec is an orchestration tool that uses Ansible to build Postgres clusters as specified by Trusted Postgres Architecture (TPA), a set of reference architectures that documents how to set up and operate Postgres in various scenarios. TPA represents the best practices followed by EDB and its recommendations are as applicable to quick testbed setups as to production environments. TPAexec packages are only available to EDB customers with an active BDR subscription. Please contact your account manager to request access. -## Capabilities and supported software - -### Configuration -The `tpaexec configure` command generates a simple YAML configuration file to describe a cluster, based on the options you select. The configuration is ready for immediate use, and you can modify it to better suit your needs. Editing the configuration file is the usual way to make any configuration changes to your cluster, both before and after it's created. - -The syntax is: - -``` -tpaexec configure --architecture --layout --bdr-version -- --2Q-repositories --extra-packages edb-livecompare edb-lasso --platform -``` - -The primary configuration options include: - -| Flags | Description | -| ---------------------------- | ----------- | -| --architecture BDR-Always-ON | Required. Set to 'BDR-Always-ON' for EDB Postgres Distributed deployments | -| --layout layoutname | Required. Specify one of the four supported architectures: bronze, silver, gold, and platinum. See [Choosing your architecture](/pgd/4/architectures) for more information. | -| --harp-consensus-protocol | Required. `bdr` is recommended if your layout is bronze or silver. `etcd` is recommended if your layout is gold or platinum. | -| --bdr-node-group groupname | Optional. Set the name of the BDR node group. The default is `bdrgroup`. | -| --bdr-database dbname | Optional. Set the name of the database with BDR enabled. The default is `bdrdb`. | -| --enable-camo | Optional. Set the pair of BDR primary instances in each region to be each other's CAMO partners. | - -For example: - -``` -[tpa]$ tpaexec configure ~/clusters/bdr \ - --architecture BDR-Always-ON \ - --layout gold \ - --harp-consensus-protocol etcd \ - --platform aws --region eu-west-1 --instance-type t3.micro \ - --distribution Debian-minimal -``` - -You may also specify any of the options described in the online help for configure-options: -``` -tpaexec help configure-options -``` - -### Provisioning -The `tpaexec provision` command creates instances and other resources required by the cluster. The details of the process depend on the architecture (for example, BDR-Always-ON) and platform (for example, AWS) that you selected while configuring the cluster. - -For example, given AWS access with the necessary privileges, TPAexec provisions EC2 instances, VPCs, subnets, routing tables, internet gateways, security groups, EBS volumes, elastic IPs, and so on. - -You can also "provision" existing servers by selecting the "bare" platform and providing connection details. Whether these are bare metal servers or those provisioned separately on a cloud platform, they can be used just as if they had been created by TPAexec. - -You are not restricted to a single platform—you can spread your cluster out across some AWS instances (in multiple regions) and some on-premise servers, or servers in other data centres, as needed. - -At the end of the provisioning stage, you will have the required number of instances with the basic operating system installed, which TPAexec can access via SSH (with sudo to root). - -### Deployment -The `tpaexec deploy` command installs and configures Postgres and other software on the provisioned servers (which may or may not have been created by TPAexec; but it doesn't matter who created them so long as SSH and sudo access is available). This includes setting up replication, backups, and so on. - -At the end of the deployment stage, Postgres is up and running. - -### Testing -The `tpaexec test` command executes various architecture and platform-specific tests against the deployed cluster to ensure that it is working as expected. - -At the end of the testing stage, you will have a fully-functioning cluster. ## Incremental changes TPAexec is carefully designed so that provisioning, deployment, and testing are idempotent. You can run through them, make a change to config.yml, and run through the process again to deploy the change. If nothing has changed in the configuration or on the instances, then rerunning the entire process does not change anything either. @@ -89,8 +30,11 @@ Once your cluster is up and running, TPAexec provides convenient cluster managem TPAexec can create complex clusters with many features configured, but the result is just Postgres. The installation follows some conventions designed to make life simpler, but there is no hidden magic or anything standing in the way between you and the database. You can do everything on a TPA cluster that you could do on any other Postgres installation. ## Getting started -See these topics to get complete your installation and then configure your first cluster: -- [TPAexec installation](https://documentation.enterprisedb.com/tpa/release/22.13-1/INSTALL/) for more detailed instructions -- [Quick start](quick_start) +See these topics to install TPAexec and use it to deploy your first EDB Postgres Distributed cluster: +- [Installing TPAexec](installing_tpaexec) +- [Using TPAexec](using_tpaexec) +- [Example: Deploying EDB Postgres Distributed](quick_start) + +For more information, see [TPAexec](https://documentation.enterprisedb.com/tpa/release/latest/). diff --git a/product_docs/docs/pgd/4/deployments/tpaexec/installing_tpaexec.mdx b/product_docs/docs/pgd/4/deployments/tpaexec/installing_tpaexec.mdx new file mode 100644 index 00000000000..43a8521983d --- /dev/null +++ b/product_docs/docs/pgd/4/deployments/tpaexec/installing_tpaexec.mdx @@ -0,0 +1,63 @@ +--- +title: "Installing TPAexec" +--- + +The TPAexec install package in available from the EDB customer portal. You must have a subscription to access the repository and the TPAexec install package. + +## Prerequisites + +### Access + +Setting up the repository requires root user permissions. If you do not have direct root access, log in as superuser. + +```shell +# To log in as a superuser: +sudo su - +``` + +If you are behind an HTTPS proxy, see [How to install RPM/APT repositories through a HTTPS proxy](https://techsupport.enterprisedb.com/kb/a/how-to-install-rpm-apt-repositories-through-a-https-proxy/). + + +### Token + +You need your token to complete the install process. It is available from the [EnterpriseDB customer portal](https://techsupport.enterprisedb.com/customer_portal/). From the left navigation pane, select **Company info > Company**. The account information for your company displays. Copy your token. + + + +## Installing TPAexec + +The examples in this section assume you are using TPAexec v22.14 and installing Postgres 14. + +1. Install the EnterpriseDB repository for TPAexec. Replace `token` with your company token. Replace `pgVersion` with the version of Postgres you are installing. + + Syntax: + ``` + curl https://techsupport.enterprisedb.com/api/repository//products/tpa/release//rpm | bash + ``` + + Example: + ``` + curl https://techsupport.enterprisedb.com/api/repository/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/products/tpa/release/14/rpm | bash + ``` + +2. Install TPAexec: + + - On RedHat + + ``` + yum install tpaexec + ``` + - On Debian/Ubuntu + ``` + apt-get install tpaexec + ``` + +3. Install additional dependencies: + ``` + /opt/EDB/TPA/bin/tpaexec setup + ``` +4. Verify the installation: + ``` + /opt/EDB/TPA/bin/tpaexec selftest + ``` + diff --git a/product_docs/docs/pgd/4/deployments/tpaexec/quick_start.mdx b/product_docs/docs/pgd/4/deployments/tpaexec/quick_start.mdx index 1bca2bbec65..a201991f10a 100644 --- a/product_docs/docs/pgd/4/deployments/tpaexec/quick_start.mdx +++ b/product_docs/docs/pgd/4/deployments/tpaexec/quick_start.mdx @@ -1,6 +1,6 @@ --- -title: "Quick start with TPAexec" -navTitle: "Quick start" +title: "Example: Deploying EDB Postgres Distributed" +navTitle: "Example: Deploying EDB Postgres Distributed" --- @@ -8,17 +8,27 @@ The following steps setup EDB Postgres Distributed with the Always On Silver architecture using Amazon EC2. 1. Generate a configuration file: + ``` - $ tpaexec configure myedbdpcluster --architecture BDR-Always-ON --layout Silver --platform aws + $ tpaexec configure myedbdpcluster --architecture BDR-Always-ON --layout Silver --platform aws --bdr-version 4 --2q --2Q-repositories products/2ndqpostgres/release ``` This creates a subdirectory directory in current working directory called `myedbdpcluster` containing the `config.yml` configuration file TPAexec uses to create the cluster. Edit the `config.yml` as needed, for example to change the IP address range used for servers or adjust locations of nodes. -1. Provision the cluster. : + We included options to specify using BDR 4 (the default version) and the 2ndQuadrant repositories to install EDB Postgres Extended Server. By default, TPAexec installs the 2ndQuadrant public repository and adds on any product repositories that the architecture requires. In this example, we specified `--2Q-repositories` so the complete list of 2ndQuadrant repositories is installed on each instance in addition to the 2ndQuadrant public repository. Since we did this, before the provisioning step, you must first export TPA_2Q_SUBSCRIPTION_TOKEN=xxx. You can get a subscription token from the [EnterpriseDB customer portal](https://techsupport.enterprisedb.com/customer_portal/) (**Support > Software subscriptions > Add**). + + ``` + cd ~/clusters/myedbdpcluster + + export TPA_2Q_SUBSCRIPTION_TOKEN= + ``` + +1. Provision the cluster: + ``` tpaexec provision myedbdpcluster ``` - Since we specified AWS as the platform, TCAexec provisions EC2 instances, VPCs, subnets, routing tables, internet gateways, security groups, EBS volumes, elastic IPs, and so on. + Since we specified AWS as the platform (the default platform), TCAexec provisions EC2 instances, VPCs, subnets, routing tables, internet gateways, security groups, EBS volumes, elastic IPs, and so on. 1. Deploy the needed packages, configuration and setup the actual EDB Postgres Distributed cluster: ``` @@ -30,4 +40,6 @@ After the successful run of the `deploy` command the cluster is ready to use. Yo It's also possible to run a test that ensures the cluster is running as expected: ``` tpaexec test myedbdpcluster -``` \ No newline at end of file +``` + + diff --git a/product_docs/docs/pgd/4/deployments/tpaexec/using_tpaexec.mdx b/product_docs/docs/pgd/4/deployments/tpaexec/using_tpaexec.mdx new file mode 100644 index 00000000000..74c92702795 --- /dev/null +++ b/product_docs/docs/pgd/4/deployments/tpaexec/using_tpaexec.mdx @@ -0,0 +1,145 @@ +--- +title: Using TPAexec + +--- + +With TPAexec you configure, provision, and deploy your EDB Postgres Distributed clusters. + +## Configure +The `tpaexec configure` command generates a simple YAML configuration file to describe a cluster, based on the options you select. The configuration is ready for immediate use and you can modify it to better suit your needs. Editing the configuration file is the usual way to make any configuration changes to your cluster both before and after it's created. + +The syntax is: + +``` +tpaexec configure --architecture --layout [options] +``` + +The required +configuration options include: + +| Flags | Description | +| ---------------------------- | ----------- | +| `--architecture` | Required. Set to `BDR-Always-ON` for EDB Postgres Distributed deployments. | +| `--layout` | Required. Specify one of the four supported architectures: bronze, silver, gold, and platinum. See [Choosing your architecture](/pgd/4/architectures) for more information. | +| `--harp-consensus-protocol` | Required. Use `bdr` if your layout is bronze or silver. Use `etcd` if your layout is gold or platinum. | + +For example: + +``` +[tpa]$ tpaexec configure ~/clusters/speedy \ + --architecture BDR-Always-ON \ + --layout gold \ + --harp-consensus-protocol etcd \ +``` + +The first argument must be the cluster directory, for example, `speedy` or `~/clusters/speedy` (the cluster is named `speedy` in both cases). We recommend that you keep all your clusters in a common directory, for example, `~/clusters`. The next argument must be `--architecture` to select an architecture, followed by `--layout`. + +The command creates a directory named ~/clusters/speedy and generates a configuration file named `config.yml` that follows the layout of the BDR-Always-ON architecture and gold layout. You can use the `tpcaexec info` command to see what values are supported for the configuration options based on what you specified when running the configure command. + +### Common configuration options + +Other configuration options include: + +#### Owner +Every cluster must be directly traceable to a person responsible for the provisioned resources. + +By default, a cluster is tagged as being owned by the login name of the user running `tpaexec provision`. If this name does not identify a person (for example, `postgres`, `ec2-user`), you must specify `--owner SomeId` to set an identifiable owner. + +You may use your initials, or "Firstname Lastname", or anything else that identifies you uniquely. + +#### Platform options +The default value for `--platform` is `aws`. It is the platform supported by the BDR-Always-ON architecture. + +Specify `--region` to specify any existing AWS region that you have access to (and that permits the required number of instances to be created). The default region is eu-west-1. + +Specify `--instance-type` with any valid instance type for AWS. The default is t3.micro. + +### Subnet selection +By default, each cluster is assigned a random /28 subnet under 10.33/16, but depending on the architecture, there may be one or more subnets, and each subnet may be anywhere between a /24 and a /29. + +Specify `--subnet` to use a particular subnet. For example, `--subnet 192.0.2.128/27`. + +Alternatively, specify `--subnet-pattern` to generate random subnets (as many as required by the architecture) matching the given pattern. For example, `--subnet-pattern 192.0.x.x`. + +### Disk space +Specify `--root-volume-size` to set the size of the root volume in GB. For example, `--root-volume-size 64`. The default is 16GB. (Depending on the image used to create instances, there may be a minimum size for the root volume.) + +For architectures that support separate postgres and barman volumes: + +Specify `--postgres-volume-size` to set the size of the Postgres volume in GB. The default is 16GB. + +Specify `--barman-volume-size` to set the size of the Barman volume in GB. The default is 32GB. + +### Distribution +Specify `--os` or `--distribution` to specify the OS to be used on the cluster's instances. The value is case-sensitive. + +The selected platform determines which distributions are available and which one is used by default. For more details, see `tpaexec info platforms/`. + +In general, you can use "Debian", "RedHat", and "Ubuntu" to select TPA images that have Postgres and other software preinstalled (to reduce deployment times). To use stock distribution images instead, append "-minimal" to the value, for example, `--distribution Debian-minimal`. + +### 2ndQuadrant repositories +By default, TPAexec installs the 2ndQuadrant public repository and adds on any product repositories that the architecture requires. + +Specify `--2Q-repositories source/name/release … ` to specify the complete list of 2ndQuadrant repositories to install on each instance in addition to the 2ndQuadrant public repository. + +If you do this, you must first export TPA_2Q_SUBSCRIPTION_TOKEN=xxx before you run tpaexec. You can get a subscription token from the [EnterpriseDB customer portal](https://techsupport.enterprisedb.com/customer_portal/) (**Support > Software subscriptions > Add**). + +### Software versions +By default TPAexec uses the latest major version of Postgres. Specify `--postgres-version` to install an earlier supported major version. + +By default, TPAexec always installs the latest version of every package. This is usually the desired behavior, but in some testing scenarios, it may be necessary to select specific package versions. For example, + +``` +--postgres-package-version 10.4-2.pgdg90+1 +--repmgr-package-version 4.0.5-1.pgdg90+1 +--barman-package-version 2.4-1.pgdg90+1 +--pglogical-package-version '2.2.0*' +--bdr-package-version '3.0.2*' +--pgbouncer-package-version '1.8*' +``` + +Specify `--extra-packages` or `--extra-postgres-packages` to install additional packages. The former lists packages to install along with system packages, while the latter lists packages to install later along with postgres packages. (If you mention packages that depend on Postgres in the former list, the installation fails because Postgres is not yet installed.) The arguments are passed on to the package manager for installation without any modifications. + +The `--extra-optional-packages` option behaves like `--extra-packages`, but it is not an error if the named packages cannot be installed. + +### Hostnames +By default, tpaexec configure randomly selects as many hostnames as it needs from a pre-approved list of several dozen names. This should be enough for most clusters. + +Specify `--hostnames-from` to select names from a different list (for example, if you need more names than are available in the canned list). The file must contain one hostname per line. + +Specify `--hostnames-pattern` to restrict hostnames to those matching the egrep-syntax pattern. If you choose to do this, you must ensure that the pattern matches only valid hostnames ([a-zA-Z0-9-]) and finds a sufficient number thereof. + +### Locations +By default, `tpaexec configure` uses the names first, second, and so on for any locations used by the selected architecture. + +Specify `--location-names` to provide more meaningful names for each location. + +### Enable Commit At Most Once +Specify `--enable-camo` to set the pair of BDR primary instances in each region to be each other's Commit At Most Once (CAMO) partners. See [Commit At Most Once (CAMO)](/bdr/latest/camo/) for more information. + + +## Provision +The `tpaexec provision` command creates instances and other resources required by the cluster. The details of the process depend on the architecture (for example, BDR-Always-ON) and platform (for example, AWS) that you selected while configuring the cluster. + +For example, given AWS access with the necessary privileges, TPAexec provisions EC2 instances, VPCs, subnets, routing tables, internet gateways, security groups, EBS volumes, elastic IPs, and so on. + +You can also "provision" existing servers by selecting the "bare" platform and providing connection details. Whether these are bare metal servers or those provisioned separately on a cloud platform, they can be used just as if they had been created by TPAexec. + +You are not restricted to a single platform—you can spread your cluster out across some AWS instances (in multiple regions) and some on-premise servers, or servers in other data centres, as needed. + +At the end of the provisioning stage, you will have the required number of instances with the basic operating system installed, which TPAexec can access via SSH (with sudo to root). + +## Deploy +The `tpaexec deploy` command installs and configures Postgres and other software on the provisioned servers (which may or may not have been created by TPAexec; but it doesn't matter who created them so long as SSH and sudo access is available). This includes setting up replication, backups, and so on. + +At the end of the deployment stage, EDB Postgres Distributed is up and running. + +## Test +The `tpaexec test` command executes various architecture and platform-specific tests against the deployed cluster to ensure that it is working as expected. + +At the end of the testing stage, you will have a fully-functioning cluster. + + +For more information, see [TPAexec](https://documentation.enterprisedb.com/tpa/release/latest/). + + diff --git a/product_docs/docs/pgd_cli/1/index.mdx b/product_docs/docs/pgd_cli/1/index.mdx index 777fc688ebb..08f964c8d7e 100644 --- a/product_docs/docs/pgd_cli/1/index.mdx +++ b/product_docs/docs/pgd_cli/1/index.mdx @@ -4,6 +4,7 @@ navTitle: "PGD CLI" indexCards: none navigation: - rel_notes + - requirements - installing_cli - using_cli - "#Command reference" @@ -34,7 +35,5 @@ You can use it to inspect and manage cluster resources and to get information on - clock skew - cluster health summary -## Supported version -The PGD CLI supports BDR 4.1 and later. diff --git a/product_docs/docs/pgd_cli/1/installing_cli.mdx b/product_docs/docs/pgd_cli/1/installing_cli.mdx index 78981db4e97..c7b995498fc 100644 --- a/product_docs/docs/pgd_cli/1/installing_cli.mdx +++ b/product_docs/docs/pgd_cli/1/installing_cli.mdx @@ -6,7 +6,7 @@ navTitle: "Installing the PGD CLI" ## Installing with TPAexec -If you are using TPAexec to deploy EDB Postgres Distributed and the default deployment installs the PGD CLI package (`edb-pgd-cli`) on each BDR node automatically. It also creates the configuration file (`/etc/edb/pgd-config.yml`) prepopulated with the endpoints of each node in the cluster so no additional configuration is necessary. +If you are using TPAexec (v22.14 or later) to deploy EDB Postgres Distributed, it installs the PGD CLI package (`edb-pgd-cli`) on each BDR node, by default. It also creates the configuration file (`/etc/edb/pgd-config.yml`) prepopulated with the endpoints of each node in the cluster so no additional configuration is necessary. If you wish to install the PGD CLI on any non-BDR instance in the cluster, you simply attach the pgdcli role to that instance in TPAexec's configuration file before deploying. See [TPAexec](/pgd/latest/deployments/tpaexec) for more information. diff --git a/product_docs/docs/pgd_cli/1/requirements.mdx b/product_docs/docs/pgd_cli/1/requirements.mdx new file mode 100644 index 00000000000..e7d9c02d635 --- /dev/null +++ b/product_docs/docs/pgd_cli/1/requirements.mdx @@ -0,0 +1,11 @@ +--- +title: Requirements +--- + +## Privileges + +The PGD CLI requires postgres superuser privileges to run. + +## Supported version + +The PGD CLI supports BDR 4.1 and later. \ No newline at end of file diff --git a/product_docs/docs/pgd_cli/1/using_cli.mdx b/product_docs/docs/pgd_cli/1/using_cli.mdx index 83eba8a3a65..683e4f697d3 100644 --- a/product_docs/docs/pgd_cli/1/using_cli.mdx +++ b/product_docs/docs/pgd_cli/1/using_cli.mdx @@ -23,9 +23,6 @@ See sample use cases for the following flags: - [`-o, --output`](#specifying-the-output-format) - [`-h, --help`](#accessing-the-command-line-help) -## Requirements - -The PGD CLI requires postgres superuser privileges to run. ## Specifying a configuration file