From 7d690ee2e8a567a13549d7fdf00ab136e4c655d4 Mon Sep 17 00:00:00 2001 From: Dj Walker-Morgan Date: Thu, 21 Sep 2023 09:00:19 +0100 Subject: [PATCH] Imported and rearranged sidebar Signed-off-by: Dj Walker-Morgan --- .../tpa/23/{reference => }/INSTALL-repo.mdx | 81 +++---- product_docs/docs/tpa/23/INSTALL.mdx | 2 +- .../docs/tpa/23/firstclusterdeployment.mdx | 222 ++++++++++++++++++ product_docs/docs/tpa/23/index.mdx | 3 + product_docs/docs/tpa/23/opensourcetpa.mdx | 32 +++ .../docs/tpa/23/reference/INSTALL-docker.mdx | 2 +- .../docs/tpa/23/reference/tpaexec-support.mdx | 4 +- .../docs/tpa/23/tpaexec-configure.mdx | 2 +- 8 files changed, 295 insertions(+), 53 deletions(-) rename product_docs/docs/tpa/23/{reference => }/INSTALL-repo.mdx (52%) create mode 100644 product_docs/docs/tpa/23/firstclusterdeployment.mdx create mode 100644 product_docs/docs/tpa/23/opensourcetpa.mdx diff --git a/product_docs/docs/tpa/23/reference/INSTALL-repo.mdx b/product_docs/docs/tpa/23/INSTALL-repo.mdx similarity index 52% rename from product_docs/docs/tpa/23/reference/INSTALL-repo.mdx rename to product_docs/docs/tpa/23/INSTALL-repo.mdx index 63d1b3f5010..28f1ac0ce2a 100644 --- a/product_docs/docs/tpa/23/reference/INSTALL-repo.mdx +++ b/product_docs/docs/tpa/23/INSTALL-repo.mdx @@ -1,4 +1,5 @@ --- +navTitle: Install from Source title: Installing TPA from source originalFilePath: INSTALL-repo.md @@ -9,78 +10,62 @@ repository. !!! Note - EDB customers must [install TPA from packages](../INSTALL/) in + EDB customers must [install TPA from packages](INSTALL/) in order to receive EDB support for the software. To run TPA from source, you must install all of the dependencies (e.g., Python 3.6+) that the packages would handle for you, or download -the source and [run TPA in a Docker container](INSTALL-docker/). +the source and [run TPA in a Docker container](reference/INSTALL-docker/). (Either way will work fine on Linux and macOS.) ## Quickstart -First, you must install the various dependencies that would have been -installed automatically along with the TPA packages. (You can use -something other than `sudo` to run these commands as root, if you -prefer.) +First, you must install the various dependencies Python 3, Python +venv, git, openvpn and patch. Installing from EDB repositories would +would install these automatically along with the TPA +packages. -```bash -# Debian or Ubuntu -$ sudo apt-get install python3 python3-pip python3-venv \ - git openvpn patch +Before you install TPA, you must install the required packages: -# RedHat, Rocky or AlmaLinux (python3 for RHEL7, python36 for RHEL8) -$ sudo yum install python36 python3-pip \ - epel-release git openvpn patch +- **Debian/Ubuntu**
`sudo apt-get install python3 python3-pip python3-venv git openvpn patch` +- **Redhat, Rocky or AlmaLinux (RHEL7)**
`sudo yum install python3 python3-pip epel-release git openvpn patch` +- **Redhat, Rocky or AlmaLinux (RHEL8)**
`sudo yum install python36 python3-pip epel-release git openvpn patch` -# MacOS X -$ brew tap discoteq/discoteq -$ brew install python@3 openvpn flock coreutils gpatch git -``` +## Clone and setup -Next, install TPA itself: +With prerequisites installed, you can now clone the repository. -```bash -$ git clone ssh://git@github.com/EnterpriseDB/tpa.git -$ ./tpa/bin/tpaexec setup -$ ./tpa/bin/tpaexec selftest ``` +git clone https://github.com/enterprisedb/tpa.git ~/tpa +``` + +This creates a `tpa` directory in your home directory. + +If you prefer to checkout with ssh use:
-## Step-by-step +``` +git clone ssh://git@github.com/EnterpriseDB/tpa.git ~/tpa +``` -Install the various dependencies as described above. +Add the bin directory, found within in your newly created clone, to your path with: -If your system does not have Python 3.6+ packages, you can use `pyenv` -to install a more recent Python in your home directory (see below), or -you can [run TPA in a Docker container](INSTALL-docker/). +`export PATH=$PATH:$HOME/tpa/bin` -Next, clone the TPA repository into, say, `~/tpa`. (It doesn't -matter where you put it, but don't use `/opt/EDB/TPA` or -`/opt/2ndQuadrant/TPA`, to avoid conflicts if you install the TPA -packages in future.) +Add this line to your `.bashrc` file (or other profile file for your preferred shell). -```bash -$ git clone ssh://git@github.com/EnterpriseDB/tpa.git ~/tpa -``` +You can now create a working tpa environment by running: -(If you're installing from source, please clone the repository instead -of downloading an archive of the source.) +`tpaexec setup` -The remaining steps are the same as if you had installed the package. +This will create the Python virtual environment that TPA will use in future. All needed packages are installed in this environment. To test this configured correctly, run the following: -```bash -# Add tpaexec to your PATH for convenience -# (Put this in your ~/.bashrc too) -$ export PATH=$PATH:$HOME/tpa/bin +`tpaexec selftest` -$ tpaexec setup -$ tpaexec selftest -``` +You now have tpaexec installed. -If the self-test completes without any errors, your TPA installation -is ready for use. +## Dependencies -## Python 3.6+ +### Python 3.6+ TPA requires Python 3.6 or later, available on most modern distributions. If you don't have it, you can use @@ -112,7 +97,7 @@ If you were not already using pyenv, please remember to add `pyenv` to your PATH in .bashrc and call `eval "$(pyenv init -)"` as described in the [pyenv documentation](https://github.com/pyenv/pyenv#installation). -## Virtual environment options +### Virtual environment options By default, `tpaexec setup` will use the builtin Python 3 `-m venv` to create a venv under `$TPA_DIR/tpa-venv`, and activate it diff --git a/product_docs/docs/tpa/23/INSTALL.mdx b/product_docs/docs/tpa/23/INSTALL.mdx index 6398a34f496..f7967e3f294 100644 --- a/product_docs/docs/tpa/23/INSTALL.mdx +++ b/product_docs/docs/tpa/23/INSTALL.mdx @@ -10,7 +10,7 @@ To use TPA, you need to install from packages or source and run the packages. If you have an EDB subscription plan, and therefore have access to the EDB repositories, you should follow these instructions. To install TPA from source, please refer to -[Installing TPA from Source](reference/INSTALL-repo/). +[Installing TPA from Source](INSTALL-repo/). See [Distribution support](reference/distributions/) for information on what platforms are supported. diff --git a/product_docs/docs/tpa/23/firstclusterdeployment.mdx b/product_docs/docs/tpa/23/firstclusterdeployment.mdx new file mode 100644 index 00000000000..3656599e79a --- /dev/null +++ b/product_docs/docs/tpa/23/firstclusterdeployment.mdx @@ -0,0 +1,222 @@ +--- +navTitle: Tutorial +title: A First Cluster Deployment +originalFilePath: firstclusterdeployment.md + +--- + +In this short tutorial, we are going to work through deploying a simple [M1 architecture](architecture-M1/) deployment onto a local Docker installation. By the end you will have four containers, one primary database, two replicas and a backup node, configured and ready for you to explore. + +For this example, we will run TPA on an Ubuntu system, but the considerations are similar for most Linux systems. + +### Installing TPA + +If you're an EDB customer, you'll want to follow the [EDB Repo instructions](INSTALL/) which will install the TPA packages straight from EDB's repositories. + +If you are an open source user of TPA, there's [instructions on how to build from the source](INSTALL-repo/) which you can download from Github.com. + +Follow those guides and then return here. + +### Installing Docker + +As we said, We are going to deploy the example deployment onto Docker and unless you already have Docker installed we'll need to set that up. + +On Debian or Ubuntu, install Docker by running: + +``` +sudo apt update +sudo apt install docker.io +``` + +For other Linux distributions, consult the [Docker Engine Install page](https://docs.docker.com/engine/install/). + +You will want to add your user to the docker group with: + +``` +sudo usermod -aG docker +newgrp docker +``` + +### CgroupVersion + +Currently, TPA requires Cgroups Version 1 be configured on your system, + +Run: + +``` +mount | grep cgroup | head -1 +``` + +and if you do not see a reference to `tmpfs` in the output, you'll need to disable cgroups v2. + +Run: + +``` +echo 'GRUB_CMDLINE_LINUX=systemd.unified_cgroup_hierarchy=false' | sudo tee /etc/default/grub.d/cgroup.cfg +``` + +To make the appropriate changes, then update Grub and reboot your system with: + +``` +sudo update-grub +sudo reboot +``` + +!!! Warning + + Giving a user the ability to speak to the Docker daemon + lets them trivially gain root on the Docker host. Only trusted users + should have access to the Docker daemon. + +### Creating a configuration with TPA + +The next step in this process is to create a configuration. TPA does most of the work for you through its `configure` command. All you have to do is supply command line flags and options to select, in broad terms, what you want to deploy. Here's our `tpaexec configure` command: + +``` +tpaexec configure demo --architecture M1 --platform docker --postgresql 15 --enable-repmgr --no-git +``` + +This creates a configuration called `demo` which has the [M1 architecture](architecture-M1/). It will therefore have a primary, replica and backup node. + +The `--platform docker` tells TPA that this configuration should be created on a local Docker instance; it will provision all the containers and OS requirements. Other platforms include [AWS](platform-aws), which does the same with Amazon Web Services and [Bare](platform-bare), which skips to operating system provisioning and goes straight to installing software on already configured Linux hosts. + +With `--postgresql 15`, we instruct TPA to use Community Postgres, version 15. There are several options here in terms of selecting software, but this is the most straightforward default for open-source users. + +Adding `--enable-repmgr` tells TPA to use configure the deployment to use [Replication Manager](https://www.repmgr.org/) to hand replication and failover. + +Finally, `--no-git` turns off the feature in TPA which allows you to revision control your configuration through git. + +Run this command, and apparently, nothing will happen on the command line. But you will find a directory called `demo` has been created containing some files including a `config.yml` file which is a blueprint for our new deployment. + +## Provisioning the deployment + +Now we are ready to create the containers (or virtual machines) on which we will run our new deployment. This can be achieved with the `provision` command. Run: + +``` +tpaexec provision demo +``` + +You will see TPA work through the various operations needed to prepare for deployment of your configuration. + +## Deploying + +Once provisioned, you can move on to deployment. This installs, if needed, operating systems and system packages. It then installs the requested Postgres architecture and performs all the needed configuration. + +``` +tpaexec deploy demo +``` + +You will see TPA work through the various operations needed to deploy your configuration. + +## Testing + +You can quickly test your newly deployed configuration using the tpaexec `test` command which will run pgbench on your new database. + +``` +tpaexec test demo +``` + +## Connecting + +To get to a psql prompt, the simplest route is to log into one of the containers (or VMs or host depending on configuration) using docker or SSH. Run + +``` +tpaexec ping demo +``` + +to ping all the connectable hosts in the deployment: You will get output that looks something like: + +``` +$ tpaexec ping demo +unfair | SUCCESS => { + "changed": false, + "ping": "pong" +} +uptake | SUCCESS => { + "changed": false, + "ping": "pong" +} +quondam | SUCCESS => { + "changed": false, + "ping": "pong" +} +uptight | SUCCESS => { + "changed": false, + "ping": "pong" +} +``` + +Select one of the nodes which responded with `SUCCESS`. We shall use `uptake` for this example. + +If you are only planning on using docker, use the command `docker exec -it uptake /bin/bash`, substituting in the appropriate hostname. + +Another option, that works with all types of TPA deployment is to use SSH. To do that, first change current directory to the created configuration directory. + +For example, our configuration is called demo, so we go to that directory. In there, we run `ssh -F ssh_config ourhostname` to connect. + +``` +cd demo +ssh -F ssh_config uptake +Last login: Wed Sep 6 10:08:01 2023 from 172.17.0.1 +[root@uptake ~]# +``` + +In both cases, you will be logged in as a root user on the container. + +We can now change user to the `postgres` user using `sudo -iu postgres`. As `postgres` we can run `psql`. TPA has already configured that user with a `.pgpass` file so there's no need to present a password. + +``` +[root@uptake ~]# +postgres@uptake:~ $ psql +psql (15.4) +Type "help" for help. + +postgres=# +``` + +And we are connected to our database. + +You can connect from the host system without SSHing into one of the containers. Obtain the IP address of the host you want to connect to from the `ssh_config` file. + +``` +$ grep "^ *Host" demo/ssh_config +Host * +Host uptight + HostName 172.17.0.9 +Host unfair + HostName 172.17.0.4 +Host quondam + HostName 172.17.0.10 +Host uptake + HostName 172.17.0.11 +``` + +We are going to connect to uptake, so the IP address is 172.17.0.11. + +You will also need to retrieve the password for the postgres user too. Run `tpaexec show-password demo postgres` to get the stored password from the system. + +``` +tpaexec show-password demo postgres +a9LmI1X^uMOpPoEnLuRdL%L$oRQak3om +``` + +Assuming you have a Postgresql client installed, you can then run: + +``` +psql --host 172.17.0.11 -U postgres +Password for user postgres: +``` + +Enter the password you previously retrieved. + +``` +psql (14.9 (Ubuntu 14.9-0ubuntu0.22.04.1), server 15.4) +WARNING: psql major version 14, server major version 15. + Some psql features might not work. +SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off) +Type "help" for help. + +postgres=# +``` + +You are now connected from the Docker host to Postgres running in one of the TPA deployed Docker containers. diff --git a/product_docs/docs/tpa/23/index.mdx b/product_docs/docs/tpa/23/index.mdx index f7dfb75ecfb..06019dc8b2b 100644 --- a/product_docs/docs/tpa/23/index.mdx +++ b/product_docs/docs/tpa/23/index.mdx @@ -3,7 +3,10 @@ navigation: - index - rel_notes - INSTALL + - opensourcetpa + - INSTALL-repo - '#Creating a cluster' + - firstclusterdeployment - tpaexec-configure - tpaexec-provision - tpaexec-deploy diff --git a/product_docs/docs/tpa/23/opensourcetpa.mdx b/product_docs/docs/tpa/23/opensourcetpa.mdx new file mode 100644 index 00000000000..831ed557ee5 --- /dev/null +++ b/product_docs/docs/tpa/23/opensourcetpa.mdx @@ -0,0 +1,32 @@ +--- +navTitle: Open Source +title: Open source TPA +originalFilePath: opensourcetpa.md + +--- + +## What is Trusted Postgres Architect (TPA)? + +TPA is an orchestration tool developed by [EnterpriseDB (EDB)](https://www.enterprisedb.com/) that uses Ansible to deploy Postgres clusters according to EDB's recommendations. + +TPA embodies the best practices followed by EDB, informed by many years of hard-earned experience with deploying and supporting Postgres. These recommendations are as applicable to quick testbed setups as to production environments. + +## Next Steps + +- [Installing TPA from Source](INSTALL-repo/) +- [Deploying your first cluster](firstclusterdeployment/) +- [TPA's full documentation online](https://www.enterprisedb.com/docs/tpa/latest/) + +## TPA Open Source FAQs + +### Can I use this if I'm not an EDB customer? + +Yes, TPA is an open source project under the GPLv3 license. It supports deploying clusters comprised of open source software, or EDB's proprietary products, or combinations. + +### Can I report an issue? + +Yes, if you're an EDB customer then please contact support. Otherwise please open a GitHub Issue. + +### Can I contribute? + +Sure, we'd love to hear from you but please open an issue before you start coding. We are quite selective with what TPA can/should do so bug fixes are more likely to get accepted than new features. diff --git a/product_docs/docs/tpa/23/reference/INSTALL-docker.mdx b/product_docs/docs/tpa/23/reference/INSTALL-docker.mdx index 34b1b2be4cd..c96498b79ba 100644 --- a/product_docs/docs/tpa/23/reference/INSTALL-docker.mdx +++ b/product_docs/docs/tpa/23/reference/INSTALL-docker.mdx @@ -6,7 +6,7 @@ originalFilePath: INSTALL-docker.md If you are using a system for which there are no [TPA packages](../INSTALL/) available, and it's difficult to run TPA after -[installing from source](INSTALL-repo/) (for example, because it's not +[installing from source](../INSTALL-repo/) (for example, because it's not easy to obtain a working Python 3.6+ interpreter), your last resort may be to build a Docker image and run TPA inside a Docker container. diff --git a/product_docs/docs/tpa/23/reference/tpaexec-support.mdx b/product_docs/docs/tpa/23/reference/tpaexec-support.mdx index 383ead3df73..196c050140f 100644 --- a/product_docs/docs/tpa/23/reference/tpaexec-support.mdx +++ b/product_docs/docs/tpa/23/reference/tpaexec-support.mdx @@ -11,9 +11,9 @@ originalFilePath: tpaexec-support.md TPA can install and configure the following major components. -- Postgres 15, 14, 13, 12, 11 +- Postgres 16, 15, 14, 13, 12, 11 -- EPAS (EDB Postgres Advanced Server) 15, 14, 13, 12 +- EPAS (EDB Postgres Advanced Server) 16, 15, 14, 13, 12 - PGD 5, 4, 3.7 diff --git a/product_docs/docs/tpa/23/tpaexec-configure.mdx b/product_docs/docs/tpa/23/tpaexec-configure.mdx index d1e72dd042d..df8b0027ba5 100644 --- a/product_docs/docs/tpa/23/tpaexec-configure.mdx +++ b/product_docs/docs/tpa/23/tpaexec-configure.mdx @@ -251,7 +251,7 @@ details. #### Postgres flavour and version TPA supports PostgreSQL, EDB Postgres Extended, and EDB Postgres -Advanced Server (EPAS) versions 11 through 15. +Advanced Server (EPAS) versions 11 through 16. You must specify both the flavour (or distribution) and major version of Postgres to install, for example: