Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting up templatized install instructions for Data Migration Service #6020

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
e960e75
JDBC - Advance Queuing update
nidhibhammar Sep 6, 2024
502ec6e
Added note on non-ascii characters not being supported
gvasquezvargas Sep 4, 2024
42fbf93
changed wording to reflect that EDB doesn't support all non-ascii cha…
gvasquezvargas Sep 9, 2024
2559801
Added more content
nidhibhammar Sep 9, 2024
61c6f25
TPA 23.34.1 rel notes
sonotley Sep 9, 2024
dc548c3
Merge pull request #6034 from EnterpriseDB/docs/tpa/rel-23.34.1
djw-m Sep 9, 2024
9f611a7
Merge pull request #6036 from EnterpriseDB/release-2024-09-09a
djw-m Sep 9, 2024
2e885cd
TPA patch for 23.24.1
djw-m Sep 9, 2024
bd330bc
Merge pull request #6037 from EnterpriseDB/djw-m-tpa-patch
djw-m Sep 9, 2024
76dddca
TDE: specify AES implemmentation is done with OpenSSL
gvasquezvargas Sep 6, 2024
2dc288d
duplication of OS and machine, adapted wording
gvasquezvargas Sep 6, 2024
a294ca8
Added more implicit info after discussing with Matt W.
gvasquezvargas Sep 6, 2024
7bd2713
Implemented feedback from Adam
gvasquezvargas Sep 10, 2024
d55a88e
Merge pull request #6038 from EnterpriseDB/2024-09-10a
gvasquezvargas Sep 10, 2024
4d385c7
Fixed references to bdr.raft_local_election_timeout in the docs.
djw-m Sep 10, 2024
c397374
Added remaining content and release notes
nidhibhammar Sep 10, 2024
33ce453
Minor edit
nidhibhammar Sep 10, 2024
57583fb
Rework
djw-m Sep 10, 2024
7c127d8
Fixed heading levels
nidhibhammar Sep 10, 2024
ba23aad
Updated to latest version
nidhibhammar Sep 10, 2024
df3881e
fixed blank spaces
nidhibhammar Sep 10, 2024
c1df12f
updated bullets
nidhibhammar Sep 10, 2024
fb4217a
fixed more blank spaces
nidhibhammar Sep 10, 2024
5430ff6
Added numbers
nidhibhammar Sep 10, 2024
99ba39c
Merge pull request #6031 from EnterpriseDB/content/docs/jdbc/advance_…
nidhibhammar Sep 10, 2024
fa53f85
more changes
djw-m Sep 10, 2024
e4876aa
Fix typo (missing space)
josh-heyer Sep 10, 2024
dedca3f
Merge pull request #6042 from EnterpriseDB/release/2024-09-10b
gvasquezvargas Sep 10, 2024
8eaafb2
Merge pull request #6041 from EnterpriseDB/DOCS-798-redux
djw-m Sep 10, 2024
05ef6df
Merge pull request #6024 from EnterpriseDB/postgresql/installer_limit…
gvasquezvargas Sep 10, 2024
4e9c345
Merge pull request #6030 from EnterpriseDB/TDE/openssl2
gvasquezvargas Sep 10, 2024
244099e
PGD: fix references to deprecated GUC "bdr.raft_election_timeout"
ibarwick Sep 11, 2024
897b06c
Merge pull request #6045 from EnterpriseDB/dev/pgd/DOCS-1024-fix-raft…
djw-m Sep 11, 2024
e986298
PGD: improve description of "transaction_id" pseudo-GUC
ibarwick Sep 12, 2024
e09f598
Merge pull request #6048 from EnterpriseDB/dev/pgd/DOCS-1028-transact…
djw-m Sep 12, 2024
eb01b77
Merge pull request #6039 from EnterpriseDB/DOCS-1021-pgd-5-x-bdr-raft…
djw-m Sep 12, 2024
15cac20
Update quick_start.mdx
kunliuedb Sep 12, 2024
0bd95eb
Merge pull request #6050 from EnterpriseDB/kunliuedb-patch-1
djw-m Sep 12, 2024
fd21648
Rebuild with test DMS content and fix right hand TOCs
djw-m Aug 27, 2024
edb4f57
Remove /2/ in supported versions
djw-m Aug 27, 2024
2846ec1
Refix links in supported versions
djw-m Aug 27, 2024
745ba08
Squashed commit of the following:
gvasquezvargas Aug 27, 2024
188a169
Last pass of edits after testing
gvasquezvargas Aug 28, 2024
47b14b8
Removed installation/removal steps as they are empty as discussed wit…
gvasquezvargas Aug 29, 2024
b6ee58e
Fixed step-by-step descriptions in index page for migration workflow
gvasquezvargas Sep 2, 2024
90700ff
Minor edits to getting started section
gvasquezvargas Sep 2, 2024
d1db7eb
Minor edits remaining sections
gvasquezvargas Sep 3, 2024
7e69d51
Removed upgrade pages, as no upgrades are available now
gvasquezvargas Sep 4, 2024
8854ff9
Clarification on page
gvasquezvargas Sep 10, 2024
3027b67
Merge pull request #6043 from EnterpriseDB/dms/migration_stop
gvasquezvargas Sep 11, 2024
3cb3526
DMS: added purl for UI link
gvasquezvargas Sep 11, 2024
150ce10
Merge pull request #6046 from EnterpriseDB/DMS/add_purl
gvasquezvargas Sep 11, 2024
10c84c0
Implementing feedback from Matt p.1
gvasquezvargas Sep 12, 2024
4ca76a4
Removed outdated release notes
gvasquezvargas Sep 12, 2024
2f7ebda
Reorganized terminology section
gvasquezvargas Sep 12, 2024
c7743eb
Added supported target databases on compatibility page
gvasquezvargas Sep 12, 2024
2c70ef0
DevGuide POC
djw-m Aug 13, 2024
5bef349
placeholder developer content
djw-m Aug 13, 2024
e1cf33e
Fix typo
djw-m Aug 13, 2024
dfd4f5e
Add singular icons for indexy links
djw-m Aug 14, 2024
690fc72
Testing redirects as a page transfer mechanism
djw-m Aug 15, 2024
21b9b96
Test 2
djw-m Aug 15, 2024
cfd18a7
Test 2 Part 2
djw-m Aug 15, 2024
7fc4a09
Make database links through links to versioned content
djw-m Aug 19, 2024
1913a58
WIP reorg
djw-m Aug 23, 2024
e2f8ac3
More reorg - pass 1 complete
djw-m Aug 23, 2024
919625f
responsive homepage
josh-heyer Aug 23, 2024
276720b
Fully remove Masonry
josh-heyer Aug 23, 2024
162fe93
Rebuild with test DMS content and fix right hand TOCs
djw-m Aug 27, 2024
5078f46
Remove sticky debug
djw-m Aug 27, 2024
b44b9f9
Page tuneup for front page - dividers are now functions
djw-m Aug 29, 2024
d787cdf
first steps to setting up templatized install instructions for Data M…
dwicinas Sep 3, 2024
37bab63
Fixed name and short name
dwicinas Sep 4, 2024
b9d8ba4
added and deleted versions
dwicinas Sep 4, 2024
eb675e7
Removing generated files that were erroneously added
dwicinas Sep 4, 2024
241b401
template revisions for RHEL9
dwicinas Sep 4, 2024
d1492a0
Updated template for RHEL 8 and Alma Linux 9
dwicinas Sep 11, 2024
6af2a0d
Revised deployment path for generated files
dwicinas Sep 11, 2024
78ee53e
Updated templates for SLES 15 and 12
dwicinas Sep 11, 2024
09c7755
Generated install files from templates and removed older install files
dwicinas Sep 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 174 additions & 0 deletions advocacy_docs/dev-guides/deploy/docker.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
title: Installing PostgreSQL in a Docker container on your local machine
navTitle: Installing PostgreSQL in a Docker container
description: Learn how to install PostgreSQL in a Docker container on your local machine for development purposes.
deepToC: true
---

## Prerequisites

* Docker-compatible OS (macOS, Windows, Linux)

Using Docker for your local PostgreSQL development environment streamlines setup, ensures consistency, and simplifies management. It provides a flexible, isolated, and portable solution that can adapt to various development needs and workflows.

## Preparing Docker

### Install Docker:

* macOS: Download and install Docker Desktop from Docker’s official website.
* Windows: Download and install Docker Desktop from Docker’s official website. Ensure WSL 2 is enabled if using Windows 10 or later.
* Linux: Install Docker using your distribution’s package manager. For example, on Ubuntu:

```
sudo apt update
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -ag docker $USER
newgrp docker
```

Make sure Docker is installed on your machine or download and install it from [Docker’s official website](https://www.docker.com/products/docker-desktop/).

### Pull the PostgreSQL Docker image:

Open a terminal or command prompt and run the following command to pull the latest PostgreSQL image from Docker Hub:

```
docker pull postgres
```

## Running and accessing the container’s PostgreSQL database

### Run the PostgreSQL Container:

Run a new container with the PostgreSQL image using the following command:

```
docker run --name my_postgres -d postgres -e POSTGRES_PASSWORD=mysecretpassword -v my_pgdata:/var/lib/postgresql/data -p 5432:5432
```

#### `--name my_postgres -d postgres`

The `--name` flag tells docker to creates a new container named `my_postgres`, while the `-d` flag tells it to use the `postgres` image which we pulled previously. Note that if we had not pulled it, this command would automatically pull the PostgreSQL image.

#### `-e POSTGRES_PASSWORD=mysecretpassword`

The `-e` flag sets an environment variable `POSTGRES_PASSWORD` to `mysecretpassword`. This is used the password for the default `postgres` user. You should use a different password.

#### `-v my_pgdata:/var/lib/postgresql/data`

Docker uses volumes to persist data in Docker containers. This flag mounts a volume named `my_pgdata` to persist data. The data in this case is whatever what Postgres writes to the `/var/lib/postgresql/data` directory within the container. These writes are persisted outside the container in a docker volume; the command `docker volume inspect my_pgdata` will show you information about that volume.

#### `-p 5432:5432`

The `-p` flag maps the container’s port 5432 to the host machine’s port 5432. Port 5432 is Postgres's default port for communications. By using this flag, it allows you to access the PostgreSQL database from your host machine.

### Verify the container is running:

To verify that the container is running, use the following command:

```
docker ps
```

This command lists all running containers. You should see the `my_postgres` container listed.

You now have a persistant, locally accessible Postgres database running in a Docker container.
Let's start using it.

### Access PostgreSQL:

To access the PostgreSQL database, without any additional tools, you can use the following command to open a PostgreSQL prompt:

```
docker exec \-it my\_postgres psql \-U postgres
```

This logs into the Docker container and runs the `psql` command as the `postgres` user from there.

TBD: Installing the psql client on your local machine.

### Using a PostgreSQL client

The `psql` command is a powerful tool for interacting with PostgreSQL databases. You should install it on your local machine to interact with the PostgreSQL database running in the Docker container.

#### macOS:

You can install the PostgreSQL client using Homebrew:

```
brew install libpq
```

#### Windows:

Download the PostgreSQL client from the [official website](https://www.enterprisedb.com/downloads/postgres-postgresql-downloads).

#### Linux:

Use your distribution’s package manager to install the PostgreSQL client. For example, on Ubuntu:

```
sudo apt-get install postgresql-client
```

#### Connecting other apps

You can also connect other applications to the PostgreSQL database running in the Docker container. You need to provide the following connection details:

* Host: `localhost`
* Port: `5432`
* Username: `postgres`
* Password: (whatever you set it to)
* Database: `postgres`

Or use the connection string:

```
postgresql://postgres:mysecretpassword@localhost:5432/postgres
```

### Verifying data persistence

1. Create a table and insert data.
Access the PostgreSQL instance and run the following SQL commands to create a table with columns and insert some data:

```sql
CREATE TABLE employees (
id SERIAL PRIMARY KEY,
first_name VARCHAR(50),
last_name VARCHAR(50),
email VARCHAR(100),
hire_date DATE
);
INSERT INTO employees (first_name, last_name, email, hire_date) VALUES
('John', 'Doe','[email protected]', '2020-01-15'),
('Jane', 'Smith', '[email protected]', '2019-03-22');
```

2. Stop and completly remove the container.
```
docker stop my_postgres
docker rm my_postgres
```

3. Recreate the container with the same volume.

```
docker run --name my_postgres -d postgres -e POSTGRES_PASSWORD=mysecretpassword -v my_pgdata:/var/lib/postgresql/data -p 5432:5432
```

4. Verify Data Persistence.
Access the PostgreSQL instance and check if the data still exists:

```sql
SELECT * FROM employees
```

If everything worked as expected, you should see the employee table with the data previously loaded still present.


## Conclusion

By following these steps, you have set up a robust local development environment for PostgreSQL using Docker. This setup ensures data persistence and provides a flexible, isolated, and consistent environment for all of your development needs.
7 changes: 7 additions & 0 deletions advocacy_docs/dev-guides/deploy/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Deploying Postgres for developers
navTitle: Deploying for developers
description: How to deploy Postgres for developers.
---


Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Developing Postgres applications with Python
navTitle: Developing with Python
description: How to develop applications that use Postgres databases with Python.
---

6 changes: 6 additions & 0 deletions advocacy_docs/dev-guides/developing/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Developing applications with Postgres
navTitle: Developing applications
description: How to develop applications that use Postgres databases.
---

32 changes: 32 additions & 0 deletions advocacy_docs/dev-guides/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: The EDB Postgres AI Developer Guides
navTitle: Developer Guides
description: The EDB Postgres AI Developer Guides provide information on how to use the EDB Postgres AI platform to build and develop Postgres and AI applications.
deepToC: true
directoryDefaults:
iconName: "CodeWriting"
indexCards: simple
prevNext: true
navigation:
- deploy
- working
- developing
---

The EDB Postgres AI Developer Guides are all about providing you, the developer, with the information you need to accelerate your development efforts using the EDB Postgres AI platform. The guides cover a wide range of topics, from setting up your development environment to deploying Postgres and AI applications.

## Deploying Postgres Locally for developers

* [Deploying Postgres Using Docker Locally](deploy/docker)

## Working with Postgres

* [PSQL for busy developers](working/psql-for-busy-developers)

## Developing Postgres Applications

* [Developing Postgres Applications with Python](developing/developing-postgres-applications-with-python)




6 changes: 6 additions & 0 deletions advocacy_docs/dev-guides/working/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Working with Postgres tools - guides for developers
navTitle: Working with tools
description: How to work with a range of Postgres tools, with a focus on developers and debugging.
---

65 changes: 65 additions & 0 deletions advocacy_docs/dev-guides/working/psql-for-busy-developers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: PSQL for busy developers
navTitle: PSQL for developers
description: How to use PSQL for common developer tasks
---

The PSQL command line tool is essential in a developer's toolkit as it provides full command-line access to PostgreSQL databases.

## Getting psql installed

Unless you've just installed Postgres natively on your machine, you'll need to install the psql client.

(Install guides)

## Connecting to a database

### Connection strings

psql can connect to a database using a connection string. The connection string is a single string that contains all the information needed to connect to a database.

!!! tip
Always wrap your connection string in single quotes to avoid any special characters being interpreted by the shell.
!!!

### PGPASSWORD environment variable

Best practice is not to have your password in your connection string or in your command history. Instead, you can use the `PGPASSWORD` environment variable to store your Postgres password. This is simple, but not very secure because some Unix systems allow other users to see the environment variables of other users.

### .pgpass file

Creating a .pgpass file is a more secure way to store your password. The .pgpass file is a plain text file that contains the connection information for your databases. The file should be stored in your home directory and should be readable only by you. The file should have the following format:

```
hostname:port:database:username:password
```

Hostname, port, database and username can all be set to wildcards to match any value. For example, `*:*:*:postgres:password` would match any database on any host for the user `postgres`.

Read more about the .pgpass file in the [Postgres documentation](https://www.postgresql.org/docs/current/libpq-pgpass.html).

## The PSQL command line

You can enter SQL commands directly into the PSQL command line. Be sure to end each command with a semicolon, otherwise PSQL doesn't execute the command.

You can use tab-completion in many situations to help you complete commands and table names. Pressing tab at the start of a line will show you a list of available SQL commands.

PSQL also has a number of built-in commands that can help you manage your databases and tables.


| Command | Description |
|-----------------|----------------------------------------------------------|
| `\l` | List all databases |
| `\c` | Connect to a database |
| `\d` | List tables, sequences and views in the current database |
| `\d table_name` | Describe a table |
| `\watch seconds` | Re-run a query every `seconds` |
| `\q` | Quit PSQL |

`\d` is a very useful command that shows a range of different information when followed by another character. Think of it as `d` for display. For example, `\dt` shows all the tables in the current database, `\dv` shows all the views, and `\ds` shows all the sequences.

`\watch` is useful when you want to repeat running a query at regular intervals. For example, you could use `\watch 5` to run a query every 5 seconds. The query that will be re-run is the last query you entered.




2 changes: 1 addition & 1 deletion advocacy_docs/edb-postgres-ai/analytics/quick_start.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ flavors in the installation when you connect.
as Delta Tables. Every cluster comes pre-loaded to point to a storage bucket
with benchmarking data inside (TPC-H, TPC-DS, Clickbench) at
scale factors 1 and 10.
* Only AWS is supported at the moment. Bring Your OWn Account (BYOA) is not supported.
* Only AWS is supported at the moment. Bring Your Own Account (BYOA) is not supported.
* You can deploy a cluster in any region that is activated in
your EDB Postgres AI Account. Each region has a bucket with a copy of the
benchmarking data, and so when you launch a cluster, it will use the
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: "Applying constraints"
---

At the beginning of your data migration journey with EDB Data Migration Service (EDB DMS), you [prepared and imported the schema](prepare_schema) of your source database. Now, connect to the target database and re-apply the constraints that were excluded from the schema and data migration.

## `PRIMARY KEY` and `UNIQUE` constraints

For `PRIMARY KEY` and `UNIQUE` constraints, you have already created the tables and constraints in the target Postgres database. This allowed EDB DMS to map them to the source objects and migrate data successfuly. You don't need do to anything else.

The same applies to `NOT NULL` constraints if you included them in your schema import.

## `FOREIGN KEY`, `REFERENCES`, `CHECK`, `CASCADE` and `EXCLUDE` constraints

You can now re-apply the `FOREIGN KEY`, `REFERENCES`, `CHECK`, `CASCADE` and `EXCLUDE` constraints you excluded during the [schema preparation and import](prepare_schema).

## Ensuring data integrity

Rows in tables that do not have `PRIMARY KEY` or `UNIQUE` constraints were migrated with at-least-once delivery, therefore, it is possible that these rows are duplicate.

Deduplication can be performed as part of the [verification](verify_migration).
Loading
Loading