Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs/transporter/preview content #4083

Closed
wants to merge 47 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
6e20fac
Transporter: alpha docs
drothery-edb Mar 9, 2023
9e8289c
adding product to the required files
drothery-edb Mar 9, 2023
f3e3a7d
Add basic text about EDB Transporter
jperozoa Jun 7, 2023
cb85100
EDB Transporter preview docs
jperozoa Jun 7, 2023
7582585
Add docs for transporter db config validation scripts
YongChoEDB Jun 13, 2023
d8c0105
EDB Transporter preview docs
jperozoa Jun 14, 2023
57f5308
EDB Transporter - Getting started
jperozoa Jun 14, 2023
81284b2
EDB Transporter - Fix link
jperozoa Jun 14, 2023
0a752d9
First pass at editing transporter preview
ebgitelman Jun 15, 2023
6c965e6
Fixed links and other formatting issues
ebgitelman Jun 15, 2023
9944a8d
Update preparing_postgres_source_databases.mdx
ebgitelman Jun 15, 2023
39833b5
Update terminology.mdx
ebgitelman Jun 15, 2023
fac436f
Update upgrading.mdx
ebgitelman Jun 15, 2023
e7e8249
EDB Transporter - Code blocks type
jperozoa Jun 19, 2023
e0c200a
Changed dollar signs to standard variable braces
ebgitelman Jun 19, 2023
24e87ac
EDB Transporter - Revert "change dollar signs" on oracle table names
jperozoa Jun 20, 2023
ee92f6b
fixed vars in postgres file
ebgitelman Jun 20, 2023
a29db99
EDB Transporter - Typos in preparing oracle source
jperozoa Jun 21, 2023
c6d7d5a
adding product to the required files
drothery-edb Mar 9, 2023
e81bfc4
EDB Transporter preview docs
jperozoa Jun 7, 2023
e351df4
baby steps
dwicinas Jul 6, 2023
1c9d108
stumbling baby steps
dwicinas Jul 6, 2023
e5faf39
more tentative edits
dwicinas Jul 10, 2023
5ceefe8
mostly completed system for EDB Transporter
dwicinas Jul 12, 2023
503d768
removed rhel for powerpc
dwicinas Jul 12, 2023
9a6c5de
generated files for Transporter
dwicinas Jul 12, 2023
7b07052
template changes for SLES platforms
dwicinas Jul 17, 2023
1be715d
fixes erroneous update on earlier commit
dwicinas Jul 17, 2023
e58b696
another sles edit, missed on previous commit
dwicinas Jul 17, 2023
8fd67e9
generated files for SLS
dwicinas Jul 17, 2023
bfb7213
Added SLES for PowerPC
dwicinas Jul 18, 2023
3ac3112
Removed 2, alpha and preview folders
dwicinas Jul 18, 2023
3cf7c0d
generated install files for 2_preview
dwicinas Jul 18, 2023
b51230c
changed version from 2 to 2_preview
dwicinas Jul 18, 2023
c09cea5
added fix to ppc index page
dwicinas Jul 18, 2023
347bb4d
generated file for ppc index page
dwicinas Jul 18, 2023
5673387
fixed reader install file name for SLES
dwicinas Jul 18, 2023
ea58203
generated files for SLES
dwicinas Jul 18, 2023
0d3a6e9
removing from PR, incorrectly added
dwicinas Jul 24, 2023
f84a39c
EDB Transporter - Removing unused files
jperozoa Jul 25, 2023
878cc7c
Added Transporter icon
djw-m Jul 26, 2023
ae192bf
Icon updates
djw-m Jul 27, 2023
16ac827
Updated to bump packages
djw-m Jul 27, 2023
2f6b86c
Re-edited content
ebgitelman Sep 28, 2023
871a668
Update preparing_oracle_source_databases.mdx
ebgitelman Sep 28, 2023
6b51e89
Update preparing_postgres_source_databases.mdx
ebgitelman Sep 28, 2023
431b61c
Applied Betsy's edits to the install templates
nidhibhammar Sep 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion build-sources.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,6 @@
"slony": true,
"tde": true,
"tpa": true,
"wait_states": true
"wait_states": true,
"transporter": true
}
1 change: 1 addition & 0 deletions gatsby-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ const sourceToPluginConfig = {
name: "wait_states",
path: "advocacy_docs/pg_extensions/wait_states",
},
transporter: { name: "transporter", path: "product_docs/docs/transporter" },
};

const externalSourcePlugins = () => {
Expand Down
43 changes: 42 additions & 1 deletion install_template/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,47 @@ products:
- name: SLES 15
arch: ppc64le
supported versions: [41]
- name: EDB Transporter
platforms:
- name: CentOS 7
arch: x86_64
supported versions: [2_preview]
- name: AlmaLinux 8 or Rocky Linux 8
arch: x86_64
supported versions: [2_preview]
- name: RHEL 7 or OL 7
arch: x86_64
supported versions: [2_preview]
- name: RHEL 8 or OL 8
arch: x86_64
supported versions: [2_preview]
- name: Debian 10
arch: x86_64
supported versions: [2_preview]
- name: Debian 11
arch: x86_64
supported versions: [2_preview]
- name: Ubuntu 18.04
arch: x86_64
supported versions: [2_preview]
- name: Ubuntu 20.04
arch: x86_64
supported versions: [2_preview]
- name: Ubuntu 22.04
arch: x86_64
supported versions: [2_preview]
- name: SLES 12
arch: x86_64
supported versions: [2_preview]
- name: SLES 12
arch: ppc64le
supported versions: [2_preview]
- name: SLES 15
arch: x86_64
supported versions: [2_preview]
- name: SLES 15
arch: ppc64le
supported versions: [2_preview]
- name: Failover Manager
platforms:
- name: CentOS 7
Expand Down Expand Up @@ -990,4 +1031,4 @@ products:
supported versions: [7]
- name: SLES 15
arch: ppc64le
supported versions: [7]
supported versions: [7]
2 changes: 1 addition & 1 deletion install_template/templates/platformBase/base.njk
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Before you begin the installation process:

Setting up the repository is a one-time task. If you have already set up your repository, you don't need to perform this step.

To determine if your repository exists, enter this command:
To determine if your repository exists, enter:

{%- filter indent(2) -%}
{% block repocheck %} {# Any changes to this block should be replicated for Debian, Ubuntu, and SLES #}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{% extends "products/edb-transporter/base.njk" %}
{% set platformBaseTemplate = "almalinux-8-or-rocky-linux-8" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}
{% block prerequisites %}{% endblock prerequisites %}
24 changes: 24 additions & 0 deletions install_template/templates/products/edb-transporter/base.njk
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{% extends "platformBase/" + platformBaseTemplate + '.njk' %}
{% set packageName = packageName or 'cdcreader=-1.4-1.4766136665.17.1.jammy' %}
{% set writerPackageName = writerPackageName or 'cdcwriter=1.3-1.4766201953.4.1.jammy' %}
{% import "platformBase/_deploymentConstants.njk" as deploy %}
{% block frontmatter %}
{#
If you modify deployment path here, please first copy the old expression
and add it to the list under "redirects:" below - this ensures we don't
break any existing links.
#}
deployPath: transporter/{{ product.version }}/installing/linux_{{platform.arch}}/transporter_{{deploy.map_platform[platform.name]}}.mdx
{% endblock frontmatter %}

{% block installCommand %}
Install CDCReader:
```shell
sudo {{packageManager}} install {{ packageName }}
```

Install CDCWriter:
```shell
sudo {{packageManager}} install {{ writerPackageName }}
```
{% endblock installCommand %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{% extends "products/edb-transporter/base.njk" %}
{% set platformBaseTemplate = "centos-7" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}
{% block prerequisites %}{% endblock prerequisites %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{% extends "products/edb-transporter/debian.njk" %}
{% set platformBaseTemplate = "debian-10" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{% extends "products/edb-transporter/debian.njk" %}
{% set platformBaseTemplate = "debian-11" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{% extends "products/edb-transporter/debian.njk" %}
{% set platformBaseTemplate = "debian-9" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{% extends "products/edb-transporter/base.njk" %}
{% block debian_ubuntu %}This section steps you through getting started with your cluster including logging in, ensuring the installation was successful, connecting to your cluster, and creating the user password.

```shell{% endblock debian_ubuntu %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{% extends "platformBase/index.njk" %}
{% set productShortname="transporter" %}

{% block frontmatter %}
{{ super() }}
{% endblock frontmatter %}
{% block navigation %}
- linux_x86_64
{% endblock navigation %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@

{% extends "platformBase/ppc64le_index.njk" %}
{% set productShortname="transporter" %}

{% block frontmatter %}
{{super()}}
{% endblock frontmatter %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{% extends "products/edb-transporter/base.njk" %}
{% set platformBaseTemplate = "rhel-7-or-ol-7" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}
{% block prerequisites %}{% endblock prerequisites %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{% extends "products/edb-transporter/base.njk" %}
{% set platformBaseTemplate = "rhel-8-or-ol-8" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}
{% block prerequisites %}{% endblock prerequisites %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{% extends "products/edb-transporter/base.njk" %}
{% set platformBaseTemplate = "sles-12" %}
{% set packageManager = "zypper" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{% extends "products/edb-transporter/sles-12.njk" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}


Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{% extends "products/edb-transporter/base.njk" %}
{% set platformBaseTemplate = "sles-15" %}
{% set packageManager = "zypper" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{% extends "products/edb-transporter/sles-15.njk" %}
{% set packageName %}cdcreader-1.4-1.4766136665.17.1.el8.x86_64{% endset %}
{% set writerPackageName %}cdcwriter-1.3-1.4766201953.4.1.el8.x86_64{% endset %}

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{% extends "products/edb-transporter/ubuntu.njk" %}
{% set platformBaseTemplate = "ubuntu-18.04" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{% extends "products/edb-transporter/ubuntu.njk" %}
{% set platformBaseTemplate = "ubuntu-20.04" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{% extends "products/edb-transporter/ubuntu.njk" %}
{% set platformBaseTemplate = "ubuntu-22.04" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{% extends "products/edb-transporter/base.njk" %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@

{% extends "platformBase/x86_64_index.njk" %}
{% set productShortname="transporter" %}

{% block frontmatter %}
{{ super() }}
{% endblock frontmatter %}
19 changes: 9 additions & 10 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
"update-icons": "node scripts/createIconNames.js && prettier --write src/components/icon/iconNames.js"
},
"dependencies": {
"@enterprisedb/icons": "^1.25.1",
"@enterprisedb/icons": "^1.26.1",
"@mdx-js/mdx": "^1.6.22",
"@mdx-js/react": "^1.6.22",
"algoliasearch": "^4.17.2",
Expand Down Expand Up @@ -68,6 +68,7 @@
"globby": "^13.2.2",
"graceful-fs": "^4.2.11",
"hast-util-to-string": "^1.0.4",
"husky": "^8.0.3",
"is-absolute-url": "^4.0.1",
"markdown-to-jsx": "^7.2.1",
"mdast-util-to-string": "^2.0.0",
Expand All @@ -87,7 +88,6 @@
"fast-glob": "^3.2.12",
"hast-util-is-element": "^2.1.3",
"hast-util-to-text": "^3.1.2",
"husky": "^7.0.4",
"prettier": "^2.8.8",
"pretty-quick": "^3.1.3",
"react-refresh": "^0.14.0",
Expand Down
48 changes: 48 additions & 0 deletions product_docs/docs/transporter/2_preview/getting_started.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: "Getting started"
---

Setting up an EDB Transporter migration consists of a number of steps.

## Planning: Ensure schema compatibility

Assess Oracle database sources for schema compatibility using the [EDB Migration Portal](https://www.enterprisedb.com/docs/migration_portal/latest/) before starting the data migration process. Address incompatible schemas or exclude them from the migration.

## Preparing source and target databases

Prepare each source and the target databases needed for the migration. To prepare, enable archive log mode and supplemental logging on the Oracle database and create a user. For details, see:

- [Preparing Oracle source databases](preparing_db/preparing_oracle_source_databases/#preparing_oracle_sources)
- [Preparing Postgres source databases](preparing_db/preparing_postgres_source_databases/#preparing_postgres_sources)
- [Preparing target databases](preparing_db/preparing_target_databases/#preparing_target_databases)

## Installing

Install and run customer-side EDB Transporter software. For details, see [Installing EDB Transporter software](installing).

EDB manages cloud assets required for the migration, such as Apache Kafka and other services offered by cloud service providers.

## Sharing migration information

Share tables-to-migrate information with EDB (schema, table, and column-name mapping between source and target).

To create the migration, EDB needs to know the schemas and tables needed to migrate to the target database.

If the target table and column names are different from the source, EDB also needs the mapping for each table and column to its new name.

## EDB migration responsibilities

EDB performs the next steps:

- Deploying infrastructure. EDB deploys cloud infrastructure, which includes control components and a message bus.

- Starting migration. EDB initiates and manages the data migration until complete.

- Validating migration. EDB performs integrity validation and post-migration checks.

## Post-migration steps

After these steps, your migration is up and running and will last as long as required. After your data is across,if you plan to then stop the migration:

- You can stop and remove customer-side software.
- EDB deprovisions cloud infrastructure.
29 changes: 29 additions & 0 deletions product_docs/docs/transporter/2_preview/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
title: EDB Transporter
directoryDefaults:
description: "EDB Transporter is a cloud-first migration solution based on Debezium and Kafka. Change-event data streaming enables secure, fault-tolerant, and performant migrations to BigAnimal, on-premise EPAS/Postgres, or other cloud-hosted databases."
navigation:
- rel_notes
- known_issues
- getting_started
- "#Concepts"
- terminology
- usage_overview
- "#Planning"
- supported_versions
- limitations
- "#Installing"
- installing
- upgrading
- "#Using"
- preparing_db
- migrating_data
---

EDB Transporter offers you an effective way to migrate database data from one database to another. It does this by using the technique known as change data capture (CDC). This process catches changes in a source database’s rows being made by applications and streams them to a destination database. It reshapes the data to the schema of the destination database.

By streaming the changes from one database to another, the process avoids the traditional “big bang” of exporting, converting, and importing that came with traditional migration.

EDB Transporter is built on Kafka, a data-streaming technology, and Debezium, which uses that technology to move captured data between databases.

When the source or destination database is on premises or in a private cloud, then you need to install EDB Transporter’s CDCReader or CDCWriter. You and EDB’s team can then plan and start your migration.
Loading