Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional migration assistant feedback #8895

Merged
merged 13 commits into from
Dec 10, 2024
2 changes: 1 addition & 1 deletion _includes/cards.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
<div class="card">
<a href="{{site.url}}/docs/latest/migration-assistant/" class='card-link'></a>
<p class="heading">Migration Assistant</p>
<p class="description">Migrate to OpenSearch from other platforms</p>
<p class="description">Migrate to OpenSearch</p>
<p class="last-link">Documentation &#x2192;</p>
</div>
</div>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
---
layout: default
title: Getting started with data migration
parent: Deploying Migration Assistant
nav_order: 10
redirect_from:
- /upgrade-to/upgrade-to/
- /upgrade-to/snapshot-migrate/
- /migration-assistant/getting-started-with-data-migration/
---

# Getting started with data migration
Expand Down Expand Up @@ -207,8 +209,8 @@ Run the following command to access the migration console:
{% include copy.html %}


`accessContainer.sh` is located in `/opensearch-migrations/deployment/cdk/opensearch-service-migration/` on the Bootstrap instance. To learn more, see [Accessing the migration console].
`{: .note}
`accessContainer.sh` is located in `/opensearch-migrations/deployment/cdk/opensearch-service-migration/` on the Bootstrap instance. To learn more, see [Accessing the migration console]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/migrating-metadata/).
{: .note}

---

Expand Down
4 changes: 2 additions & 2 deletions _migration-assistant/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@

The Metadata migration tool integrated into the Migration CLI can be used independently to migrate cluster metadata, including index mappings, index configuration settings, templates, component templates, and aliases.

### reindex-from-snapshot
### Reindex-from-Snapshot

`Reindex-from-Snapshot` (RFS) reindexes data from an existing snapshot. Workers on Amazon Elastic Container Service (Amazon ECS) coordinate the migration of documents from an existing snapshot, reindexing the documents in parallel to a target cluster.

Expand All @@ -70,6 +70,6 @@
2. An Application Load Balancer with capture proxies relays traffic to a source while replicating data to Amazon Managed Streaming for Apache Kafka (Amazon MSK).
3. Using the migration console, you can initiate metadata migration to establish indexes, templates, component templates, and aliases on the target cluster.
4. With continuous traffic capture in place, you can use a `reindex-from-snapshot` process to capture data from your current index.
4. Once `reindex-from-snapshot` is complete, captured traffic is replayed from Amazon MSK to the target cluster by the traffic replayer.
4. Once `Reindex-from-Snapshot` is complete, captured traffic is replayed from Amazon MSK to the target cluster by the traffic replayer.

Check failure on line 73 in _migration-assistant/index.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _migration-assistant/index.md#L73

[OpenSearch.Spelling] Error: replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.
Raw output
{"message": "[OpenSearch.Spelling] Error: replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_migration-assistant/index.md", "range": {"start": {"line": 73, "column": 128}}}, "severity": "ERROR"}
5. Performance and behavior of traffic sent to the source and target clusters are compared by reviewing logs and metrics.
6. After confirming that the target cluster's functionality meets expectations, clients are redirected to the new target.
1 change: 1 addition & 0 deletions _migration-assistant/migration-console/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ title: Migration console
nav_order: 30
has_children: true
permalink: /migration-console/
has_toc: false
redirect_from:
- /migration-console/index/
---
Expand Down
11 changes: 9 additions & 2 deletions _migration-assistant/migration-phases/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,21 @@ layout: default
title: Migration phases
nav_order: 50
has_children: true
has_toc: false
permalink: /migration-phases/
redirect_from:
- /migration-phases/index/
---

This page details how to conduct a migration with Migration Assistant. It encompasses a variety of scenarios including:
# Migration phases

This page details how to conduct a migration with Migration Assistant. It shows you how to [plan for your migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/planning-your-migration/index/) and encompasses a variety of migration scenarios, including:

- [**Metadata migration**]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/migrating-metadata/): Migrating cluster metadata, such as index settings, aliases, and templates.
- [**Backfill migration**]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/backfill/): Migrating existing or historical data from a source to a target cluster.
- **Live traffic migration**: Replicating live ongoing traffic from a source to a target cluster.
- [**Live traffic migration**]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/using-traffic-replayer/): Replicating live ongoing traffic from [a source cluster]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/switching-traffic-from-the-source-cluster/) to a target cluster.





Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
layout: default
title: Live traffic migration
nav_order: 99
parent: Migration phases
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
has_toc: false
has_children: true
---

# Live traffic migration

Live traffic migration intercepts HTTP requests to a source cluster and stores them in a durable stream before forwarding them to the source cluster. The stored requests are then duplicated and replayed to the target cluster. This process synchronizes the source and target clusters while highlighting behavioral and performance differences between them. Kafka is used to manage the data flow and reconstruct HTTP requests. You can monitor the replication process through CloudWatch metrics and the [migration console]({{site.url}}{{site.baseurl}}/migration-console/), which provides results in JSON format for analysis.

To start with live traffic migration, use the following steps:

1. [Using Traffic Replayer]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/using-traffic-replayer/)

Check failure on line 16 in _migration-assistant/migration-phases/live-traffic-migration/index.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _migration-assistant/migration-phases/live-traffic-migration/index.md#L16

[OpenSearch.Spelling] Error: Replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.
Raw output
{"message": "[OpenSearch.Spelling] Error: Replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_migration-assistant/migration-phases/live-traffic-migration/index.md", "range": {"start": {"line": 16, "column": 19}}}, "severity": "ERROR"}
2. [Switching traffic from the source cluster]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/switching-traffic-from-the-source-cluster/)

Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
layout: default
title: Switching traffic from the source cluster
nav_order: 110
parent: Migration phases
grand_parent: Migration phases
parent: Live traffic migration
redirect_from:
- /migration-assistant/migration-phases/switching-traffic-from-the-source-cluster/
---

# Switching traffic from the source cluster
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
layout: default
title: Using Traffic Replayer
nav_order: 100
parent: Migration phases
grand_parent: Migration phases
parent: Live traffic migration
redirect_from:
- /migration-assistant/migration-phases/using-traffic-replayer/
---

# Using Traffic Replayer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@
layout: default
title: Assessing your cluster for migration
nav_order: 60
parent: Migration phases
parent: Planning your migration
grand_parent: Migration phases
redirect_from:
- /migration-assistant/migration-phases/assessing-your-cluster-for-migration/
---

# Assessing your cluster for migration


The goal of the Migration Assistant is to streamline the process of migrating from one location or version of Elasticsearch/OpenSearch to another. However, completing a migration sometimes requires resolving client compatibility issues before they can communicate directly with the target cluster.

## Understanding breaking changes
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
layout: default
title: Planning your migration
nav_order: 59
parent: Migration phases
has_toc: false
has_children: true
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
---

# Planning your migration

This section describes how to plan for your migration to OpenSearch by:

- [Assessing your current cluster for migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/planning-your-migration/assessing-your-cluster-for-migration/).
- [Verifying that you have the tools for migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/planning-your-migration/verifying-migration-tools/).
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
layout: default
title: Verifying migration tools
nav_order: 70
parent: Migration phases
parent: Planning your migration
grand_parent: Migration phases
redirect_from:
- /migration-assistant/migration-phases/verifying-migration-tools/
---

# Verifying migration tools
Expand Down
Loading