Existing S3 Snapshot Repo Configuration

This page lays out additional configuration that is necessary for configuring the Migration tooling to use an existing S3 snapshot repository. These steps are expected to be consolidated in the near future to be a single configuration update.

Configuration Options

Before deployment, the CDK context options should specify the desired existing S3 snapshot repository to use for Reindex-from-Snapshot

Name	Example	Description
<BUCKET_NAME>	app-us-east-1-bucket	The name of the existing S3 bucket
<REPOSITORY_PATH>	app-repo	The path used by the source cluster snapshot repository to store snapshots
<SOURCE_VERSION>	ES_7_17	The source cluster version in the format in the format of OS_x_y or ES_x_y
<AWS_REGION>	us-east-1	The AWS region of the S3 bucket
<SNAPSHOT_NAME>	app-snapshot	The name of the existing snapshot in the source cluster snapshot repository, or the name that will be used if creating a new snapshot (default is rfs-snapshot)

{
  "default": {
    ...
    "reindexFromSnapshotServiceEnabled": true,
    "reindexFromSnapshotExtraArgs": "--s3-repo-uri s3://<BUCKET_NAME>/<REPOSITORY_PATH> --source-version <SOURCE_VERSION> --s3-region <AWS_REGION> --snapshot-name <SNAPSHOT_NAME>"
  }
}

IAM Role Updates

After deployment, the task roles for Reindex-from-Snapshot and the Migration Console will need to be updated to have S3 permissions for the existing S3 bucket.

These roles would look similar to OSMigrations-dev-us-east--reindexfromsnapshotTaskRo-lasdkfj223 and OSMigrations-dev-us-east--migrationconsoleTaskRole1-akdenrt234

A policy similar to below could be added to these task roles:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "s3:*",
            "Resource": [
                "<BUCKET_NAME>/*",
                "<BUCKET_NAME>"
            ],
            "Effect": "Allow"
        }
    ]
}

Migration Console YAML Updates

After deployment, the migration_services.yaml on the migration-console will need to be updated in order to perform any metadata migrations as well as perform any snapshot actions, such as creating any new snapshots in the existing S3 snapshot repository.

1. Access the Migration Console, steps here
2. Fill in the snapshot details to use:

snapshot:
  snapshot_name: <SNAPSHOT_NAME>
  s3:
    repo_uri: s3://<BUCKET_NAME>/<REPOSITORY_PATH>
    aws_region: <AWS_REGION>

3. Use Vim to modify the migration_services.yaml and add the details from the previous step

vim /etc/migration_services.yaml