From 78c7726d50d3767b481c5eb21d1f356933836808 Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Tue, 25 Jun 2024 19:04:56 +0300 Subject: [PATCH 1/5] PBM-1228 Documented multiple storages for backups modified: docs/features/comparison.md new file: docs/features/multi-storage.md modified: docs/reference/pbm-commands.md modified: mkdocs-base.yml --- docs/features/comparison.md | 1 + docs/features/multi-storage.md | 127 +++++++++++++++++++++++++ docs/reference/pbm-commands.md | 169 +++++++++++++++++++++++++++++++-- mkdocs-base.yml | 1 + 4 files changed, 290 insertions(+), 8 deletions(-) create mode 100644 docs/features/multi-storage.md diff --git a/docs/features/comparison.md b/docs/features/comparison.md index 01852603..5b329a06 100644 --- a/docs/features/comparison.md +++ b/docs/features/comparison.md @@ -11,6 +11,7 @@ Percona Backup for MongoDB is a fully supported community backup solution that c | Incremental backup (physical) | Yes | No |Yes | Yes | Backup management interfaces | Percona Backup for MongoDB (CLI)
PMM
mongodump / mongorestore (CLI) | -
-
mongodump / mongorestore (CLI) | Ops Manager
Cloud Manager
mongodump / mongorestore (CLI) | Atlas backups
mongodump / mongorestore (CLI) | Sharded cluster restores supported | Yes | No | Yes | Yes +| Multiple storages for backups| Yes | No | No | No ## What you get with Percona Backup for MongoDB diff --git a/docs/features/multi-storage.md b/docs/features/multi-storage.md new file mode 100644 index 00000000..5fa11825 --- /dev/null +++ b/docs/features/multi-storage.md @@ -0,0 +1,127 @@ +# Multiple storages for backups + +!!! admonition "Version added: [](../release-notes/.md)" + +A good practice in a backup policy is to follow the 3-to-1 rule: have 3 copies of data stored on 2 different storages and have 1 copy kept offsite. Instead of transferring data to multiple storages, you can configure those storages and have PBM make backups to them directly, at different schedules. + +For example, you can configure backups as follows: + +* Make full physical backups with point-in-time recovery enabled weekly and store them on AWS S3 bucket, +* Make incremental backups daily on another storage. +* Make EBS snapshots of the whole database monthly and store it on an external offsite server. + +This ability to define multiple storages for backups brings the following benefits: + +* Saves costs on data transfer in case of cloud storages +* Increases effectiveness of following your organization’s backup policy either via your own applications and tools interfaced with PBM or via Percona Everest + +## Configuration profiles + +By default, PBM stores backups and point-in-time recovery oplog slices to the remote backup storage which you defined in the configuration file during the initial setup. This is the **main** backup storage. + +To make backups to additional – **external** backup storages, a concept of a configuration profile is introduced. A configuration profile is a file that stores only the configuration for an external backup storage. + +Here’s the example of the configuration profile: + +```yaml title="minio.yaml" +storage: + type: s3 + s3: + endpointUrl: "http://localhost:9000" + bucket: mybucket + region: my-region + credentials: + access_key_id: myaccesskey + secret_access_key: mysecretkey + +``` + +To upload the configuration profile to PBM, use the [`pbm profile add`](../reference/pbm-commands.md#pbm-profile-add) command and specify the path to the profile. + +```{.bash data-prompt="$"} +$ pbm profile add /path/to/profile.yaml +``` + +To show the information about the external backup storage, use the [`pbm profile show`](../reference/pbm-commands.md#pbm-profile-show) command: + +```{.bash data-prompt="$"} +$ pbm profile show +``` + +See the full list of the configuration profile management commands in the [pbm commands](../reference/pbm-commands.md) reference. + +## Make a backup to an external storage + +To make a backup to an external backup storage, pass the profile name with the `--profile` flag for the `pbm backup` command. For example, to run a physical backup and store it in the MinIO storage defined via the `minio` configuration profile, run the following command: + +```{.bash data-prompt="$"} +$ pbm backup -t physical --profile=minio --wait +``` + +??? example "Sample output" + + {.text .no-copy} + Starting backup '2024-06-25T11:25:30Z'....Backup '2024-06-25T11:25:30Z' to remote store 's3://http://minio:9000/backup' has started + +You can make any type of backups except snapshot-based ones on an external storage. + +Note that point-in-time recovery oplog slicing is not stopped automatically for backups made to an external storage. Thus, PBM saves oplog chunks related to such backups on both the main and the external storages. + +When you make incremental backups, make sure to keep the whole backup chain on the same storage. To switch the backup storage, you must make a new base backup on it to start the new incremental chain. + +## Restore from an external storage + +Before you start, make sure that `pbm-agents` have the read permissions to backups on the remote storage(s). Also, [make all preparations for the restore](../usage/restore.md#before-you-start). + +1. List backups by running the `pbm list` or `pbm status` commands. + + ```{.bash data-prompt="$"} + $ pbm list + ``` + + The output shows the backup names and timestamps. External backups are marked with an asterisk: + + ??? example "Sample output" + + ```{.text .no-copy} + Backup snapshots: + 2024-06-25T10:53:57Z [restore_to_time: 2024-06-25T10:54:02Z] + 2024-06-25T10:54:55Z [restore_to_time: 2024-06-25T10:55:02Z] + 2024-06-25T10:57:49Z [restore_to_time: 2024-06-25T10:57:56Z] + + PITR : + 2024-06-25T10:54:03Z - 2024-06-25T10:57:51Z + ``` + +2. To make a point-in-time restore, you must explicitly pass the backup name for the `pbm restore` command: + + ```{.bash data-prompt="$"} + $ pbm-restore --time= --base-snapshot + ``` + +3. After the restore is complete, do the required post-restore steps depending on the restore type. +4. Make a fresh backup to serve as the new base for future restores. + +## Delete backups + +You can delete backups from an external storage only by name. + +Run the `pbm delete` command and pass the backup name: + +```{.bash data-prompt="$"} +$ pbm delete-backup 2024-06-25T10:54:55Z +``` + +## Implementation specifics + +1. You can make backups of any type except snapshot-based ones on the external storage. +2. To start point-in-time recovery oplog slicing, you must make a backup on the main storage. A backup from an external storage is not considered a valid base backup for oplog slicing. +3. PBM keeps backups and oplog ranges only on the main storage. For any external storage, PBM saves only backup snapshots. +4. Backup process on the external storage doesn’t stop point-in-time recovery oplog slicing on the main storage. Thus, PBM saves oplog chunks related to such backups on both the main and the external storages +5. The whole incremental chain must be stored on the same storage. To change the storage for incremental backups, you must start a new backup chain with the incremental base backup on the new storage. +6. To restore from a backup on external storage, pbm-agents must have read permissions on it. +7. To make a point-in-time recovery, you must specify the backup name via the `--base-snapshot` flag. Without it, PBM searches for the base backup on the main storage. +8. You can delete backups from external storages only by name using the `pbm delete-backup ` command. Bulk deletion of backups older than the specified time is supported only for the main storage. + + + diff --git a/docs/reference/pbm-commands.md b/docs/reference/pbm-commands.md index 8fb794c1..81a26d17 100644 --- a/docs/reference/pbm-commands.md +++ b/docs/reference/pbm-commands.md @@ -371,12 +371,23 @@ The command accepts the following flags: ```json { "snapshots": [ - { - "name": "", - "status": "done", - "completeTS": Timestamp, - "pbmVersion": "1.6.0" - } + { + "name": "", + "status": "done", + "restoreTo": Timestamp, + "pbmVersion": "", + "type": "logical", + "src": "" + }, + { + "name": "", + "status": "done", + "restoreTo": Timestamp, + "pbmVersion": "", + "type": "logical", + "src": "", + "storage": "" + } ], "pitr": { "on": false, @@ -529,6 +540,144 @@ The command accepts the following flags: | `end=timestamp` | The end time for the oplog replay. | | `--replset-remapping` | Maps the replica set names for the oplog replay. The value format is `to_name_1=from_name_1,to_name_2=from_name_2`. | +## pbm profile add + +Saves an external storage configuration to PBM. This information is defined via configuration profiles. + +To learn more about configuration profiles, see [Multiple storages for backups](../features/multi-storage.md). + +The command has the following syntax: + +```{.bash data-prompt="$"} +$ pbm profile add [] +``` + +The command accepts the following flags: + +| Flag | Description | +| ----------------------- | ------------------------------------ | +| --sync | Add a profile and sync the backup list from this storage| +|--wait | Wait for the profile to be added. The flag blocks the shell session.| +|-o, --out=text | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json`| + +??? admonition "Add profile" + + ```json + { + "msg": "OK" + } + ``` + +## pbm profile list + +Provides information about [configuration profiles](../features/multi-storage.md#configuration-profiles) added to PBM. + +The command has the following syntax: + +```{.bash data-prompt="$"} +$ pbm profile list [] +``` + +The command accepts the following flags: + +| Flag | Description | +| ----------------------- | ------------------------------------ | +| `-o`, `--out=text` | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json` | + +??? admonition "List profiles" + + ```json + { + "profiles": [ + { + "name": "test1", + "profile": true, + "storage": { + "type": "filesystem", + "filesystem": { + "path": "/tmp/local_backups" + } + } + } + ] + } + ``` + +## pbm profile remove + +Removes the specified configuration profile from PBM. + +The command has the following syntax: + +```{.bash data-prompt="$"} +$ pbm profile remove [] +``` + +The command accepts the following flags: + +| Flag | Description | +| ----------------------- | ------------------------------------ | +| `--wait` | Wait for the profile to be removed. The flag blocks the shell session.| +| `-o`, `--out=text` | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json` | + +??? admonition "Delete profile" + + ```json + { + "msg": "OK" + } + ``` + +## pbm profile show + +Shows the external storage configuration according to the specified configuration profile. + +The command has the following syntax: + +```{.bash data-prompt="$"} +$ pbm profile show [] +``` + +The command accepts the following flags: + +| Flag | Description | +| ----------------------- | ------------------------------------ | +| `-o`, `--out=text` | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json` | + +??? admonition "Show profile" + + ```json + { + "name": "test1", + "profile": true, + "storage": { + "type": "filesystem", + "filesystem": { + "path": "/tmp/local_backups" + } + } + } + ``` + +## pbm profile sync + +Syncs the backup list from the external storage according to the specified configuration profile. + +The command has the following syntax: + +```{.bash data-prompt="$"} +$ pbm profile sync [] +``` + +The command accepts the following flags: + +| Flag | Description | +| ----------------------- | ------------------------------------ | +| `--all` | Syncs backup lists from all the storages.| +| `--clear` | Clears the backup list from the storage. To clear the backup list from a specific storage, pass the profile name. When used with `--all`, clears backup lists from all storages. | +| `--wait` | Wait for the profile to be synced. The flag blocks the shell session.| +| `-o`, `--out=text` | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json` | + ## pbm restore @@ -621,8 +770,12 @@ The command accepts the following flags: "name": "", "size": 3143396168, "status": "done", - "completeTS": Timestamp, - "pbmVersion": "1.6.0" + "restoreTo": Timestamp, + "pbmVersion": "2.5.0", + "type": "logical", + "src": "", + "storage": "" + }, }, ], "pitrChunks": { diff --git a/mkdocs-base.yml b/mkdocs-base.yml index d35f1b82..24e8ffc8 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -151,6 +151,7 @@ nav: - features/incremental-backup.md - features/point-in-time-recovery.md - features/snapshots.md + - features/multi-storage.md - Comparison with MongoDB: 'features/comparison.md' - Get started: - Quickstart guide: installation.md From 3926bcf9418962ef29ac0e31ace95baf04ca6431 Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Mon, 5 Aug 2024 14:26:35 +0300 Subject: [PATCH 2/5] PBM-1303 Added --wait-time flag to pbm command reference modified: docs/reference/pbm-commands.md --- docs/reference/pbm-commands.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/reference/pbm-commands.md b/docs/reference/pbm-commands.md index 81a26d17..32a82f08 100644 --- a/docs/reference/pbm-commands.md +++ b/docs/reference/pbm-commands.md @@ -556,8 +556,9 @@ The command accepts the following flags: | Flag | Description | | ----------------------- | ------------------------------------ | -| --sync | Add a profile and sync the backup list from this storage| +| --sync | Add a profile defining an external storage and sync the backup list from this storage| |--wait | Wait for the profile to be added. The flag blocks the shell session.| +| `--wait-time` | The time to wait for PBM to report the status of adding the profile and backup sync. Use this flag together with the `--wait` flag. You can specify the duration in minutes or hours (e.g. 5m, 1h).

When not set, PBM waits till the command executes.

If it takes longer than the defined waiting time to execute the command, PBM prints the `Operation is in progress. Check pbm status and logs` error message and unblocks the shell session. The `pbm-agent` continues to execute the command enabling you to track its progress via the `pbm status` command. Available starting with version 2.6.0.| |-o, --out=text | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json`| ??? admonition "Add profile" @@ -618,6 +619,7 @@ The command accepts the following flags: | Flag | Description | | ----------------------- | ------------------------------------ | | `--wait` | Wait for the profile to be removed. The flag blocks the shell session.| +| `--wait-time` | The time to wait for PBM to report the status of the profile removal. Use this flag together with the `--wait` flag. You can specify the duration in minutes or hours (e.g. 5m, 1h).

When not set, PBM waits till the command executes.

If it takes longer than the defined waiting time to execute the command, PBM prints the `Operation is in progress. Check pbm status and logs` error message and unblocks the shell session. The `pbm-agent` continues to execute the command enabling you to track its progress via the `pbm status` command. Available starting with version 2.6.0.| | `-o`, `--out=text` | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json` | ??? admonition "Delete profile" @@ -676,6 +678,7 @@ The command accepts the following flags: | `--all` | Syncs backup lists from all the storages.| | `--clear` | Clears the backup list from the storage. To clear the backup list from a specific storage, pass the profile name. When used with `--all`, clears backup lists from all storages. | | `--wait` | Wait for the profile to be synced. The flag blocks the shell session.| +| `--wait-time` | The time to wait for PBM to report the status of the profile sync. Use this flag together with the `--wait` flag. You can specify the duration in minutes or hours (e.g. 5m, 1h).

When not set, PBM waits till the command executes.

If it takes longer than the defined waiting time to execute the command, PBM prints the `Operation is in progress. Check pbm status and logs` error message and unblocks the shell session. The `pbm-agent` continues to execute the command enabling you to track its progress via the `pbm status` command. Available starting with version 2.6.0.| | `-o`, `--out=text` | Shows the output format as either plain text or a JSON object. Supported values: `text`, `json` | From 5a94b33b06cb50cbdc0b3a52cdf46003bc9f8775 Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Wed, 14 Aug 2024 13:42:27 +0300 Subject: [PATCH 3/5] Reword clause about pitr chunks --- docs/features/multi-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/features/multi-storage.md b/docs/features/multi-storage.md index 5fa11825..040640fe 100644 --- a/docs/features/multi-storage.md +++ b/docs/features/multi-storage.md @@ -116,7 +116,7 @@ $ pbm delete-backup 2024-06-25T10:54:55Z 1. You can make backups of any type except snapshot-based ones on the external storage. 2. To start point-in-time recovery oplog slicing, you must make a backup on the main storage. A backup from an external storage is not considered a valid base backup for oplog slicing. -3. PBM keeps backups and oplog ranges only on the main storage. For any external storage, PBM saves only backup snapshots. +3. PBM saves point-int-time recovery oplog ranges only on the main storage. Backups are saved on the storage that you define when starting a backup. 4. Backup process on the external storage doesn’t stop point-in-time recovery oplog slicing on the main storage. Thus, PBM saves oplog chunks related to such backups on both the main and the external storages 5. The whole incremental chain must be stored on the same storage. To change the storage for incremental backups, you must start a new backup chain with the incremental base backup on the new storage. 6. To restore from a backup on external storage, pbm-agents must have read permissions on it. From 8b6ac2a2b64e2c5e17a627999d23e8e90678b50c Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Wed, 14 Aug 2024 15:16:59 +0300 Subject: [PATCH 4/5] Update docs/features/multi-storage.md Co-authored-by: Sandra Romanchenko <53295797+sandraromanchenko@users.noreply.github.com> --- docs/features/multi-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/features/multi-storage.md b/docs/features/multi-storage.md index 040640fe..91f1d970 100644 --- a/docs/features/multi-storage.md +++ b/docs/features/multi-storage.md @@ -116,7 +116,7 @@ $ pbm delete-backup 2024-06-25T10:54:55Z 1. You can make backups of any type except snapshot-based ones on the external storage. 2. To start point-in-time recovery oplog slicing, you must make a backup on the main storage. A backup from an external storage is not considered a valid base backup for oplog slicing. -3. PBM saves point-int-time recovery oplog ranges only on the main storage. Backups are saved on the storage that you define when starting a backup. +3. PBM saves point-in-time recovery oplog ranges only on the main storage. Backups are saved on the storage that you define when starting a backup. 4. Backup process on the external storage doesn’t stop point-in-time recovery oplog slicing on the main storage. Thus, PBM saves oplog chunks related to such backups on both the main and the external storages 5. The whole incremental chain must be stored on the same storage. To change the storage for incremental backups, you must start a new backup chain with the incremental base backup on the new storage. 6. To restore from a backup on external storage, pbm-agents must have read permissions on it. From 73748adfcd66cd794b0f590f16fc548aae86db87 Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Wed, 14 Aug 2024 15:17:06 +0300 Subject: [PATCH 5/5] Update docs/features/multi-storage.md Co-authored-by: Sandra Romanchenko <53295797+sandraromanchenko@users.noreply.github.com> --- docs/features/multi-storage.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/features/multi-storage.md b/docs/features/multi-storage.md index 91f1d970..2889fe26 100644 --- a/docs/features/multi-storage.md +++ b/docs/features/multi-storage.md @@ -118,10 +118,10 @@ $ pbm delete-backup 2024-06-25T10:54:55Z 2. To start point-in-time recovery oplog slicing, you must make a backup on the main storage. A backup from an external storage is not considered a valid base backup for oplog slicing. 3. PBM saves point-in-time recovery oplog ranges only on the main storage. Backups are saved on the storage that you define when starting a backup. 4. Backup process on the external storage doesn’t stop point-in-time recovery oplog slicing on the main storage. Thus, PBM saves oplog chunks related to such backups on both the main and the external storages -5. The whole incremental chain must be stored on the same storage. To change the storage for incremental backups, you must start a new backup chain with the incremental base backup on the new storage. -6. To restore from a backup on external storage, pbm-agents must have read permissions on it. -7. To make a point-in-time recovery, you must specify the backup name via the `--base-snapshot` flag. Without it, PBM searches for the base backup on the main storage. -8. You can delete backups from external storages only by name using the `pbm delete-backup ` command. Bulk deletion of backups older than the specified time is supported only for the main storage. +5. The whole incremental chain must be stored on the same storage. To change the storage for incremental backups, you must start a new backup chain with the incremental base backup on the new storage. +6. To restore from a backup on external storage, pbm-agents must have read permissions on it. +7. To make a point-in-time recovery, you must specify the backup name via the `--base-snapshot` flag. Without it, PBM searches for the base backup on the main storage. +8. You can delete backups from external storages only by name using the `pbm delete-backup ` command. Bulk deletion of backups older than the specified time is supported only for the main storage.