--history = : This option enables the history feature and allows the user to specify a backup series name that will be placed within the history record.
--incremental-history-name = : This option allows an incremental backup to be made based on a specific history series by name. xtrabackup will search the history table looking for the most recent (highest to_lsn) backup in the series and take the to_lsn value to use as it\u2019s starting lsn. This is mutually exclusive with --incremental-history-uuid, --incremental-basedir and --incremental-lsn options. If no valid LSN can be found (no series by that name) xtrabackup will return with an error.
--incremental-history-uuid = : Allows an incremental backup to be made based on a specific history record identified by UUID. xtrabackup will search the history table looking for the record matching UUID and take the to_lsn value to use as it\u2019s starting LSN. This options is mutually exclusive with --incremental-basedir, --incremental-lsn and --incremental-history-name options. If no valid LSN can be found (no record by that UUID or missing to_lsn), xtrabackup will return with an error.
Note
Backup that\u2019s currently being performed will NOT exist in the xtrabackup_history table within the resulting backup set as the record will not be added to that table until after the backup has been taken.
If you want access to backup history outside of your backup set in the case of some catastrophic event, you will need to either perform a mysqldump, partial backup or SELECT * on the history table after xtrabackup completes and store the results with you backup set.
For the necessary privileges, see Permissions and Privileges Needed.
"},{"location":"store-backup-history.html#percona_schemaxtrabackup_history-table","title":"PERCONA_SCHEMA.XTRABACKUP_HISTORY table","text":"This table contains the information about the previous server backups. Information about the backups will only be written if the backup was taken with --history option.
FLUSH TABLES WITH READ LOCK binlog_pos Binlog file and position at end of FLUSH TABLES WITH READ LOCK innodb_from_lsn LSN at beginning of backup which can be used to determine prior backups innodb_to_lsn LSN at end of backup which can be used as the starting lsn for the next incremental partial Is this a partial backup, if N that means that it\u2019s the full backup incremental Is this an incremental backup format Description of result format (xbstream) compact Is this a compact backup compressed Is this a compressed backup encrypted Is this an encrypted backup"},{"location":"store-backup-history.html#limitations","title":"Limitations","text":"-
--historyoption must be specified only on the command line and not within a configuration file in order to be effective. -
--incremental-history-nameand--incremental-history-uuidoptions must be specified only on the command line and not within a configuration file in order to be effective.
Percona XtraBackup supports streaming mode. Streaming mode sends a backup to STDOUT in the xbstream format instead of copying the files to the backup directory.
This method allows you to use other programs to filter the output of the backup, providing greater flexibility for storage of the backup. For example, compression is achieved by piping the output to a compression utility. One of the benefits of streaming backups and using Unix pipes is that the backups can be automatically encrypted.
To use the streaming feature, you must use the --stream, providing the format of the stream (xbstream ) and where to store the temporary files:
$ xtrabackup --stream=xbstream --target-dir=/tmp\nxtrabackup uses xbstream to stream all of the data files to STDOUT, in a special xbstream format. After it finishes streaming all of the data files to STDOUT, it stops xtrabackup and streams the saved log file too.
When compression is enabled, xtrabackup compresses the output data, except for the meta and non-InnoDB files which are not compressed, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:
quicklz
Note
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either LZ4 or Zstandard (ZSTD) compression algorithms.
The resulting files have the qpress archive format. Every \\*.qp file produced by xtrabackup is essentially a one-file qpress archive and can be extracted and uncompressed by the qpress file archiver which is available from Percona Software repositories.
lz4
To compress files using the lz4 compression algorithm, set --compress option to lz4:
$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)
The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the Zstandard (ZSTD) compression algorithm. ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. To compress files using the ZSTD compression algorithm, use the --compress=zstd option. The resulting files have the \\*.zst format.
You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The defaul value is 1.
$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nTo decompress files, use the --decompress option.
Using xbstream as a stream option, backups can be copied and compressed in parallel. This option can significantly improve the speed of the backup process. In case backups were both compressed and encrypted, they must be decrypted before they are uncompressed.
Task Command Stream the backup into an archived namedbackup.xbstream $ xtrabackup --backup --stream=xbstream --target-dir=./ > backup.xbstream Stream the backup into a compressed archive named backup.xbstream $ xtrabackup --backup --stream=xbstream --compress --target-dir=./ > backup.xbstream Encrypt the backup $ xtrabackup --backup --stream=xbstream ./ > backup.xbstream gzip -`` | openssl des3 -salt -k \u201cpassword\u201d backup.xbstream.gz.des3 Unpack the backup to the current directory $ xbstream -x < backup.xbstream Send the backup compressed directly to another host and unpack it $ xtrabackup --backup --compress --stream=xbstream --target-dir=./ | ssh user@otherhost \"xbstream -x\" Send the backup to another server using netcat On the destination host:$ nc -l 9999 | cat - > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | nc desthost 9999 Send the backup to another server using a one-liner $ ssh user@desthost \u201c( nc -l 9999 > /data/backups/backup.xbstream & )\u201d && xtrabackup --backup --stream=xbstream ./ | nc desthost 9999 Throttle the throughput to 10MB/sec using the pipe viewer tool $ xtrabackup --backup --stream=xbstream ./ | pv -q -L10m ssh user@desthost \u201ccat - > /data/backups/backup.xbstream\u201d Checksum the backup during the streaming On the destination host:$ nc -l 9999 | tee >(sha1sum > destination_checksum) > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | tee >(sha1sum > source_checksum) | nc desthost 9999Compare the checksums on the source host:$ cat source_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbadCompare the checksums on the destination host:$ cat destination_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbad Parallel compression with parallel copying backup $ xtrabackup --backup --compress --compress-threads=8 --stream=xbstream --parallel=4 --target-dir=./ > backup.xbstream Important
The streamed backup must be prepared before restoration. Streaming mode does not prepare the backup.
"},{"location":"throttling-backups.html","title":"Throttling backups","text":"Although xtrabackup does not block your database\u2019s operation, any backup can add load to the system being backed up. On systems that do not have much spare I/O capacity, it might be helpful to throttle the rate at which xtrabackup reads and writes data. You can do this with the --throttle option. This option limits the number of chunks copied per second. The chunk +size is 10 MB.
The image below shows how throttling works when --throttle is set to 1.
When specified with the --backup option, this option limits the number of pairs of read-and-write operations per second that xtrabackup will perform. If you are creating an incremental backup, then the limit is the number of read I/O operations per second.
By default, there is no throttling, and xtrabackup reads and writes data as quickly as it can. If you set too strict of a limit on the IOPS, the backup might be so slow that it will never catch up with the transaction logs that InnoDB is writing, so the backup might never complete.
"},{"location":"toolkit-version-check.html","title":"Percona Toolkit version checking","text":"Some Percona software contains \u201cversion checking\u201d functionality which is a feature that enables Percona software users to be notified of available software updates to improve your environment security and performance. Alongside this, the version check functionality also provides Percona with information relating to which software versions you are running, coupled with the environment confirmation which the software is running within. This helps enable Percona to focus our development effort accordingly based on trends within our customer community.
The purpose of this document is to articulate the information that is collected, as well as to provide guidance on how to disable this functionality if desired.
"},{"location":"toolkit-version-check.html#usage","title":"Usage","text":"Version Check was implemented in Percona Toolkit 2.1.4, and was enabled by default in version 2.2.1. Currently, it is supported as a --[no]version-check option by a number of tools in Percona Toolkit, Percona XtraBackup, and Percona Monitoring and Management (PMM).
When launched with Version Check enabled, the tool that supports this feature connects to a Percona\u2019s version check service via a secure HTTPS channel. It compares the locally installed version for possible updates, and also checks versions of the following software:
-
Operating System
-
Percona Monitoring and Management (PMM)
-
MySQL
-
Perl
-
MySQL driver for Perl (DBD::mysql)
-
Percona Toolkit
Then it checks for and warns about versions with known problems if they are identified as running in the environment.
Each version check request is logged by the server. Stored information consists of the checked system unique ID followed by the software name and version. The ID is generated either at installation or when the version checking query is submitted for the first time.
Note
Prior to version 3.0.7 of Percona Toolkit, the system ID was calculated as an MD5 hash of a hostname, and starting from Percona Toolkit 3.0.7 it is generated as an MD5 hash of a random number. Percona XtraBackup continues to use hostname-based MD5 hash.
As a result, the content of the sent query is as follows:
Expected output85624f3fb5d2af8816178ea1493ed41a;DBD::mysql;4.044\nc2b6d625ef3409164cbf8af4985c48d3;MySQL;MySQL Community Server (GPL) 5.7.22-log\n85624f3fb5d2af8816178ea1493ed41a;OS;Manjaro Linux\n85624f3fb5d2af8816178ea1493ed41a;Percona::Toolkit;3.0.11-dev\n85624f3fb5d2af8816178ea1493ed41a;Perl;5.26.2\nAlthough the version checking feature does not collect any personal information, you might prefer to disable this feature, either one time or permanently. To disable it one time, use --no-version-check option when invoking the tool from a Percona product which supports it. Here is a simple example which shows running pt-diskstats tool from the Percona Toolkit with version checking turned off:
pt-diskstats --no-version-check\nDisabling version checking permanently can be done by placing no-version-check option into the configuration file of a Percona product (see correspondent documentation for exact file name and syntax). For example, in case of Percona Toolkit this can be done in a global configuration file /etc/percona-toolkit/percona-toolkit.conf:
# Disable Version Check for all tools:\nno-version-check\nIn case of Percona XtraBackup this can be done in its configuration file in a similar way:
[xtrabackup]\nno-version-check\nWe believe having this functionality enabled improves security and performance of environments running Percona Software and it is good choice for majority of the users.
"},{"location":"toolkit-version-check.html#why-not-rely-on-operating-systems-built-in-functionality-for-software-updates","title":"Why not rely on Operating System\u2019s built in functionality for software updates?","text":"In many environments the Operating Systems repositories may not carry the latest version of software and newer versions of software often installed manually, so not being covered by operating system wide check for updates.
"},{"location":"toolkit-version-check.html#why-do-you-send-more-information-than-just-the-version-of-software-being-run-as-a-part-of-version-check-service","title":"Why do you send more information than just the version of software being run as a part of version check service?","text":"Compatibility problems can be caused by versions of various components in the environment, for example problematic versions of Perl, DBD or MySQL could cause operational problems with Percona Toolkit.
"},{"location":"trademark-policy.html","title":"Trademark policy","text":"This Trademark Policy is to ensure that users of Percona-branded products or services know that what they receive has really been developed, approved, tested and maintained by Percona. Trademarks help to prevent confusion in the marketplace, by distinguishing one company\u2019s or person\u2019s products and services from another\u2019s.
Percona owns a number of marks, including but not limited to Percona, XtraDB, Percona XtraDB, XtraBackup, Percona XtraBackup, Percona Server, and Percona Live, plus the distinctive visual icons and logos associated with these marks. Both the unregistered and registered marks of Percona are protected.
Use of any Percona trademark in the name, URL, or other identifying characteristic of any product, service, website, or other use is not permitted without Percona\u2019s written permission with the following three limited exceptions.
First, you may use the appropriate Percona mark when making a nominative fair use reference to a bona fide Percona product.
Second, when Percona has released a product under a version of the GNU General Public License (\u201cGPL\u201d), you may use the appropriate Percona mark when distributing a verbatim copy of that product in accordance with the terms and conditions of the GPL.
Third, you may use the appropriate Percona mark to refer to a distribution of GPL-released Percona software that has been modified with minor changes for the sole purpose of allowing the software to operate on an operating system or hardware platform for which Percona has not yet released the software, provided that those third party changes do not affect the behavior, functionality, features, design or performance of the software. Users who acquire this Percona-branded software receive substantially exact implementations of the Percona software.
Percona reserves the right to revoke this authorization at any time in its sole discretion. For example, if Percona believes that your modification is beyond the scope of the limited license granted in this Policy or that your use of the Percona mark is detrimental to Percona, Percona will revoke this authorization. Upon revocation, you must immediately cease using the applicable Percona mark. If you do not immediately cease using the Percona mark upon revocation, Percona may take action to protect its rights and interests in the Percona mark. Percona does not grant any license to use any Percona mark for any other modified versions of Percona software; such use will require our prior written permission.
Neither trademark law nor any of the exceptions set forth in this Trademark Policy permit you to truncate, modify or otherwise use any Percona mark as part of your own brand. For example, if XYZ creates a modified version of the Percona Server, XYZ may not brand that modification as \u201cXYZ Percona Server\u201d or \u201cPercona XYZ Server\u201d, even if that modification otherwise complies with the third exception noted above.
In all cases, you must comply with applicable law, the underlying license, and this Trademark Policy, as amended from time to time. For instance, any mention of Percona trademarks should include the full trademarked name, with proper spelling and capitalization, along with attribution of ownership to Percona LLC and/or its affiliates. For example, the full proper name for XtraBackup is Percona XtraBackup. However, it is acceptable to omit the word \u201cPercona\u201d for brevity on the second and subsequent uses, where such omission does not cause confusion.
In the event of doubt as to any of the conditions or exceptions outlined in this Trademark Policy, please contact trademarks@percona.com for assistance and we will do our very best to be helpful.
"},{"location":"update-curl-utility.html","title":"Update curl utility","text":""},{"location":"update-curl-utility.html#update-the-curl-utility-in-debian-10","title":"Update the curl utility in Debian 10","text":"The default curl version, 7.64.0, in Debian 10 has known issues when attempting to reuse an already closed connection. This issue directly affects xbcloud and users may see intermittent backup failures.
For more details, see curl #3750 or curl #3763.
Follow these steps to upgrade curl to version 7.74.0:
-
Edit the
/etc/apt/sources.listto add the following:deb http://ftp.de.debian.org/debian buster-backports main\n -
Refresh the
aptsources:$ sudo apt update\n -
Install the version from
buster-backports:$ sudo apt install curl/buster-backports\n -
Verify the version number:
Expected output$ curl --version\ncurl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0\n
One way to verify if the backup is consistent is by setting up the replication and running pt-table-checksum. This can be used to verify any type of backups, but before setting up replication, backup should be prepared and be able to run (this means that incremental backups should be merged to full backups, encrypted backups decrypted etc.).
"},{"location":"varify-backup.html#set-up-the-replication","title":"Set up the replication","text":"How to set up a replica for replication in 6 simple steps with Percona XtraBackup guide provides a detailed instructions on how to take the backup and set up the replication.
For checking the backup consistency you can use either the original server where the backup was taken, or another test server created by using a different backup method (such as cold backup, mysqldump or LVM snapshots) as the source server in the replication setup.
"},{"location":"varify-backup.html#use-pt-table-checksum","title":"Use pt-table-checksum","text":"This tool is part of the Percona Toolkit. It performs an online replication consistency check by executing checksum queries on the source, which produces different results on replicas that are inconsistent with the source.
After you confirmed that replication has been set up successfully, you can install or download pt-table-checksum. This example shows downloading the latest version of pt-table-checksum:
$ wget percona.com/get/pt-table-checksum\nNote
In order for pt-table-checksum to work correctly libdbd-mysql-perl will need to be installed on Debian/Ubuntu systems or perl-DBD-MySQL on RHEL/CentOS. If you installed the percona-toolkit package from the Percona repositories package manager should install those libraries automatically.
After this command has been run, pt-table-checksum will be downloaded to your current working directory.
Running the pt-table-checksum on the source will create percona database with the checksums table which will be replicated to the replicas as well. Example of the pt-table-checksum will look like this:
$ ./pt-table-checksum\nThe results are similar to the following:
Expected outputTS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE\n04-30T11:31:50 0 0 633135 8 0 5.400 exampledb.aka_name\n04-30T11:31:52 0 0 290859 1 0 2.692 exampledb.aka_title\nChecksumming exampledb.user_info: 16% 02:27 remain\nChecksumming exampledb.user_info: 34% 01:58 remain\nChecksumming exampledb.user_info: 50% 01:29 remain\nChecksumming exampledb.user_info: 68% 00:56 remain\nChecksumming exampledb.user_info: 86% 00:24 remain\n04-30T11:34:38 0 0 22187768 126 0 165.216 exampledb.user_info\n04-30T11:38:09 0 0 0 1 0 0.033 mysql.time_zone_name\n04-30T11:38:09 0 0 0 1 0 0.052 mysql.time_zone_transition\n04-30T11:38:09 0 0 0 1 0 0.054 mysql.time_zone_transition_type\n04-30T11:38:09 0 0 8 1 0 0.064 mysql.user\nIf all the values in the DIFFS column are 0 that means that backup is consistent with the current setup.
In case backup was not consistent pt-table-checksum should spot the difference and point to the table that does not match. Following example shows adding new user on the backed up replica in order to simulate the inconsistent backup:
mysql> grant usage on exampledb.* to exampledb@localhost identified by 'thisisnewpassword';\nIf we run the pt-table-checksum now difference should be spotted
$ ./pt-table-checksum\nThe results are similar to the following:
Expected outputTS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE\n04-30T11:31:50 0 0 633135 8 0 5.400 exampledb.aka_name\n04-30T11:31:52 0 0 290859 1 0 2.692 exampledb.aka_title\nChecksumming exampledb.user_info: 16% 02:27 remain\nChecksumming exampledb.user_info: 34% 01:58 remain\nChecksumming exampledb.user_info: 50% 01:29 remain\nChecksumming exampledb.user_info: 68% 00:56 remain\nChecksumming exampledb.user_info: 86% 00:24 remain\n04-30T11:34:38 0 0 22187768 126 0 165.216 exampledb.user_info\n04-30T11:38:09 0 0 0 1 0 0.033 mysql.time_zone_name\n04-30T11:38:09 0 0 0 1 0 0.052 mysql.time_zone_transition\n04-30T11:38:09 0 0 0 1 0 0.054 mysql.time_zone_transition_type\n04-30T11:38:09 1 0 8 1 0 0.064 mysql.user\nThis output shows that source and the replica aren\u2019t in consistent state and that the difference is in the mysql.user table.
More information on different options that pt-table-checksum provides can be found in the pt-table-checksum documentation.
"},{"location":"work-with-apparmor.html","title":"Work with AppArmor","text":"The Linux Security Module implements mandatory access controls (MAC) with AppArmor. Debian and Ubuntu systems install AppArmor by default. AppArmor uses profiles which define which files and permissions are needed for application.
Percona XtraBackup does not have a profile and is not confined by AppArmor.
For a list of common AppArmor commands, see Percona Server for MySQL - AppArmor
"},{"location":"work-with-apparmor.html#develop-a-profile","title":"Develop a profile","text":"Download the profile from:
https://github.com/percona/percona-xtrabackup/tree/8.0/packaging/percona/apparmor/apparmor.d
The following profile sections should be updated with your system information, such as location of the backup destination directory.
Expected output# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\nMove the updated file:
$ sudo mv usr.sbin.xtrabackup /etc/apparmor.d/\nInstall the profile with the following command:
$ sudo apparmor_parser -r -T -W /etc/apparmor.d/usr.sbin.xtrabackup\nRun the backup as usual.
No additional AppArmor-related actions are required to restore a backup.
"},{"location":"work-with-selinux.html","title":"Work with SELinux","text":"Percona XtraBackup is installed as an unconfined process running in an undefined domain. SELinux allows unconfined processes almost all access and the processes only use Discretionary Access Control (DAC) rules.
You find the current state of the Percona XtraBackup file with the following command:
$ ls -Z /usr/bin | grep xtrabackup\n-rwxr-xr-x. root root system_u:object_r:bin_t:s0 xtrabackup\nThe SELinux context is the following:
-
user (root)
-
role (object_r)
-
type (bin_t)
-
level (s0)
The unconfined domain supports the network-facing services, which are protected by SELinux. These domains are not exposed. In this configuration, SELinux protects against remote intrusions but local intrusions, which require local access, are not confined.
Percona XtraBackup works locally. The service is not network-facing and cannot be exploited externally. The service interacts only with the local user, who provides the parameters. Percona XtraBackup requires access to the target-dir location.
You can modify your security configuration to confine Percona XtraBackup. The first question is where to store the backup files. The service requires read and write access to the selected location.
You can use either of the following methods:
-
Allow Percona XtraBackup to write to any location. The user provides any path to the
target-dirparameter. -
Allow Percona XtraBackup to write to a specific location, such as /backups or the user\u2019s home directory.
The first option opens the entire system to read and write. Select the second option to harden your security.
"},{"location":"work-with-selinux.html#install-selinux-tools","title":"Install SELinux tools","text":"To work with policies, you must install the SELinux tools. To find which package provides the semanage command and install the package. The following is an example on CentOS 7.
$ yum provides *bin/semanage\n...\npolicycoreutils-python-2.5-34.el7.x86_64 : SELinux policy core python utilities\n...\nTo install missing packages, run the following:
$ sudo yum install -y policycoreutils-python\nThe following is an example on CentOS 8:
$ yum provides *bin/semanage\n...\npolicycoreutils-python-utils-2.8-16.1.el8.noarch : SELinux policy core python utilities\nRun the following to install the missing packages:
$ sudo yum install -y policycoreutils-python-utils\nUse a modular approach to create an SELinux policy. Create a policy module to manage XtraBackup. You must create a .te file for type enforcement, and an optional .fc file for the file contexts.
Use $ ps -efZ | grep xtrabackup to verify the service is not confined by SELinux.
Create the xtrabackup.fc file and add content. This file defines the security contexts.
/usr/bin/xtrabackup -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcrypt -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbstream -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcloud -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/backups(/.*)? system_u:object_r:xtrabackup_data_t:s0\nNote
If you are using the /backups directory you must have the last line. If you are storing the backups in the user\u2019s home directory, you can omit this line.
Download the xtrabackup.te file from the following location:
https://github.com/percona/percona-xtrabackup/tree/8.0/packaging/percona/selinx
Note
In the file, the sections in bold should be modified for your system. The fc file can also be downloaded from the same location.
Compile the policy module:
$ make -f /usr/share/selinux/devel/Makefile xtrabackup.pp\nInstall the module:
$ semodule -i xtrabackup.pp\nTag the PXB binaries with the proper SELinux tags, such as xtrabackup_exec_t.
$ restorecon -v /usr/bin/*\nIf you store your backups at /backups, restore the tag in that location:
$ restorecon -v /backups\nNote
Remember to add the standard Linux DAC permissions for this directory.
Perform the backup in the standard way.
"},{"location":"working-with-binary-logs.html","title":"Work with binary logs","text":"The xtrabackup binary integrates with the log_status table. This integration enables xtrabackup to print out the backup\u2019s corresponding binary log position, so that you can use this binary log position to provision a new replica or perform point-in-time recovery.
You can find the binary log position corresponding to a backup after the backup has been taken. If your backup is from a server with binary logging enabled, xtrabackup creates a file named xtrabackup_binlog_info in the target directory. This file contains the binary log file name and position of the exact point when the backup was taken.
210715 14:14:59 Backup created in directory '/backup/'\nMySQL binlog position: filename 'binlog.000002', position '156'\n. . .\n210715 14:15:00 completed OK!\nNote
As of Percona XtraBackup 8.0.26-18.0, xtrabackup no longer creates the xtrabackup_binlog_pos_innodb file. This change is because MySQL and Percona Server no longer update the binary log information on global transaction system section of ibdata. You should rely on xtrabackup_binlog_info regardless of the storage engine in use.
To perform a point-in-time recovery from an xtrabackup backup, you should prepare and restore the backup, and then replay binary logs from the point shown in the xtrabackup_binlog_info file.
A more detailed procedure is found here.
"},{"location":"working-with-binary-logs.html#set-up-a-new-replication-replica","title":"Set up a new replication replica","text":"To set up a new replica, you should prepare the backup, and restore it to the data directory of your new replication replica. If you are using version 8.0.22 or earlier, in your CHANGE MASTER TO command, use the binary log filename and position shown in the xtrabackup_binlog_info file to start replication.
If you are using 8.0.23 or later, use the CHANGE_REPLICATION_SOURCE_TO and the appropriate options. CHANGE_MASTER_TO is deprecated.
A more detailed procedure is found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.
"},{"location":"xbcloud-azure.html","title":"Use the xbcloud binary with Microsoft Azure Cloud Storage","text":"Implemented in Percona XtraBackup 8.0.27-19, the xbcloud binary adds support for the Microsoft Azure Cloud Storage using the REST API.
"},{"location":"xbcloud-azure.html#options","title":"Options","text":"The following are the options, environment variables, and descriptions for uploading a backup to Azure using the REST API. The environment variables are recognized by xbcloud, which maps them automatically to the corresponding parameters:
Option name Environment variables Description --azure-storage-account=name AZURE_STORAGE_ACCOUNT An Azure storage account is a unique namespace to access and store your Azure data objects. --azure-container-name=name AZURE_CONTAINER_NAME A container name is a valid DNS name that conforms to the Azure naming rules --azure-access-key=name AZURE_ACCESS_KEY A generated key that can be used to authorize access to data in your account using the Shared Key authorization. --azure-endpoint=name AZURE_ENDPOINT The endpoint allows clients to securely access data --azure-tier-class=name AZURE_STORAGE_CLASS Cloud tier can decrease the local storage required while maintaining the performance. When enabled, this feature has the following categories: Hot - Frequently accessed or modified data Cool - Infrequently accessed or modified data Archive - Rarely accessed or modified dataTest your Azure applications with the Azurite open-source emulator. For testing purposes, the xbcloud binary adds the --azure-development-storage option that uses the default access_key and storage account of azurite and testcontainer for the container. You can overwrite these options, if needed.
All the available options for xbcloud, such as parallel, max-retries, and others, can be used. For more information, see the xbcloud binary overview.
"},{"location":"xbcloud-azure.html#examples","title":"Examples","text":"An example of an xbcloud backup.
$ xtrabackup --backup --stream=xbstream | \nxbcloud put backup_name --azure-storage-account=pxbtesting --azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\nAn example of restoring a backup from xbcloud.
$ xbcloud get backup_name --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure --parallel=10 2>download.log | xbstream -x -C restore\nAn example of deleting a backup from xbcloud.
$ xbcloud delete backup_name --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\nAn example of using a shortcut restore.
$ xbcloud get azure://operator-testing/bak22 ...\nThe feature is in tech preview.
Percona XtraBackup 8.0.33-28 implements a new data sink that uses the first in, first out (FIFO) method. With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to an object storage.
Percona XtraBackup 8.0.33-28 implements the following options:
--fifo-streams=#- specifies the number of FIFO files to use for parallel data stream. To disable FIFO data sink and send stream to STDOUT, set--fifo-streams=1. The default value is1(STDOUT) to ensure the backward compatibility. The--fifo-streamsvalue must match on both the XtraBackup and xbcloud sides.--fifo-dir=name- specifies a directory to write Named Pipe.--fifo-timeout=#- specifies the number of seconds to wait for the other end to open the stream for reading. The default value is60seconds.
To use FIFO data sink, you can either run two commands in separate terminal sessions or run xtrabackup in the background.
For example, run the following commands in separate terminal sessions:
$ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo\n$ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\nRun xtrabackup in the background with the following commands:
$ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo &\n$ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\nWhen taking a backup, you can save the files locally or stream the files to either a different server or an object storage.
Before Percona XtraBackup 8.0.33-28, XtraBackup streams data to an object storage writing to STDOUT (a pipe) and using xbcloud to read from STDIN (standard input device).
XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then each copy thread writes the data chunks to a pipe (STDOUT). An xbcloud read thread reads each chunk of STDIN data. Then xbcloud uploads the chunks to an object storage using an async request, and adds a callback to an event handler list. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).
The streaming capacity for XtraBackup using STDOUT is 1.8G. This capacity is sufficient for streaming data using Wide Area Network (WAN) into, for example, Amazon Web Services or Google Cloud Platform.
Starting with Percona XtraBackup 8.0.33-28, when you stream backups to Amazon S3 compatible storage using LAN with streaming capacity of 10Gbps, XtraBackup can use multiple FIFO streams to stream the backups faster.
XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then multiple FIFO files are created to store the data from XtraBackup. Each XtraBackup copy thread writes the data chunks to a specific FIFO file. Xbcloud reads from the FIFO streams and uploads data to an object storage using async request. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).
"},{"location":"xbcloud-binary-fifo-datasink.html#performance-improvement","title":"Performance improvement","text":"Consider an example of using FIFO data sink compared to STDOUT method.
The database has 1TB of data in multiple tables. The link speed between the source server and destination server using MinIO is ~ 9.2 Gbps.
Both STDOUT and FIFO data sink scenarios push 1TB of data from two servers.
For the FIFO data sink we configure 8 parallel streams with --fifo-streams=8 both for XtraBackup and xbcloud.
The results are the following:
- The
STDOUTmethod takes 01:25:24 to push 1TB of data using 239 MBps (1.8 Gbps). - The
FIFOmethod, using 8 streams, takes 00:16:01 to push 1TB of data using 1.15 Gbps (9.2 Gbps).
The purpose of xbcloud is to download from the cloud and upload to the cloud the full or part of an xbstream archive. xbcloud will not overwrite the backup with the same name. xbcloud accepts input via a pipe from xbstream so that it can be invoked as a pipeline with xtrabackup to stream directly to the cloud without needing a local storage.
Note
In a Bash shell, the $? parameter returns the exit code from the last binary. If you use pipes, the ${PIPESTATUS[x]} array parameter returns the exit code for each binary in the pipe string.
$ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ | xbcloud put [options] full_backup\n...\n$ ${PIPESTATUS[x]}\n0 0\n$ true | false\n$ echo $?\n1\n\n# with PIPESTATUS\n$ true | false\n$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}\n0 1\nThe xbcloud binary stores each chunk as a separate object with a name backup_name/database/table.ibd.NNN..., where NNN... is a 0-padded serial number of chunk within a file. The size of chunk produced by xtrabackup and xbstream changed to 10M.
To adjust the chunk size use --read-buffer-size. To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.
xbcloud has three essential operations: put, get, and delete. With these operations, backups are created, stored, retrieved, restored, and deleted. xbcloud operations clearly map to similar operations within the AWS Amazon S3 API.
The Exponential Backoff feature was implemented in Percona XtraBackup 8.0.26-18. During the backup, a chunk fails to upload or download operation; this feature adds an exponential backoff, a sleep time, and retries the operation. This feature increases the chances of completing a backup or a restore operation.
The FIFO datasink feature was implemented in Percona XtraBackup 8.0.33-28. With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to object storage.
Important
To prevent intermittent backup failures, update the curl utility in Debian 10.
"},{"location":"xbcloud-binary-overview.html#supported-cloud-storage-types","title":"Supported cloud storage types","text":"The following cloud storage types are supported:
-
OpenStack Object Storage (Swift) - see Using the xbcloud binary with Swift
-
Amazon Simple Storage (S3) - see Using the xbcloud binary with Amazon S3
-
Azure Cloud Storage - see Using the xbcloud binary with Microsoft Azure Cloud Storage
-
Google Cloud Storage (gcs) - see Using the xbcloud binary with Google Cloud Storage
-
MinIO - see Using the xbcloud binary with MinIO
In addition to OpenStack Object Storage (Swift), which has been the only option for storing backups in a cloud storage until Percona XtraBackup 2.4.14, xbcloud supports Amazon S3, MinIO, and Google Cloud Storage. Other Amazon S3-compatible storages, such as Wasabi or Digital Ocean Spaces, are also supported.
See also
OpenStack Object Storage(\u201cSwift\u201d)
Amazon Simple Storage Service
MinIO
Google Cloud Storage
Wasabi
Digital Ocean Spaces
"},{"location":"xbcloud-binary-overview.html#usage","title":"Usage","text":"The following sample command creates a full backup:
$ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ --extra-lsndirk=/storage/backups/| xbcloud \\\nput [options] full_backup\nAn incremental backup only includes the changes since the last backup. The last backup can be either a full or incremental backup.
The following sample command creates an incremental backup:
$ xtrabackup --backup --stream=xbstream --incremental-basedir=/storage/backups \\\n--target-dir=/storage/inc-backup | xbcloud put [options] inc_backup\nTo prepare an incremental backup, you must first download the full backup with the following command:
$ xtrabackup get [options] full_backup | xbstream -xv -C /tmp/full-backup\nYou must prepare the full backup:
$ xtrabackup --prepare --apply-log-only --target-dir=/tmp/full-backup\nAfter the full backup has been prepared, download the incremental backup:
xbcloud get [options] inc_backup | xbstream -xv -C /tmp/inc-backup\nThe downloaded backup is prepared by running the following command:
$ xtrabackup --prepare --target-dir=/tmp/full-backup --incremental-dir=/tmp/inc-backup\nYou do not need the full backup to restore only a specific database. You can specify only the tables to be restored:
xbcloud get [options] ibdata1 sakila/payment.ibd /tmp/partial/partial.xbs\nxbstream -xv -C /tmp/partial < /tmp/partial/partial.xbs\nEach storage type has mandatory parameters that you can supply on the command line, in a configuration file, and via environment variables.
"},{"location":"xbcloud-binary-overview.html#configuration-files","title":"Configuration files","text":"The parameters the values of which do not change frequently can be stored in my.cnf or in a custom configuration file. The following example is a template of configuration options under the [xbcloud] group:
[xbcloud]\nstorage=s3\ns3-endpoint=http://localhost:9000/\ns3-access-key=minio\ns3-secret-key=minio123\ns3-bucket=backupsx\ns3-bucket-lookup=path\ns3-api-version=4\nNote
If you explicitly use a parameter on the command line and in a configuration file, xbcloud uses the value provided on the command line.
"},{"location":"xbcloud-binary-overview.html#environment-variables","title":"Environment variables","text":"If you explicitly use a parameter on the command line, in a configuration file, and the corresponding environment variable contains a value, xbcloud uses the value provided on the command line or in the configuration file.
"},{"location":"xbcloud-binary-overview.html#shortcuts","title":"Shortcuts","text":"For all operations (put, get, and delete), you can use a shortcut to specify the storage type, bucket name, and backup name as one parameter instead of using three distinct parameters (\u2013storage, \u2013s3-bucket, and backup name per se).
Note
Use the following format: storage-type://bucket-name/backup-name
In this example s3 refers to a storage type, operator-testing is a bucket name, and bak22 is the backup name.
$ xbcloud get s3://operator-testing/bak22 ...\nThis shortcut expands as follows:
$ xbcloud get --storage=s3 --s3-bucket=operator-testing bak22 ...\nYou can supply the mandatory parameters on the command line, configuration files, and in environment variables.
"},{"location":"xbcloud-binary-overview.html#additional-parameters","title":"Additional parameters","text":"xbcloud accepts additional parameters that you can use with any storage type. The --md5 parameter computes the MD5 hash value of the backup chunks. The result is stored in files that following the backup_name.md5 pattern.
$ xtrabackup --backup --stream=xbstream \\\n--parallel=8 2>backup.log | xbcloud put s3://operator-testing/bak22 \\\n--parallel=8 --md5 2>upload.log\nYou may use the --header parameter to pass an additional HTTP header with the server side encryption while specifying a customer key.
An example of using the --header for AES256 encryption.
$ xtrabackup --backup --stream=xbstream --parallel=4 | \\\nxbcloud put s3://operator-testing/bak-enc/ \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key: CuStoMerKey=\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key-MD5: CuStoMerKeyMd5==\" \\\n--parallel=8\nThe --header parameter is also useful to set the access control list (ACL) permissions: --header=\"x-amz-acl: bucket-owner-full-control
First, you need to make the full backup on which the incremental one is going to be based:
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/storage/backups/ \\\n--target-dir=/storage/backups/ | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup\nThen you can make the incremental backup:
$ xtrabackup --backup --incremental-basedir=/storage/backups \\\n--stream=xbstream --target-dir=/storage/inc_backup | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup\nTo prepare a backup you first need to download the full backup:
$ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup | xbstream -xv -C /storage/downloaded_full\nOnce you download the full backup it should be prepared:
$ xtrabackup --prepare --apply-log-only --target-dir=/storage/downloaded_full\nAfter the full backup has been prepared you can download the incremental backup:
$ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup | xbstream -xv -C /storage/downloaded_inc\nOnce the incremental backup has been downloaded you can prepare it by running:
$ xtrabackup --prepare --apply-log-only \\\n--target-dir=/storage/downloaded_full \\\n--incremental-dir=/storage/downloaded_inc\n\n$ xtrabackup --prepare --target-dir=/storage/downloaded_full\nIf you do not want to download the entire backup to restore the specific database you can specify only the tables you want to restore:
$ xbcloud get --swift-container=test_backup\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ full_backup \\\nibdata1 sakila/payment.ibd \\\n> /storage/partial/partial.xbs\n\n$ xbstream -xv -C /storage/partial < /storage/partial/partial.xbs\nThis feature was implemented in Percona XtraBackup 8.0.26-18.0 in the xbcloud binary.
Exponential backoff increases the chances for the completion of a backup or a restore operation. For example, a chunk upload or download may fail if you have an unstable network connection or other network issues. This feature adds an exponential backoff, or sleep, time and then retries the upload or download.
When a chunk upload or download operation fails, xbcloud checks the reason for the failure. This failure can be a CURL error or an HTTP error, or a client-specific error. If the error is listed in the Retriable errors list, xbcloud pauses for a calculated time before retrying the operation until that time reaches the --max-backoff value.
The operation is retried until the --max-retries value is reached. If the chunk operation fails on the last retry, xbcloud aborts the process.
The default values are the following:
-
\u2013max-backoff = 300000 (5 minutes)
-
\u2013max-retries = 10
You can adjust the number of retries by adding the --max-retries parameter and adjust the maximum length of time between retries by adding the --max-backoff parameter to an xbcloud command.
Since xbcloud does multiple asynchronous requests in parallel, a calculated value, measured in milliseconds, is used for max-backoff. This algorithm calculates how many milliseconds to sleep before the next retry. A number generated is based on the combining the power of two (2), the number of retries already attempted and adds a random number between 1 and 1000. This number avoids network congestion if multiple chunks have the same backoff value. If the default values are used, the final retry attempt to be approximately 17 minutes after the first try. The number is no longer calculated when the milliseconds reach the --max-backoff setting. At that point, the retries continue by using the --max-backoff setting until the max-retries parameter is reached.
We retry for the following CURL operations:
-
CURLE_GOT_NOTHING
-
CURLE_OPERATION_TIMEOUT
-
CURLE_RECV_ERROR
-
CURLE_SEND_ERROR
-
CURLE_SEND_FAIL_REWIND
-
CURLE_PARTIAL_FILE
-
CURLE_SSL_CONNECT_ERROR
We retry for the following HTTP operation status codes:
-
503
-
500
-
504
-
408
Each cloud provider may return a different CURL error or an HTTP error, depending on the issue. Add new errors by setting the following variables --curl-retriable-errors or --http-retriable-errors on the command line or in my.cnf or in a custom configuration file under the [xbcloud] section. You can add multiple errors using a comma-separated list of codes.
The error handling is enhanced when using the --verbose output. This output specifies which error caused xbcloud to fail and what parameter a user must add to retry on this error.
The following is an example of a verbose output:
Expected output210701 14:34:23 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210701 14:34:23 /work/pxb/ins/8.0/bin/xbcloud: Curl error (52) Server returned nothing (no headers, no data) is not configured as retriable. You can allow it by adding --curl-retriable-errors=52 parameter\nThe following example adjusts the maximum number of retries and the maximum time between retries.
xbcloud [options] --max-retries=5 --max-backoff=10000\nThe following list describes the process using --max-backoff=10000:
-
The chunk
xtrabackup_logfile.00000000000000000006fails to upload the first time and sleeps for 2384 milliseconds. -
The same chunk fails for the second time and the sleep time is increased to 4387 milliseconds.
-
The same chunk fails for the third time and the sleep time is increased to 8691 milliseconds.
-
The same chunk fails for the fourth time. The
max-backoffparameter has been reached. All retries sleep the same amount of time after reaching the parameter. -
The same chunk is successfully uploaded.
210702 10:07:05 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:05 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 2384 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:23 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:23 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 4387 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:52 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:07:52 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 8691 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:08:47 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:08:47 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 10000 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:10:12 /work/pxb/ins/8.0/bin/xbcloud: successfully uploaded chunk: backup3/xtrabackup_logfile.00000000000000000006, size: 8388660\nThe support for Google Cloud Storage is implemented using the interoperability mode. This mode was especially designed to interact with cloud services compatible with Amazon S3.
See also
Cloud Storage Interoperability
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=google \\\n--google-endpoint=`storage.googleapis.com` \\\n--google-access-key='YOUR-ACCESSKEYID' \\\n--google-secret-key='YOUR-SECRETACCESSKEY' \\\n--google-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\nThe following options are available when using Google Cloud Storage:
-
\u2013google-access-key =
-
\u2013google-secret-key =
-
\u2013google-bucket =
-
\u2013google-storage-class=name
-
STANDARD
-
NEARLINE
-
COLDLINE
-
ARCHIVE
- Who am I?
- What can I do?
Note
The Google storage class name options are the following:
See also
Google storage classes - the default Google storage class depends on the storage class of the bucket
"},{"location":"xbcloud-iam-profile.html","title":"Use the xbcloud binary with an IAM instance profile","text":"This feature is tech preview. Before using this feature in production, we recommend that you test restoring from physical backups in your environment and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.31-24 adds the ability to use the IAM instance profile when running xbcloud from an EC2 instance.
An authentication system has two elements:
A role defines \u201cwhat can I do.\u201d A role provides a method to define a collection of permissions. Roles are assigned to users, services and EC2 instances, the \u201cwho am I\u201d element.
The IAM instance profile is the \u201cwho\u201d for an EC2 instance and assumes the IAM role, which has permissions. The instance profile has the same name as the IAM role.
An IAM instance profile does not need the --s3-secret-key nor the --s3-access-key if they are running xbcloud from an Amazon EC2 instance. To configure or attach an instance metadata to an EC2 instance, see How can I grant my Amazon EC2 instance access to an Amazon S3 bucket.
An example of the command:
$ xtrabackup ... | xbcloud put --storage=s3 --s3-bucket=bucket-name backup-name\nThe xbcloud binary outputs a connect message when successful.
Expected output221121 13:16:26 Using instance metadata for access and secret key\n221121 13:16:26 xbcloud: Successfully connected.\nAn important consideration is that the instance metadata has a time to live (TTL) of 6 hours. A backup that takes more than that time contains Expired token errors. Use Exponential Backoff to retry the upload after fetching new keys from the instance metadata.
Output when keys have expired221121 13:04:52 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:52 xbcloud: Sleeping for 2384 ms before retrying test/mysql.ibd.00000000000000000002 [1]\n221121 13:04:55 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:55 xbcloud: Sleeping for 2887 ms before retrying test/mysql.ibd.00000000000000000003 [1]\n221121 13:04:58 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:58 xbcloud: Sleeping for 2778 ms before retrying test/undo_002.00000000000000000000 [1]\n221121 13:05:00 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:00 xbcloud: Sleeping for 2916 ms before retrying test/undo_002.00000000000000000001 [1]\n221121 13:05:03 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:03 xbcloud: Sleeping for 2794 ms before retrying test/undo_002.00000000000000000002 [1]\n221121 13:05:06 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:06 xbcloud: Sleeping for 2336 ms before retrying test/undo_001.00000000000000000000 [1]\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000002, size: 5242923\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000003, size: 23\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000000, size: 10485802\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000001, size: 6291498\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000002, size: 22\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000000, size: 10485802\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000001, size: 6291498\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000002, size: 22\n. . .\n221121 13:05:18 xbcloud: successfully uploaded chunk: test/xtrabackup_tablespaces.00000000000000000001, size: 36\n221121 13:05:19 xbcloud: Upload completed. \n$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='play.minio.io:9000' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='s3.amazonaws.com' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\nThe following options are available when using Amazon S3:
Option Details \u2013s3-access-key Use to supply the AWS access key ID \u2013s3-secret-key Use to supply the AWS secret access key \u2013s3-bucket Use supply the AWS bucket name \u2013s3-region Use to specify the AWS region. The default value is us-east-1 \u2013s3-api-version = Select the signing algorithm. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-bucket-lookup = Specify whether to use bucket.endpoint.com or endpoint.com/bucket*style requests. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-storage-class= Specify the S3 storage class. The default storage class depends on the provider. The name options are the following:- STANDARD
- STANDARD_IA
- GLACIER
Following the principle of \u201cleast-privilege\u201d, these are the minimum bucket permissions needed for xbcloud to write backups to S3: LIST/PUT/GET/DELETE.
The following example shows the policy definition for writing to the xbcloud-testing bucket on the AWS S3 storage.
{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": \"arn:aws:s3:::xbcloud-testing\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:PutObject\",\n\"s3:PutObjectAcl\",\n\"s3:GetObject\",\n\"s3:GetObjectAcl\",\n\"s3:DeleteObject\"\n],\n\"Resource\": \"arn:aws:s3:::xbcloud-testing/*\"\n}\n]\n}\nThe following environment variables are recognized. xbcloud maps them automatically to corresponding parameters applicable to the selected storage.
-
AWS_ACCESS_KEY_ID (or ACCESS_KEY_ID)
-
AWS_SECRET_ACCESS_KEY (or SECRET_ACCESS_KEY)
-
AWS_DEFAULT_REGION (or DEFAULT_REGION)
-
AWS_ENDPOINT (or ENDPOINT)
-
AWS_CA_BUNDLE
$ xbcloud get s3://operator-testing/bak22 \\\n--s3-endpoint=https://storage.googleapis.com/ \\\n--parallel=10 2>download.log | xbstream -x -C restore --parallel=8\nThe following example shows how to make a full backup and upload it to Swift.
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\n--parallel=10 \\\nfull_backup\nThe following OpenStack environment variables are also recognized and mapped automatically to the corresponding swift parameters (--storage=swift):
-
OS_AUTH_URL
-
OS_TENANT_NAME
-
OS_TENANT_ID
-
OS_USERNAME
-
OS_PASSWORD
-
OS_USER_DOMAIN
-
OS_USER_DOMAIN_ID
-
OS_PROJECT_DOMAIN
-
OS_PROJECT_DOMAIN_ID
-
OS_REGION_NAME
-
OS_STORAGE_URL
-
OS_CACERT
$ xbcloud get [options] <name> [<list-of-files>] | xbstream -x\nThe following example shows how to fetch and restore the backup from Swift:
$ xbcloud get --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\nfull_backup | xbstream -xv -C /tmp/downloaded_full\n\n$ xbcloud delete --storage=swift --swift-user=xtrabackup \\\n--swift-password=xtrabackup123! --swift-auth-version=3 \\\n--swift-auth-url=http://openstack.ci.percona.com:5000/ \\\n--swift-container=mybackup1 --swift-domain=Default\nxbcloud has the following command line options:
"},{"location":"xbcloud-swift.html#-storageswifts3google","title":"--storage(=[swift|s3|google])","text":"Cloud storage option. xbcloud supports Swift, MinIO, and AWS S3. The default value is swift.
The URL of the Swift cluster
"},{"location":"xbcloud-swift.html#-swift-storage-url","title":"--swift-storage-url()","text":"The xbcloud tries to get the object-store URL for a given region (if any are specified) from the keystone response. You can override that URL by passing \u2013swift-storage-url=URL argument.
"},{"location":"xbcloud-swift.html#-swift-user","title":"--swift-user()","text":"The Swift username (X-Auth-User, specific to Swift)
"},{"location":"xbcloud-swift.html#-swift-key","title":"--swift-key()","text":"The Swift key/password (X-Auth-Key, specific to Swift)
"},{"location":"xbcloud-swift.html#-swift-container","title":"--swift-container()","text":"The container to back up into (specific to Swift)
"},{"location":"xbcloud-swift.html#-paralleln","title":"--parallel(=N)","text":"The maximum number of concurrent upload/download requests. The default value is 1.
The path to the file with CA certificates
"},{"location":"xbcloud-swift.html#-insecure","title":"--insecure()","text":"Do not verify server\u2019s certificate
"},{"location":"xbcloud-swift.html#swift-authentication-options","title":"Swift authentication options","text":"The Swift specification describes several authentication options. The xbcloud tool can authenticate against keystone with API version 2 and 3.
"},{"location":"xbcloud-swift.html#-swift-auth-version","title":"--swift-auth-version()","text":"Specifies the swift authentication version. The possible values are: 1.0 - TempAuth, 2.0 - Keystone v2.0, and 3 - Keystone v3. The default value is 1.0.
For v2 additional options are:
"},{"location":"xbcloud-swift.html#-swift-tenant","title":"--swift-tenant()","text":"Swift tenant name
"},{"location":"xbcloud-swift.html#-swift-tenant-id","title":"--swift-tenant-id()","text":"Swift tenant ID
"},{"location":"xbcloud-swift.html#-swift-region","title":"--swift-region()","text":"Swift endpoint region
"},{"location":"xbcloud-swift.html#-swift-password","title":"--swift-password()","text":"Swift password for the user
For v3 additional options are:
"},{"location":"xbcloud-swift.html#-swift-user-id","title":"--swift-user-id()","text":"Swift user ID
"},{"location":"xbcloud-swift.html#-swift-project","title":"--swift-project()","text":"Swift project name
"},{"location":"xbcloud-swift.html#-swift-project-id","title":"--swift-project-id()","text":"Swift project ID
"},{"location":"xbcloud-swift.html#-swift-domain","title":"--swift-domain()","text":"Swift domain name
"},{"location":"xbcloud-swift.html#-swift-domain-id","title":"--swift-domain-id()","text":"Swift domain ID
"},{"location":"xbcrypt-binary-overview.html","title":"The xbcrypt binary overview","text":"To support encryption and decryption of the backups, a new tool xbcrypt was introduced to Percona XtraBackup.
Percona XtraBackup 8.0.28-20 implements the XBCRYPT_ENCRYPTION_KEY environment variable. The variable is only used in place of the --encrypt_key=name option. You can use the environment variable or command line option. If you use both, the command line option takes precedence over the value specified in the environment variable.
This utility has been modeled after The xbstream binary to perform encryption and decryption outside of Percona XtraBackup. xbcrypt has following command line options:
Decrypt data input to output.
"},{"location":"xbcrypt-binary-overview.html#-i-inputname","title":"-i(, --input(=name)","text":"Optional input file. If not specified, input will be read from standard input.
"},{"location":"xbcrypt-binary-overview.html#-o-outputname","title":"-o(, --output(=name)","text":"Optional output file. If not specified, output will be written to standard output.
"},{"location":"xbcrypt-binary-overview.html#-a-encrypt-algoname","title":"-a(, --encrypt-algo(=name)","text":"Encryption algorithm.
"},{"location":"xbcrypt-binary-overview.html#-k-encrypt-keyname","title":"-k(, --encrypt-key(=name)","text":"Encryption key.
"},{"location":"xbcrypt-binary-overview.html#-f-encrypt-key-filename","title":"-f(, --encrypt-key-file(=name)","text":"File which contains encryption key.
"},{"location":"xbcrypt-binary-overview.html#-s-encrypt-chunk-size","title":"-s(, --encrypt-chunk-size(=#)","text":"Size of working buffer for encryption in bytes. The default value is 64K.
"},{"location":"xbcrypt-binary-overview.html#-encrypt-threads","title":"--encrypt-threads(=#)","text":"This option specifies the number of worker threads that will be used for parallel encryption/decryption.
"},{"location":"xbcrypt-binary-overview.html#-v-verbose","title":"-v(, --verbose()","text":"Display verbose status output.
"},{"location":"xbstream-binary-overview.html","title":"The xbstream binary overview","text":"To support simultaneous compression and streaming, a new custom streaming format called xbstream was introduced to Percona XtraBackup in addition to the TAR format. That was required to overcome some limitations of traditional archive formats such as tar, cpio and others which did not allow streaming dynamically generated files, for example dynamically compressed files. Other advantages of xbstream over traditional streaming/archive format include ability to stream multiple files concurrently (so it is possible to use streaming in the xbstream format together with the \u2013parallel option) and more compact data storage.
This utility has a tar-like interface:
-
with the
-xoption it extracts files from the stream read from its standard input to the current directory unless specified otherwise with the-coption. Support for parallel extraction with the--paralleloption has been implemented in Percona XtraBackup 2.4.7. -
with the
-coption it streams files specified on the command line to its standard output. -
with the
--decrypt=ALGOoption specified xbstream will automatically decrypt encrypted files when extracting input stream. Supported values for this option are:AES128,AES192, andAES256. Either--encrypt-keyor--encrypt-key-fileoptions must be specified to provide encryption key, but not both. This option has been implemented in Percona XtraBackup 2.4.7. -
with the
--encrypt-threadsoption you can specify the number of threads for parallel data encryption. The default value is1. This option has been implemented in Percona XtraBackup 2.4.7. -
the
--encrypt-keyoption is used to specify the encryption key that will be used. It can\u2019t be used with--encrypt-key-fileoption because they are mutually exclusive. This option has been implemented in Percona XtraBackup 2.4.7. -
the
--encrypt-key-fileoption is used to specify the file that contains the encryption key. It can\u2019t be used with--encrypt-keyoption. because they are mutually exclusive. This option has been implemented in Percona XtraBackup 2.4.7.
The utility also tries to minimize its impact on the OS page cache by using the appropriate posix_fadvise() calls when available.
When compression is enabled with xtrabackup all data is being compressed, including the transaction log file and meta data files, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:
quicklz
Note
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either LZ4 or Zstandard (ZSTD) compression algorithms.
The resulting files have the qpress archive format. Every \\*.qp file produced by xtrabackup is essentially a one-file qpress archive and can be extracted and uncompressed by the qpress file archiver. This means that there is no need to decompress entire backup to restore a single table as with tar.gz.
To decompress individual files, run xbstream with the --decompress option. You may control the number of threads used for decompressing by passing the --decompress-threads option.
Also, files can be decompressed using the qpress tool. The tool supports multi-threaded decompression.
lz4
To compress files using the lz4 compression algorithm, set --compress option to lz4:
$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)
The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the Zstandard (ZSTD) compression algorithm. ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.
To compress files using the ZSTD compression algorithm, use the --compress=zstd option. The resulting files have the \\*.zst format.
You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The defaul value is 1.
$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nTo decompress files, use the --decompress option.
The xtrabackup binary is a compiled C program that is linked with the InnoDB libraries and the standard MySQL client libraries.
xtrabackup enables point-in-time backups of InnoDB / XtraDB tables together with the schema definitions, MyISAM tables, and other portions of the server.
The InnoDB libraries provide the functionality to apply a log to data files. The MySQL client libraries are used to parse command-line options and configuration file.
The tool runs in either --backup or --prepare mode, corresponding to the two main functions it performs. There are several variations on these functions to accomplish different tasks, and there are two less commonly used modes, --stats and --print-param.
The xtrabackup binary exits with the traditional success value of 0 after a backup when no error occurs. If an error occurs during the backup, the exit value is 1.
In certain cases, the exit value can be something other than 0 or 1, due to the command-line option code included from the MySQL libraries. An unknown command-line option, for example, will cause an exit code of 255.
"},{"location":"xtrabackup-files.html","title":"Index of files created by Percona XtraBackup","text":"- Information related to the backup and the server
backup-my.cnf This file contains information to start the mini instance of InnoDB during the --prepare. This **not** a backup of the original my.cnf. The InnoDB configuration is read from the file backup-my.cnf created by xtrabackup when the backup was made. The --prepare uses InnoDB configuration from backup-my.cnf by default, or from --defaults-file, if specified. The InnoDB's configuration in this context means server variables that affect dataformat, i.e. innodb_page_size option, innodb_log_block_size, etc. Location-related variables, like innodb_log_group_home_dir or innodb_data_file_path are always ignored by --prepare, so preparing a backup always works with data files from the back directory, rather than any external ones. xtrabackup_checkpointsThe type of the backup (for example, full or incremental), its state (for example, prepared) and the LSN range contained in it. This information is used for incremental backups. Example of the xtrabackup_checkpoints after taking a full backup:
backup_type = full-backuped\nfrom lsn= 0\nto_lsn = 15188961605\nlast_lsn = 15188961605\nExample of the
xtrabackup_checkpoints after taking an incremental backup: backup_type = incremental\nfrom_lsn = 15188961605\nto_lsn = 15189350111\nlast_lsn = 15189350111\n
xtrabackup_binlog_info The binary log file used by the server and its position at the moment of the backup. A result of the following query:
SELECT server_uuid, local, replication, storage_engines FROM performance_schema.log_status;\n
xtrabackup_binlog The xtrabackup binary used in the process. xtrabackup_logfile Contains data needed for running the: --prepare. > The bigger this file is the --prepare process > will take longer to finish. <table_name>.delta.meta This file is going to be created when performing the incremental backup. > It contains the per-table delta metadata: page size, size of compressed > page (if the value is 0 it means the tablespace isn\u2019t compressed) and > space id. Example of this file:
page_size = 16384\nzip_size = 0\nspace_id = 0\n
-
Information related to the replication environment (if using the
--slave-infooption):xtrabackup_slave_infoThe
CHANGE MASTERstatement needed for setting up a replica. -
Information related to the Galera and Percona XtraDB Cluster (if using the
--galera-infooption):xtrabackup_galera_infoContains the values of
wsrep_local_state_uuidandwsrep_last_committedstatus variables
This page contains notes on various internal aspects of the xtrabackup tool\u2019s operation.
"},{"location":"xtrabackup-implementation-details.html#file-permissions","title":"File permissions","text":"xtrabackup opens the source data files in read-write mode, although it does not modify the files. This means that you must run xtrabackup as a user who has permission to write the data files. The reason for opening the files in read-write mode is that xtrabackup uses the embedded InnoDB libraries to open and read the files, and InnoDB opens them in read-write mode because it normally assumes it is going to write to them.
"},{"location":"xtrabackup-implementation-details.html#tune-the-os-buffers","title":"Tune the OS buffers","text":"Because xtrabackup reads large amounts of data from the filesystem, it uses posix_fadvise() where possible, to instruct the operating system not to try to cache the blocks it reads from disk. Without this hint, the operating system would prefer to cache the blocks, assuming that xtrabackup is likely to need them again, which is not the case. Caching such large files can place pressure on the operating system\u2019s virtual memory and cause other processes, such as the database server, to be swapped out. The xtrabackup tool avoids this with the following hint on both the source and destination files:
posix_fadvise(file, 0, 0, POSIX_FADV_DONTNEED)\nIn addition, xtrabackup asks the operating system to perform more aggressive read-ahead optimizations on the source files:
posix_fadvise(file, 0, 0, POSIX_FADV_SEQUENTIAL)\nWhen copying the data files to the target directory, xtrabackup reads and writes 1 MB of data at a time. This is not configurable. When copying the log file, xtrabackup reads and writes 512 bytes at a time. This is also not possible to configure, and matches InnoDB\u2019s behavior (workaround exists in Percona Server for MySQL because it has an option to tune innodb_log_block_size for XtraDB, and in that case Percona XtraBackup will match the tuning).
After reading from the files, xtrabackup iterates over the 1MB buffer a page at a time, and checks for page corruption on each page with InnoDB\u2019s buf_page_is_corrupted() function. If the page is corrupt, it re-reads and retries up to 10 times for each page. It skips this check on the doublewrite buffer.
Here you can find all of the command-line options for the xtrabackup binary.
"},{"location":"xtrabackup-option-reference.html#modes-of-operation","title":"Modes of operation","text":"You invoke xtrabackup in one of the following modes:
-
--backupmode to make a backup in a target directory -
--preparemode to restore data from a backup (created in--backupmode) -
--copy-backto copy data from a backup to the location that contained the original data; to move data instead of copying use the alternate--move-backmode. -
--statsmode to scan the specified data files and print out index statistics.
When you intend to run xtrabackup in any of these modes, use the following syntax:
$ xtrabackup [--defaults-file=#] --backup|--prepare|--copy-back|--stats [OPTIONS]\nFor example, the --prepare mode is applied as follows:
$ xtrabackup --prepare --target-dir=/data/backup/mysql/\nFor all modes, the default options are read from the xtrabackup and mysqld configuration groups from the following files in the given order:
-
/etc/my.cnf -
/etc/mysql/my.cnf -
/usr/etc/my.cnf -
~/.my.cnf.
As the first parameter to xtrabackup (in place of the --defaults-file, you may supply one of the following:
-
--print-defaultsto have xtrabackup print the argument list and exit. -
--no-defaultsto forbid reading options from any file but the login file. -
--defaults-fileto read the default options from the given file. -
--defaults-extra-fileto read the specified additional file after the global files have been read. -
--defaults-group-suffixto read the configuration groups with the given suffix. The effective group name is constructed by concatenating the default configuration groups (xtrabackup and mysqld) with the given suffix. -
--login-pathto read the given path from the login file.
There is a large group of InnoDB options that are normally read from the my.cnf configuration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify them explicitly. These options have the same behavior in InnoDB and XtraDB. See --innodb-miscellaneous for more information.
This option causes only the redo stage to be performed when preparing a backup. It is very important for incremental backups.
"},{"location":"xtrabackup-option-reference.html#-backup","title":"--backup()","text":"Make a backup and place it in --target-dir. See Creating a backup.
The timeout in seconds for attempts to acquire metadata locks.
"},{"location":"xtrabackup-option-reference.html#-backup-lock-retry-count","title":"--backup-lock-retry-count()","text":"The number of attempts to acquire metadata locks.
"},{"location":"xtrabackup-option-reference.html#-backup-locks","title":"--backup-locks()","text":"This option controls if backup locks should be used instead of FLUSH TABLES WITH READ LOCK on the backup stage. The option has no effect when backup locks are not supported by the server. This option is enabled by default, disable with --no-backup-locks.
This option checks if Percona XtraBackup has all required privileges. If a missing privilege is required for the current operation, it will terminate and print out an error message. If a missing privilege is not required for the current operation, but may be necessary for some other XtraBackup operation, the process is not aborted and a warning is printed.
xtrabackup: Error: missing required privilege LOCK TABLES on *.*\nxtrabackup: Warning: missing required privilege REPLICATION CLIENT on *.*\nDo not keep files opened. When xtrabackup opens tablespace it normally doesn\u2019t close its file handle in order to handle the DDL operations correctly. However, if the number of tablespaces is really huge and can not fit into any limit, there is an option to close file handles once they are no longer accessed. Percona XtraBackup can produce inconsistent backups with this option enabled. Use at your own risk.
"},{"location":"xtrabackup-option-reference.html#-compress","title":"--compress()","text":"This option tells xtrabackup to compress all output data, including the transaction log file and meta data files, using either the quicklz, lz4, or ZSTD compression algorithm. quicklz is chosen by default.
Note
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either LZ4 or Zstandard (ZSTD) compression algorithms. See Create a compressed backup for more information.
When using --compress=quicklz or --compress, the resulting files have the qpress archive format. Every \\*.qp file produced by xtrabackup is essentially a one-file qpress archive and can be extracted and uncompressed by the qpress file archiver.
--compress=lz4 produces \\*.lz4 files. You can extract the contents of these files by using lz4 program.
--compress=zstd produces \\*.zst files. You can extract the contents of these files by using the --decompress option. You can specify ZSTD compression level with the --compress-zstd-level(=#) option.
Size of working buffer(s) for compression threads in bytes. The default value is 64K.
"},{"location":"xtrabackup-option-reference.html#-compress-threads","title":"--compress-threads(=#)","text":"This option specifies the number of worker threads used by xtrabackup for parallel data compression. This option defaults to 1. Parallel compression (--compress-threads) can be used together with parallel file copying (--parallel). For example, --parallel=4 --compress --compress-threads=2 will create 4 I/O threads that will read the data and pipe it to 2 compression threads.
This option is tech preview quality. Before using --compress-zstd-level(=#) in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
This option specifies ZSTD compression level. The default value is 1. Allowed range of values is from 1 to 19. The option has been implemented in Percona XtraBackup 8.0.30-22.
Copy all the files in a previously made backup from the backup directory to their original locations. This option will not copy over existing files unless --force-non-empty-directories option is specified.
Write core on fatal signals.
"},{"location":"xtrabackup-option-reference.html#-databases","title":"--databases(=#)","text":"This option specifies a list of databases and tables that should be backed up. The option accepts the list of the form \"databasename1[.table_name1] databasename2[.table_name2] . . .\".
Excluding databases based on name, Operates the same way as --databases, but matched names are excluded from backup. Note that this option has a higher priority than --databases.
This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form databasename1[.table_name1], one element per line.
The source directory for the backup. This should be the same as the datadir for your MySQL server, so it should be read from my.cnf if that exists; otherwise you must specify it on the command line.
When combined with the --copy-back or --move-back option, --datadir refers to the destination directory.
Once connected to the server, in order to perform a backup you will need READ and EXECUTE permissions at a filesystem level in the server\u2019s datadir.
This is a debug-only option used by the xtrabackup test suite.
"},{"location":"xtrabackup-option-reference.html#-debug-syncname","title":"--debug-sync(=name)","text":"The debug sync point. This option is only used by the xtrabackup test suite.
"},{"location":"xtrabackup-option-reference.html#-decompress","title":"--decompress()","text":"Decompresses all files in a backup previously made with the --compress option. The --parallel option will allow multiple files to be decrypted simultaneously. In order to decompress, the qpress utility MUST be installed and accessible within the path. Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory users should use --remove-original option.
The --decompress option may be used with xbstream to decompress individual qpress files.
Force xbstream to use the specified number of threads for decompressing.
"},{"location":"xtrabackup-option-reference.html#-decryptencryption-algorithm","title":"--decrypt(=ENCRYPTION-ALGORITHM)","text":"Decrypts all files with the .xbcrypt extension in a backup previously made with --encrypt option. The --parallel option will allow multiple files to be decrypted simultaneously. Percona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should use --remove-original option.
Read this file after the global files are read. Must be given as the first option on the command-line.
"},{"location":"xtrabackup-option-reference.html#-defaults-filemycnf","title":"--defaults-file(=[MY.CNF])","text":"Only read default options from the given file. Must be given as the first option on the command-line. Must be a real file; it cannot be a symbolic link.
"},{"location":"xtrabackup-option-reference.html#-defaults-groupgroup-name","title":"--defaults-group(=GROUP-NAME)","text":"This option is to set the group which should be read from the configuration file. This is used by xtrabackup if you use the --defaults-group option. It is needed for mysqld_multi deployments.
Also reads groups with concat(group, suffix).
"},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool","title":"--dump-innodb-buffer-pool()","text":"This option controls whether or not a new dump of buffer pool content should be done.
With --dump-innodb-buffer-pool, xtrabackup makes a request to the server to start the buffer pool dump (it takes some time to complete and is done in background) at the beginning of a backup provided the status variable innodb_buffer_pool_dump_status reports that the dump has been completed.
$ xtrabackup --backup --dump-innodb-buffer-pool --target-dir=/home/user/backup\nBy default, this option is set to OFF.
If innodb_buffer_pool_dump_status reports that there is running dump of buffer pool, xtrabackup waits for the dump to complete using the value of --dump-innodb-buffer-pool-timeout
The file ib_buffer_pool stores tablespace ID and page ID data used to warm up the buffer pool sooner.
This option contains the number of seconds that xtrabackup should monitor the value of innodb_buffer_pool_dump_status to determine if buffer pool dump has completed.
This option is used in combination with --dump-innodb-buffer-pool. By default, it is set to 10 seconds.
This option contains the percentage of the most recently used buffer pool pages to dump.
This option is effective if --dump-innodb-buffer-pool option is set to ON. If this option contains a value, xtrabackup sets the MySQL system variable innodb_buffer_pool_dump_pct. As soon as the buffer pool dump completes or it is stopped (see --dump-innodb-buffer-pool-timeout), the value of the MySQL system variable is restored.
This option instructs xtrabackup to encrypt backup copies of InnoDB data files using the algorithm specified in the ENCRYPTION_ALGORITHM. Currently supported algorithms are: AES128, AES192 and AES256
A proper length encryption key to use. It is not recommended to use this option where there is uncontrolled access to the machine as the command line and thus the key can be viewed as part of the process info.
"},{"location":"xtrabackup-option-reference.html#-encrypt-key-fileencryption_key_file","title":"--encrypt-key-file(=ENCRYPTION_KEY_FILE)","text":"The name of a file where the raw key of the appropriate length can be read from. The file must be a simple binary (or text) file that contains exactly the key to be used.
It is passed directly to the xtrabackup child process. See the xtrabackup documentation for more details.
"},{"location":"xtrabackup-option-reference.html#-encrypt-threads","title":"--encrypt-threads(=#)","text":"This option specifies the number of worker threads that will be used for parallel encryption/decryption. See the xtrabackup documentation for more details.
"},{"location":"xtrabackup-option-reference.html#-encrypt-chunk-size","title":"--encrypt-chunk-size(=#)","text":"This option specifies the size of the internal working buffer for each encryption thread, measured in bytes. It is passed directly to the xtrabackup child process.
To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.
This option is in tech preview. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Implemented in Percona XtraBackup 8.0.32-26, the option lets you enable or disable the Smart memory estimation feature. The default value is OFF. Enable the feature by setting --estimate-memory=ON in the backup phase and setting the --use-free-memory-pct option in the --prepare phase. If the --estimate-memory setting is disabled, the --use-free-memory-pct setting is ignored.
An example of how to enable the Smart memory estimation feature:
$ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n$ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\nCreate files necessary for exporting tables. See Restoring Individual Tables.
"},{"location":"xtrabackup-option-reference.html#-extra-lsndirdirectory","title":"--extra-lsndir(=DIRECTORY)","text":"(for \u2013backup): save an extra copy of the xtrabackup_checkpoints and xtrabackup_info files in this directory.
When specified, it makes --copy-back and --move-back option transfer files to non-empty directories. No existing files will be overwritten. If files that need to be copied/moved from the backup directory already exist in the destination directory, it will still fail with an error.
This option specifies time in seconds that xtrabackup should wait for queries that would block FLUSH TABLES WITH READ LOCK before running it. If there are still such queries when the timeout expires, xtrabackup terminates with an error. Default is 0, in which case it does not wait for queries to complete and starts FLUSH TABLES WITH READ LOCK immediately. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
This option specifies the query run time threshold which is used by xtrabackup to detect long-running queries with a non-zero value of --ftwrl-wait-timeout. FLUSH TABLES WITH READ LOCK is not started until such long-running queries exist. This option has no effect if --ftwrl-wait-timeout is 0. Default value is 60 seconds. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
This option specifies which types of queries are allowed to complete before xtrabackup will issue the global lock. Default is all.
This option creates the xtrabackup_galera_info file which contains the local node state at the time of the backup. Option should be used when performing the backup of Percona XtraDB Cluster. It has no effect when backup locks are used to create the backup.
Generate a new master key when doing a copy-back.
"},{"location":"xtrabackup-option-reference.html#-generate-transition-key","title":"--generate-transition-key()","text":"xtrabackup needs to access the same keyring file or vault server during prepare and copy-back but it should not depend on whether the server keys have been purged.
--generate-transition-key creates and adds to the keyring a transition key for xtrabackup to use if the master key used for encryption is not found because it has been rotated and purged.
Get the server public key
"},{"location":"xtrabackup-option-reference.html#-help","title":"--help()","text":"When run with this option or without any options xtrabackup displays information about how to run the program on the command line along with all supported options and variables with default values where appropriate.
"},{"location":"xtrabackup-option-reference.html#-historyname","title":"--history(=NAME)","text":"This option enables the tracking of backup history in the PERCONA_SCHEMA.xtrabackup_history table. An optional history series name may be specified that will be placed with the history record for the current backup being taken.
This option accepts a string argument that specifies the host to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-incremental-basedirdirectory","title":"--incremental-basedir(=DIRECTORY)","text":"When creating an incremental backup, this is the directory containing the full backup that is the base dataset for the incremental backups.
"},{"location":"xtrabackup-option-reference.html#-incremental-dirdirectory","title":"--incremental-dir(=DIRECTORY)","text":"When preparing an incremental backup, this is the directory where the incremental backup is combined with the full backup to make a new full backup.
"},{"location":"xtrabackup-option-reference.html#-incremental-force-scan","title":"--incremental-force-scan()","text":"When creating an incremental backup, force a full scan of the data pages in that instance.
"},{"location":"xtrabackup-option-reference.html#-incremental-history-namename","title":"--incremental-history-name(=name)","text":"This option specifies the name of the backup series stored in the PERCONA_SCHEMA.xtrabackup_history history record to base an incremental backup on. xtrabackup will search the history table looking for the most recent (highest innodb_to_lsn), successful backup in the series and take the to_lsn value to use as the starting lsn for the incremental backup. This will be mutually exclusive with --incremental-history-uuid, --incremental-basedir and --incremental-lsn. If no valid lsn can be found (no series by that name, no successful backups by that name) xtrabackup will return with an error. It is used with the --incremental option.
This option specifies the UUID of the specific history record stored in the PERCONA_SCHEMA.xtrabackup_history to base an incremental backup on. --incremental-history-name, --incremental-basedir and --incremental-lsn. If no valid lsn can be found (no success record with that UUID) xtrabackup will return with an error. It is used with the \u2013incremental option.
When creating an incremental backup, you can specify the log sequence number (LSN) instead of specifying --incremental-basedir. For databases created in 5.1 and later, specify the LSN as a single 64-bit integer.
Important
If a wrong LSN value is specified (a user error which Percona XtraBackup does not detect), the backup is unusable. Be careful!
"},{"location":"xtrabackup-option-reference.html#-innodbname","title":"--innodb([=name])","text":"This option is ignored for MySQL option compatibility.
"},{"location":"xtrabackup-option-reference.html#-innodb-miscellaneous","title":"--innodb-miscellaneous()","text":"There is a large group of InnoDB options that are normally read from the my.cnf configuration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify these explicitly. These options have the same behavior in InnoDB and XtraDB:
The path to the keyring file. Combine this option with --xtrabackup-plugin-dir.
This option specifies the number of seconds xtrabackup waits between starting FLUSH TABLES WITH READ LOCK and killing those queries that block it. Default is 0 seconds, which means xtrabackup will not attempt to kill any queries. In order to use this option xtrabackup user should have the PROCESS and SUPER privileges. Where supported, xtrabackup automatically uses Backup locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
This option specifies which types of queries should be killed to unblock the global lock. Default is \u201cselect\u201d.
"},{"location":"xtrabackup-option-reference.html#-lock-ddl","title":"--lock-ddl()","text":"Issue LOCK TABLES FOR BACKUP if it is supported by server (otherwise use LOCK INSTANCE FOR BACKUP) at the beginning of the backup to block all DDL operations.
Note
Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.
As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.
"},{"location":"xtrabackup-option-reference.html#-lock-ddl-per-table","title":"--lock-ddl-per-table()","text":"Lock DDL for each table before xtrabackup starts to copy it and until the backup is completed.
Note
As of Percona XtraBackup 8.0.15, the \u2013lock-ddl-per-table option is deprecated. Use the \u2013lock-ddl option instead.
If LOCK TABLES FOR BACKUP or LOCK INSTANCE FOR BACKUP does not return within given timeout, abort the backup.
This option is ignored for MySQL
"},{"location":"xtrabackup-option-reference.html#-log-bin","title":"--log-bin()","text":"The base name for the log sequence.
"},{"location":"xtrabackup-option-reference.html#-log-bin-indexname","title":"--log-bin-index(=name)","text":"File that holds the names for binary log files.
"},{"location":"xtrabackup-option-reference.html#-log-copy-interval","title":"--log-copy-interval(=#)","text":"This option specifies the time interval between checks done by the log copying thread in milliseconds (default is 1 second).
"},{"location":"xtrabackup-option-reference.html#-login-path","title":"--login-path()","text":"Read the given path from the login file.
"},{"location":"xtrabackup-option-reference.html#-move-back","title":"--move-back()","text":"Move all the files in a previously made backup from the backup directory to their original locations. As this option removes backup files, it must be used with caution.
"},{"location":"xtrabackup-option-reference.html#-no-backup-locks","title":"--no-backup-locks()","text":"Explicity disables the --backup-locks option which is enabled by default.
The default options are only read from the login file.
"},{"location":"xtrabackup-option-reference.html#-no-lock","title":"--no-lock()","text":"Disables table lock with FLUSH TABLES WITH READ LOCK. Use it only if all your tables are InnoDB and you do not care about the binary log position of the backup. This option shouldn\u2019t be used if there are any DDL statements being executed or if any updates are happening on non-InnoDB tables (this includes the system MyISAM tables in the mysql database), otherwise it could lead to an inconsistent backup. Where supported xtrabackup will automatically use Backup locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables. If you are considering to use this because your backups are failing to acquire the lock, this could be because of incoming replication events are preventing the lock from succeeding. Please try using --safe-slave-backup to momentarily stop the replication replica thread, this may help the backup to succeed and you do not need to use this option.
Implemented in Percona XtraBackup 8.0.21.
The --no-server-version-check option disables the server version check.
The default behavior runs a check that compares the source system version to the Percona XtraBackup version. If the source system version is higher than the XtraBackup version, the backup is aborted with a message.
Adding the option overrides this check, and the backup proceeds, but there may be issues with the backup.
See Server Version and Backup Version Comparison for more information.
"},{"location":"xtrabackup-option-reference.html#-no-version-check","title":"--no-version-check()","text":"This option disables the version check. If you do not pass this option, the automatic version check is enabled implicitly when xtrabackup runs in the --backup mode. To disable the version check, you should pass explicitly the --no-version-check option when invoking xtrabackup.
When the automatic version check is enabled, xtrabackup performs a version check against the server on the backup stage after creating a server connection. xtrabackup sends the following information to the server:
-
MySQL flavour and version
-
Operating system name
-
Percona Toolkit version
-
Perl version
Each piece of information has a unique identifier. This is a MD5 hash value that Percona Toolkit uses to obtain statistics about how it is used. This is a random UUID; no client information is either collected or stored.
"},{"location":"xtrabackup-option-reference.html#-open-files-limit","title":"--open-files-limit(=#)","text":"The maximum number of file descriptors to reserve with setrlimit().
"},{"location":"xtrabackup-option-reference.html#-parallel","title":"--parallel(=#)","text":"This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer). In Percona XtraBackup 2.3.10 and newer, this option can be used with the --copy-back option to copy the user data files in parallel (redo logs and system tablespaces are copied in the main thread).
This option specifies the password to use when connecting to the database. It accepts a string argument. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-plugin-load","title":"--plugin-load()","text":"List of plugins to load.
"},{"location":"xtrabackup-option-reference.html#-portport","title":"--port(=PORT)","text":"This option accepts a string argument that specifies the port to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-prepare","title":"--prepare()","text":"Makes xtrabackup perform a recovery on a backup created with --backup, so that it is ready to use. See preparing a backup.
Print the program argument list and exit. Must be given as the first option on the command-line.
"},{"location":"xtrabackup-option-reference.html#-print-param","title":"--print-param()","text":"Makes xtrabackup print out parameters that can be used for copying the data files back to their original locations to restore them.
"},{"location":"xtrabackup-option-reference.html#-read-buffer-size","title":"--read-buffer-size()","text":"Set the read buffer size. The given value is scaled up to page size. The default size is 10MB. Use this option to increase the xbcloud/xbstream chunk size from the default size. To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.
Rebuilds indexes in a compact backup. This option only has effect when the --prepare and --rebuild-threads options are provided.
Uses the given number of threads to rebuild indexes in a compact backup. This option only has effect with the --prepare and --rebuild-indexes options.
The --register-redo-log-consumer parameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.
Implemented in Percona XtraBackup 8.0.30-23.
"},{"location":"xtrabackup-option-reference.html#-remove-original","title":"--remove-original()","text":"Implemented in Percona XtraBackup 2.4.6, this option when specified will remove .qp, .xbcrypt and .qp.xbcrypt files after decryption and decompression.
RocksDB data directory
"},{"location":"xtrabackup-option-reference.html#-rocksdb-wal-dir","title":"--rocksdb-wal-dir()","text":"RocksDB WAL directory.
"},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-age","title":"--rocksdb-checkpoint-max-age()","text":"The checkpoint cannot be older than this number of seconds when the backup completes.
"},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-count","title":"--rocksdb-checkpoint-max-count()","text":"Complete the backup even if the checkpoint age requirement has not been met after this number of checkpoints.
"},{"location":"xtrabackup-option-reference.html#-rollback-prepared-trx","title":"--rollback-prepared-trx()","text":"Force rollback prepared InnoDB transactions.
"},{"location":"xtrabackup-option-reference.html#-rsync","title":"--rsync()","text":"Uses the rsync utility to optimize local file transfers. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. This option cannot be used together with --stream.
When specified, xtrabackup will stop the replica SQL thread just before running FLUSH TABLES WITH READ LOCK and wait to start backup until Slave_open_temp_tables in SHOW STATUS is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds. The replication SQL thread will be restarted when the backup finishes. This option is implemented in order to deal with replicating temporary tables and isn\u2019t necessary with Row-Based-Replication.
Note
Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.
As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.
"},{"location":"xtrabackup-option-reference.html#-safe-slave-backup-timeoutseconds","title":"--safe-slave-backup-timeout(=SECONDS)","text":"How many seconds --safe-slave-backup should wait for Slave_open_temp_tables to become zero. Defaults to 300 seconds.
Refuse client connecting to server if it uses old (pre-4.1.1) protocol. (Enabled by default; use \u2013skip-secure-auth to disable.)
"},{"location":"xtrabackup-option-reference.html#-server-id","title":"--server-id(=#)","text":"The server instance being backed up.
"},{"location":"xtrabackup-option-reference.html#-server-public-key-path","title":"--server-public-key-path()","text":"The file path to the server public RSA key in the PEM format.
"},{"location":"xtrabackup-option-reference.html#-skip-tables-compatibility-check","title":"--skip-tables-compatibility-check()","text":"See --tables-compatibility-check.
This option is useful when backing up a replication replica server. It prints the binary log position of the source server. It also writes the binary log coordinates to the xtrabackup_slave_info file as a CHANGE MASTER command. A new replica for this source can be set up by starting a replica server on this backup and issuing a CHANGE MASTER command with the binary log position saved in the xtrabackup_slave_info file.
This option accepts a string argument that specifies the socket to use when connecting to the local database server with a UNIX domain socket. It is passed to the mysql child process without alteration. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-ssl","title":"--ssl()","text":"Enable secure connection.
"},{"location":"xtrabackup-option-reference.html#-ssl-ca","title":"--ssl-ca()","text":"Path of the file which contains list of trusted SSL CAs.
"},{"location":"xtrabackup-option-reference.html#-ssl-capath","title":"--ssl-capath()","text":"Directory path that contains trusted SSL CA certificates in PEM format.
"},{"location":"xtrabackup-option-reference.html#-ssl-cert","title":"--ssl-cert()","text":"Path of the file which contains X509 certificate in PEM format.
"},{"location":"xtrabackup-option-reference.html#-ssl-cipher","title":"--ssl-cipher()","text":"List of permitted ciphers to use for connection encryption.
"},{"location":"xtrabackup-option-reference.html#-ssl-crl","title":"--ssl-crl()","text":"Path of the file that contains certificate revocation lists. MySQL server documentation.
"},{"location":"xtrabackup-option-reference.html#-ssl-crlpath","title":"--ssl-crlpath()","text":"Path of directory that contains certificate revocation list files.
"},{"location":"xtrabackup-option-reference.html#-ssl-fips-mode","title":"--ssl-fips-mode()","text":"SSL FIPS mode (applies only for OpenSSL); permitted values are: OFF, ON, STRICT.
"},{"location":"xtrabackup-option-reference.html#-ssl-key","title":"--ssl-key()","text":"Path of file that contains X509 key in PEM format.
"},{"location":"xtrabackup-option-reference.html#-ssl-mode","title":"--ssl-mode()","text":"Security state of connection to server.
"},{"location":"xtrabackup-option-reference.html#-ssl-verify-server-cert","title":"--ssl-verify-server-cert()","text":"Verify server certificate Common Name value against host name used when connecting to server.
"},{"location":"xtrabackup-option-reference.html#-stats","title":"--stats()","text":"Causes xtrabackup to scan the specified data files and print out index statistics.
"},{"location":"xtrabackup-option-reference.html#-streamformat","title":"--stream(=FORMAT)","text":"Stream all backup files to the standard output in the specified format. Currently, this option only supports the xbstream format.
"},{"location":"xtrabackup-option-reference.html#-strict","title":"--strict()","text":"If this option is specified, xtrabackup fails with an error when invalid parameters are passed.
"},{"location":"xtrabackup-option-reference.html#-tablesname","title":"--tables(=name)","text":"A regular expression against which the full tablename, in databasename.tablename format, is matched. If the name matches, the table is backed up. See partial backups.
Enables the engine compatibility warning. The default value is ON. To disable the engine compatibility warning use --skip-tables-compatibility-check.
Filtering by regexp for table names. Operates the same way as --tables, but matched names are excluded from backup. Note that this option has a higher priority than --tables.
A file containing one table name per line, in databasename.tablename format. The backup will be limited to the specified tables.
"},{"location":"xtrabackup-option-reference.html#-target-dirdirectory","title":"--target-dir(=DIRECTORY)","text":"This option specifies the destination directory for the backup. If the directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup will succeed. xtrabackup will not overwrite existing files, however; it will fail with operating system error 17, file exists.
If this option is a relative path, it is interpreted as being relative to the current working directory from which xtrabackup is executed.
In order to perform a backup, you need READ, WRITE, and EXECUTE permissions at a filesystem level for the directory that you supply as the value of --target-dir.
Directory where temp tablespace files live, this path can be absolute.
"},{"location":"xtrabackup-option-reference.html#-throttle","title":"--throttle(=#)","text":"This option limits the number of chunks copied per second. The chunk size is 10 MB.
To limit the bandwidth to 10 MB/s, set the option to 1.
"},{"location":"xtrabackup-option-reference.html#-tls-ciphersuites","title":"--tls-ciphersuites()","text":"TLS v1.3 cipher to use.
"},{"location":"xtrabackup-option-reference.html#-tls-version","title":"--tls-version()","text":"TLS version to use, permitted values are: TLSv1, TLSv1.1, TLSv1.2, TLSv1.3.
"},{"location":"xtrabackup-option-reference.html#-tmpdirname","title":"--tmpdir(=name)","text":"Specify the directory that will be used to store temporary files during the backup
"},{"location":"xtrabackup-option-reference.html#-transition-keyname","title":"--transition-key(=name)","text":"This option is used to enable processing the backup without accessing the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and uses it to encrypt tablespace keys of tablespaces being backed up.
If --transition-key does not have any value, xtrabackup will ask for it. The same passphrase should be specified for the --prepare command.
The --use-free-memory-pct is a tech preview option. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Implemented in Percona XtraBackup 8.0.30-23, this option lets you configure the Smart memory estimation feature. The option controls the amount of free memory that can be used to --prepare a backup. The default value is 0 (zero) which defines the option as disabled. For example, if you set --use-free-memory-pct=50, then 50% of the free memory is used to prepare a backup. The maximum allowed value is 100.
This option works, only if --estimate-memory option is enabled. If the --estimate-memory option is disabled, the --use-free-memory-pct setting is ignored.
An example of how to enable the Smart memory estimation feature:
$ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n$ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\nThis option affects how much memory is allocated and is similar to innodb_buffer_pool_size. This option is only relevant in the --prepare phase or when analyzing statistics with --stats. The default value is 100MB. The recommended value is between 1GB to 2GB. Multiple values are supported if you provide the unit (for example, 1MB, 1M, 1GB, 1G).
This option specifies the MySQL username used when connecting to the server, if that\u2019s not the current user. The option accepts a string argument. See mysql \u2013help for details.
"},{"location":"xtrabackup-option-reference.html#-v","title":"-v()","text":"See --version
This option prints xtrabackup version and exits.
"},{"location":"xtrabackup-option-reference.html#-xtrabackup-plugin-dirdirname","title":"--xtrabackup-plugin-dir(=DIRNAME)","text":"The absolute path to the directory that contains the keyring plugin.
A version number identifies the product release. The product contains the latest Generally Available (GA) features at the time of that release.
8.0.30 -23 Base version Minor build versionPercona uses semantic version numbering, which follows the pattern of base version and build version. Percona assigns unique, non-negative integers in increasing order for each version release. The version number combines the base MySQL 8.0 version number and the minor build version.
Note that Percona XtraBackup numbering changed after the 8.0.14 version to align Percona XtraBackup versions with Percona Server and MySQL. Find more information in Aligning Percona XtraBackup Versions with Percona Server for MySQL blog post.
The version numbers for Percona XtraBackup 8.0.30-23 define the following information:
-
Base version - the leftmost numbers indicate the MySQL 8.0 version used as a base. An increase in base version resets the minor build version to 0.
-
Minor build version - an internal number that denotes the version of the software. A build version increases by one each time the Percona XtraBackup is released.
Percona XtraBackup 8.0.28-20 and 8.0.28-21 are multiple releases based on MySQL 8.0.28.
"},{"location":"yum-download-rpm.html","title":"Install Percona XtraBackup 8.0 using downloaded RPM packages","text":"Download RPM packages of the desired series for your architecture from the download page.
The following example downloads Percona XtraBackup 8.0.4 release package for CentOS 7:
$ wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-8.0.4/binary/redhat/7/x86_64/percona-xtrabackup-80-8.0.4-1.el7.x86_64.rpm\nInstall Percona XtraBackup by running:
$ yum localinstall percona-xtrabackup-80-8.0.4-1.el7.x86_64.rpm\nWhen installing packages manually like this, you\u2019ll need to make sure to resolve all the dependencies and install missing packages yourself.
"},{"location":"yum-files.html","title":"Files in the RPM package built for Percona XtraBackup 8.0","text":"The following tables show what data each RPM package contains:
Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page. The Percona yum repository supports popular RPM-based operating systems, including the Amazon Linux AMI.
The easiest way to install the Percona Yum repository is to install an RPM that configures yum and installs the Percona GPG key.
Specific information on the supported platforms, products, and versions is described in Percona Software and Platform Lifecycle.
"},{"location":"yum-repo.html#install-percona-xtrabackup-from-percona-yum-repository","title":"Install Percona XtraBackup from Perconayum repository","text":"-
Install the Percona yum repository by running the following command as the
rootuser or with sudo:$ sudo yum install \\\nhttps://repo.percona.com/yum/percona-release-latest.\\\nnoarch.rpm\n -
Enable the repository:
$ sudo percona-release enable-only tools release\nIf Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the `tools repository:
$ sudo percona-release enable-only tools\n -
Install Percona XtraBackup by running:
$ sudo yum install percona-xtrabackup-80\nWarning
Make sure that you have the
libevpackage installed before installing Percona XtraBackup on CentOS 6. For this operating system, thelibevpackage is available from the EPEL repositories. -
To decompress backups made using
Install theLZ4orZSTDcompression algorithm, install the corresponding package:lz4packageInstall thezstdpackage$ sudo yum install lz4\n$ sudo yum install zstd\n
See also
To install Percona XtraBackup using downloaded rpm packages, see Install with package manager.
To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup
"},{"location":"yum-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.0 on Red Hat Enterprise Linux and CentOS","text":"To completely uninstall Percona XtraBackup, remove all the installed packages:
$ yum remove percona-xtrabackup\nMost of the Linux distributions do not enable by default to accept TCP/IP connections from outside in their MySQL or Percona Server packages.
You can check it with netstat on a shell:
$ netstat -lnp | grep mysql\ntcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 2480/mysqld\nunix 2 [ ACC ] STREAM LISTENING 8101 2480/mysqld /tmp/mysql.sock\nYou should check two things:
-
there is a line starting with
tcp(the server is indeed accepting TCP connections) -
the first address (
0.0.0.0:3306in this example) is different from127.0.0.1:3306(the bind address is not localhost\u2019s).
In the first case, the first place to look is the my.cnf file. If you find the option skip-networking, comment it out or just delete it. Also check that if the variable bind_address is set, then it should not be set to localhost\u2019s but to the host\u2019s IP. Then restart the MySQL server and check it again with netstat. If the changes had no effect, then you should look at your distribution\u2019s startup scripts (like rc.mysqld). You should comment out flags like --skip-networking and/or change the bind-address.
After you get the server listening to remote TCP connections properly, the last thing to do is checking that the port (3306 by default) is indeed open. Check your firewall configurations (iptables -L) and that you are allowing remote hosts on that port (in /etc/hosts.allow).
And we\u2019re done! We have a MySQL server running which is able to communicate with the world through TCP/IP.
"},{"location":"archive/incremental_backups.html","title":"Incremental backups","text":"xtrabackup supports incremental backups. It copies only the data that has changed since the last full backup. You can perform multiple incremental backups between each full backup. You can set up a backup process with a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.
Note
Incremental backups on the MyRocks storage engine do not determine if an earlier full backup or incremental backup contains the same files. Percona XtraBackup copies all of the MyRocks files each time it takes a backup.
The write-ahead log (WAL) is immutable and append-only. All writes to the WAL are sequential. Every record has a unique, uniformly increasing log sequence number (LSN) which is assigned to page changes. Each page contains a Page LSN that shows the most recent update to that page. An incremental backup copies each page whose LSN is newer than the previous incremental or the full backup\u2019s LSN. An algorithm finds the pages that match the criteria. The algorithm reads the data pages and checks the page LSN.
Incremental backups do not actually compare the data files to the previous backup\u2019s data files. In fact, you can use --incremental-lsn to perform an incremental backup without even having the previous backup, if you know its LSN. Incremental backups simply read the pages and compare their LSN to the last backup\u2019s LSN. You still need a full backup to recover the incremental changes, however; without a full backup to act as a base, the incremental backups are useless.
Percona XtraBackup 8.0.30 removes the algorithm that used the changed page tracking feature in Percona Server for MySQL. Percona Server for MySQL 8.0.27 deprecated the changed page tracking feature.
With PXB 8.0.27 or earlier, another algorithm enabled the Percona Server for MySQL changed page tracking feature. The algorithm generates a bitmap file. The xtrabackup binary uses that bitmap file to read only those pages needed for the incremental backup. This method potentially saves resources. The backup enables the algorithm by default if the xtrabackup binary discovers the bitmap file. You can override the algorithm with --incremental-force-scan which forces a read of all pages even if the bitmap file is available.
To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. Create the full backup with a command such as the following:
$ xtrabackup --backup --target-dir=/data/backups/base --datadir=/var/lib/mysql/\nIf you look at the xtrabackup_checkpoints file, you should see contents similar to the following:
backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 1291135\nNow that you have a full backup, you can make an incremental backup based on it. Use a command such as the following:
$ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base --datadir=/var/lib/mysql/\nThe /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 1291135. If you examine the xtrabackup_checkpoints file in this directory, you should see something similar to the following:
backup_type = incremental\nfrom_lsn = 1291135\nto_lsn = 1291340\nThe meaning should be self-evident. It\u2019s now possible to use this directory as the base for yet another incremental backup:
$ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1 --datadir=/var/lib/mysql/\nThe --prepare step for incremental backups is not the same as for normal backups. In normal backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing a backup, because transactions that were uncommitted at the time of your backup may be in progress, and it is likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.
Note
If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups will be useless. After transactions have been rolled back, further incremental backups cannot be applied.
Beginning with the full backup you created, you can prepare it, and then apply the incremental differences to it. Recall that you have the following backups:
/data/backups/base\n/data/backups/inc1\n/data/backups/inc2\nTo prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base\n101107 20:49:43 InnoDB: Shutdown completed; log sequence number 1291135\nThe log sequence number should match the to_lsn of the base backup, which you saw previously.
This backup is actually safe to restore as-is now, even though the rollback phase has been skipped. If you restore it and start MySQL, InnoDB will detect that the rollback phase was not performed, and it will do that in the background, as it usually does for a crash recovery upon start. It will notify you that the database was not shut down normally.
To apply the first incremental backup to the full backup, you should use the following command:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1\nThis applies the delta files to the files in /data/backups/base, which rolls them forward in time to the time of the incremental backup. It then applies the redo log as usual to the result. The final data is in /data/backups/base, not in the incremental directory.
incremental backup from 1291135 is enabled.\nxtrabackup: cd to /data/backups/base/\nxtrabackup: This target seems to be already prepared.\nxtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(1291340)\nApplying /data/backups/inc1/ibdata1.delta ...\nApplying /data/backups/inc1/test/table1.ibd.delta ...\n.... snip\n101107 20:56:30 InnoDB: Shutdown completed; log sequence number 1291340\nAgain, the LSN should match what you saw from your earlier inspection of the first incremental backup. If you restore the files from /data/backups/base, you should see the state of the database as of the first incremental backup.
Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:
$ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2\nNote
--apply-log-only should be used when merging all incrementals except the last one. That\u2019s why the previous line doesn\u2019t contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase.
If you wish to avoid the notice that InnoDB was not shut down normally, when you applied the desired deltas to the base backup, you can run --prepare again without disabling the rollback phase.
After preparing the incremental backups, the base directory contains the same data as the full backup. To restoring this backup, you can use this command: xtrabackup --copy-back --target-dir=BASE-DIR
You may have to change the ownership as detailed on Restoring a Backup.
"},{"location":"archive/incremental_backups.html#incremental-streaming-backups-using-xbstream","title":"Incremental streaming backups using xbstream","text":"Incremental streaming backups can be performed with the xbstream streaming option. Currently backups are packed in custom xbstream format. With this feature, you need to take a BASE backup as well.
"},{"location":"archive/incremental_backups.html#make-a-base-backup","title":"Make a base backup","text":"$ xtrabackup --backup --target-dir=/data/backups\n$ xtrabackup --backup --incremental-lsn=LSN-number --stream=xbstream --target-dir=./ > incremental.xbstream\n$ xbstream -x < incremental.xbstream\n$ xtrabackup --backup --incremental-lsn=LSN-number --stream=xbstream --target-dir=./\n$ ssh user@hostname \" cat - | xbstream -x -C > /backup-dir/\"\nIn order to make a compressed backup, use the --compress option along with the --backup and --target-dir options. Percona XtraBackup supports the following compression algorithms:
quicklz
Note
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either LZ4 or Zstandard (ZSTD) compression algorithms.
To compress files using the quicklz compression algorithm, use --compress option:
$ xtrabackup --backup --compress --target-dir=/data/backup\nlz4
To compress files using the lz4 compression algorithm, set --compress option to lz4:
$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)
The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the Zstandard (ZSTD) compression algorithm. ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.
To compress files using the ZSTD compression algorithm, set --compress option to zstd:
$ xtrabackup --backup --compress=zstd --target-dir=/data/backup\nYou can specify ZSTD compression level with the --compress-zstd-level(=#) option. The defaul value is 1.
$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nIf you want to speed up the compression you can use the parallel compression, which can be enabled with --compress-threads option. The following example uses four threads for compression.
$ xtrabackup --backup --compress --compress-threads=4 --target-dir=/data/backup\nOutput should look like this:
Expected output...\n\n[01] Compressing ./imdb/comp_cast_type.ibd to /data/backup/imdb/comp_cast_type.ibd.qp\n[01] ...done\n...\n130801 11:50:24 xtrabackup: completed OK\nBefore you can prepare the backup you\u2019ll need to uncompress all the files.
If you used quicklz compression algorithm, use qpress (which is available from Percona Software repositories).
You can use the following one-liner to uncompress all the files:
$ for bf in `find . -iname \"*\\.qp\"`; do qpress -d $bf $(dirname $bf) && rm $bf; done\nIf you used lz4 compression algorithm, change this script to search for \\*.lz4 files:
$ for bf in `find . -iname \"*\\.lz4\"`; do lz4 -d $bf $(dirname $bf) && rm $bf; done\nPercona XtraBackup has implemented --decompress option that can be used to decompress the backup made using quicklz, lz4, or ZSTD compression algorithm.
$ xtrabackup --decompress --target-dir=/data/compressed/\nWhen the files are uncompressed you can prepare the backup with the --apply-log-only option:
$ xtrabackup --apply-log-only --target-dir=/data/backup/\nYou should check for a confirmation message:
Expected output130802 02:51:02 xtrabackup: completed OK!\nNow the files in /data/backup/ are ready to be used by the server.
Note
Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory users should remove the \\*.qp files.
Once the backup has been prepared you can use the --copy-back to restore the backup.
$ xtrabackup --copy-back --target-dir=/data/backup/\nThis will copy the prepared data back to its original location as defined by the datadir variable in the my.cnf file.
After the confirmation message, you should check the file permissions after copying the data back.
Expected output130802 02:58:44 xtrabackup: completed OK!\nYou may need to adjust the file permissions. The following example demonstrates how to do it recursively by using chown:
$ chown -R mysql:mysql /var/lib/mysql\nNow, your data directory contains the restored data. You are ready to start the server.
"},{"location":"archive/recipes_xbk_full.html","title":"Make a full backup","text":"Backup the InnoDB data and log files located in /var/lib/mysql/ to /data/backups/mysql/ (destination). Then, prepare the backup files to be ready to restore or use (make the data files consistent).
$ xtrabackup --backup --target-dir=/data/backup/mysql/\n$ xtrabackup --prepare --target-dir=/data/backup/mysql/\nNote
Starting with Percona Server for MySQL 8.0.30-22 only run prepare once.
"},{"location":"archive/recipes_xbk_full.html#success-criteria","title":"Success criteria","text":"- The exit status of xtrabackup is 0.
Note
You might want to set the --use-memory option to a value close to the size of your buffer pool, if you are on a dedicated server that has enough free memory. The xtrabackup Option reference contains more details.
Review Full Backups for a more detailed explanation.
"},{"location":"archive/recipes_xbk_inc.html","title":"Make an incremental backup","text":"Backup all InnoDB data and log files - located in /var/lib/mysql/ - once, then make two daily incremental backups in /data/backups/mysql/ (destination). Finally, prepare the backup files to be ready to restore or use.
Making an incremental backup requires a full backup as a base:
$ xtrabackup --backup --target-dir=/data/backups/mysql/\nIt is important that you do not run the --prepare command yet.
Suppose the full backup is on Monday, and you will create an incremental one on Tuesday:
$ xtrabackup --backup --target-dir=/data/backups/inc/tue/ \\\n--incremental-basedir=/data/backups/mysql/\nand the same policy is applied on Wednesday:
$ xtrabackup --backup --target-dir=/data/backups/inc/wed/ \\\n--incremental-basedir=/data/backups/inc/tue/\nPrepare the base backup (Monday\u2019s backup):
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/\nRoll Monday\u2019s data forward to the state on Tuesday:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/ \\\n--incremental-dir=/data/backups/inc/tue/\nRoll forward again to the state on Wednesday (without --apply-log-only):
$ xtrabackup --prepare --target-dir=/data/backups/mysql/ \\\n--incremental-dir=/data/backups/inc/wed/\nNote
Starting with Percona Server for MySQL 8.0.30-22, preparing a backup in two steps, --prepare --apply-log-only and then a second --prepare is not supported. The second --prepare skips the dynamic metadata and causes corruption.
"},{"location":"archive/recipes_xbk_inc.html#notes","title":"Notes","text":"-
You might want to set the
--use-memoryto speed up the process if you are on a dedicated server that has enough free memory. See xtrabackup Option Reference -
See Incremental Backups
Many Linux distributions only install the ssh client by default. If you don\u2019t have the ssh server installed already, the easiest way of doing it is by using your distribution\u2019s packaging system:
Using apt, run the following:
$ sudo apt install openssh-server\nUsing Red Hat Linux or a derivative, use the following:
$ sudo yum install openssh-server\nReview your distribution\u2019s documentation on how to configure the server.
"},{"location":"release-notes/8.0/8.0-3-rc1.html","title":"Percona XtraBackup 8.0-3-rc1","text":"Percona is glad to announce the release of Percona XtraBackup 8.0-3-rc1 on October 31 2018. Downloads are available from our download site and from apt and yum repositories.
This is a Release Candidate quality release and it is not intended for production. If you want a high quality, Generally Available release, use the current Stable version (the most recent stable version at the time of writing is 2.4.12 in the 2.4 series).
"},{"location":"release-notes/8.0/8.0-3-rc1.html#things-to-note","title":"Things to Note","text":"-
innobackupexwas previously deprecated and has been removed -
Due to the new MySQL redo log and data dictionary formats the Percona XtraBackup 8.0.x versions will only be compatible with MySQL 8.0.x and the upcoming Percona Server for MySQL 8.0.x
-
For experimental migrations from earlier database server versions, you will need to backup and restore and using XtraBackup 2.4 and then use
mysql_upgradefrom MySQL 8.0.x
- PXB-1655: The
--lock-ddloption is supported when backing up MySQL 8
-
PXB-1678: Incremental backup prepare with the
--apply-log-onlyoption could roll back uncommitted transactions -
PXB-1672: The MTS slave without GTID could be backed up when the
--safe-slave-backupoption was applied.
-
Date
March 16, 2020
Installing Percona XtraBackup 8.0
Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.10.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-1982: The history table showed a wrong value for
lock_time. -
PXB-2099:
copy-backdid not move undo files to the data directory. -
PXB-2107: If the undo tablespace was created in a directory which is a symbolic link to another directory then the backup failed at backup stage.
-
PXB-2118: Undo log file was renamed incorrectly to undo tablespace name after restore
-
Date
April 13, 2020
Installing Percona XtraBackup 8.0
Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
This release fixes security vulnerability CVE-2020-10997.
"},{"location":"release-notes/8.0/8.0.12.html","title":"Percona XtraBackup 8.0.12","text":"-
Date
May 27, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.12 now supports backup and restore processing for all versions of MySQL; previous versions of Percona XtraBackup will not work with MySQL 8.0.20 and higher.
"},{"location":"release-notes/8.0/8.0.12.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-2133: Correct prtype stored for DECIMAL columns in .cfg file by \u2013export
-
PXB-2146: Amazon S3 session token support added to xbcloud
-
PXB-2179: Modify
xtrabackup prepareto shut down keyring plugin when run with \u2013generate-transition-key -
PXB-2157: Modify
xtrabackup \u2013prepareto generate cfg files for partition tables when \u2013export is used
-
Date
June 17, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.13 supports backup and restore processing for all versions of MySQL and has been tested with the latest MySQL 8.0.20.
"},{"location":"release-notes/8.0/8.0.13.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-2165: Modify xbcloud to store backups using s3 access key parameters if AWS access key env variables are set
-
PXB-2164: Modify xbcloud to return the error when the backup doesn\u2019t exist in s3 bucket
-
PXB-2127: Modify xbcloud to upload backups with empty database to min.io storage
-
PXB-2198: Modify xbcloud delete to return the error when the backup doesn\u2019t exist in s3 bucket
-
PXB-2155: Correct corruption when redo logs are encrypted using xtrabackup \u2013backup \u2013compress=lz4
-
PXB-2166: Modify xbcloud to check for bucket existence and return
okstatus when domain name can be resolved
-
Date
August 31, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.14 supports backup and restore processing for all versions of MySQL and has been tested with the latest MySQL 8.0.21.
"},{"location":"release-notes/8.0/8.0.14.html#improvements","title":"Improvements","text":"-
PXB-1605: Document how to use Percona Xtrabackup with Docker
-
PXB-2252: Introduce debug option to print redo log records scanned and applied
-
PXB-2215: Modify Import tablespace process to correctly calculate pack_length
-
PXB-2114: Document when table is ignored by full backup it is not automatically ignored for incremental backup
-
PXB-2255: Modify processing to Error out if undo truncation during backup
-
PXB-2249: Verify perl binary exists before completing version check
-
PXB-2243: Correct processing when undo truncation happens between full backup and incremental
-
PXB-2238: Provide binary tarball with shared libs and glibc suffix & minimal tarballs
-
PXB-2202: Modify Xbcloud to display an error when xtrabackup fails to create a backup
-
Date
December 14, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
NOTE: The naming structure for the releases has changed. See the aligning Percona XtraBackup versions with Percona Server for MySQL blog post for more information.
"},{"location":"release-notes/8.0/8.0.22-15.0.html#new-features","title":"New Features","text":"-
PXB-2112: xbcloud: support storage_class option with \u2013storage=s3 (Thanks to user rluisr for reporting this issue)
-
PXB-2242: Prevent back up if Source system (DB) version is higher then current PXB version
- PXB-2254: Redesign \u2013lock-ddl-per-table
-
PXB-793: Fix syntax error when executing \u2013lock-ddl-per-table queries
-
PXB-953: Improve stdout for the end of usage of \u2013lock-ddl-per-table
-
PXB-787: Modify scan to provide correct status when finding corrupted tablespace
-
PXB-2314: Handle Disk format changes introduced in MySQL 8.0.22 release
-
PXB-2279: Modify to look for prefixes in xtrabackup_tablespaces* without extensions (Thanks to user mrmainnet for reporting this issue)
-
PXB-2336: Modify to check to see if first block is an all-zero block and process accordingly
-
PXB-2275: Modify backup processing to add validations if an encrypted table is created
-
PXB-2272: Modify regexp to consider all #sql as temporary tables
-
PXB-2257: Modify \u2013lock-ddl-per-table to properly close database connection
-
Date
March 22, 2021
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
This release fixes the security vulnerability CVE-2020-29488
"},{"location":"release-notes/8.0/8.0.23-16.0.html#improvements","title":"Improvements","text":"-
PXB-2280: Provide SELinux and AppArmor default policies
-
PXB-1979: Enable \u2013lock-ddl by default to prevent corruption of the backup
-
PXB-2274: Correct restore processing when there are DML statements running during backup stage by writing the last_checkpoint and LSN from ps.log_status instead of the redo log (Thanks to user Li Biao for reporting this issue)
-
PXB-2430: Correct build failure by removing the dependency of no-server-version-check with version-check (Thanks to user agarner for reporting this issue)
-
PXB-2415: Add build dependencies to correct Debian/Ubuntu packages in docker (Thanks to user Matt Cole for reporting this issue)
-
PXB-2429: Correct Backup failure for encrypted tablespace by skipping the encryption reset operation during the redo scan phase
-
PXB-2418: Remove PROTOBUF_LITE_LIBRARY - it is not used and is not needed
-
PXB-2395: Update versions for xbstream and xbcrypt
-
PXB-2394: Correct spellings in xbcloud help
-
PXB-2357: Correct hang in backup with redo log archive by using block number instead of checksum (checksum of redo file and archive file can be different)
-
PXB-2180: Correct incremental prepare failure with logical redo by skipping the apply of logical redos (MLOG_TABLE_DYNAMIC_META) during the incremental prepare (except the last prepare).
-
Date
June 3, 2021
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.25-17.0.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-2442 : Backup cannot be decompressed using the AppArmor profile
-
PXB-2444 : The xbcloud binary fails to upload backup with enforcing SELinux mode
-
PXB-2443 : Version check fails with AppArmor profile
-
PXB-2445 : Initializing the libgcrypt in xbcloud
-
PXB-2455 : XtraBackup prepare fails if the checkpoint LSN is greater than the last LSN
-
PXB-2473 : SELinux errors in audit logs
-
PXB-1462 : Long gtid_executed breaks \u2014history functionality
-
PXB-2369 : Issues with \u201cInstalling\u201d page in XtraBackup documentation
-
PXB-2457 : Incorrect binlog names if binlog name contains periods
-
PXB-2106 : Copy-back creates wrong binlog.index
-
Date
September 2, 2021
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.26-18.0.html#improvements","title":"Improvements","text":"-
PXB-2477: The xbcloud should retry on error and utilize incremental backoff (Thanks to Baptiste Mille-Mathias for reporting this issue)
-
PXB-2580: With the xbcloud binary, a chunk-upload on an SSL connect error to S3 was not retried (Thanks to Tim Vaillancourt for providing the patch)
-
PXB-2317: Remove the obsolete LOCKLESS binary log functionality since the
performance_schema.log_statustable is now used to get the log information on all the storages without locking them
-
PXB-2101: The Prepare step fails due to duplicate Serialized Dictionary Information (SDI) (Thanks to Fungo Wang for reporting this issue)
-
PXB-1504: The FIND_GCRYPT macro is broken (Thanks to Maxim Bublis for reporting this issue)
-
PXB-1961: The Prepare step of a full backup fails for encrypted tables with a generic error
-
PXB-2585: XtraBackup fails to backup archive RocksDB Write-Ahead Log (WAL) files
-
Date
February 2, 2022
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.27-19.0.html#release-highlights","title":"Release Highlights","text":"The following list contains a number of the bug fixes for MySQL 8.0.27, provided by Oracle, and included in Percona Server for MySQL:
-
The
default_authentication_pluginis deprecated. Support for this plugin may be removed in future versions. Use theauthentication_policyvariable. -
The
binaryoperator is deprecated. Support for this operator may be removed in future versions. UseCAST(... AS BINARY). -
When a parent table updated or deleted a row, this operation initiated a cascading
SET NULLoperation on the child table. On the child table, a virtual column value could be set to NULL instead of the value derived from the base column value.
Find the full list of bug fixes and changes in the MySQL 8.0.27 Release Notes.
"},{"location":"release-notes/8.0/8.0.27-19.0.html#new-features","title":"New Features","text":"- PXB-1883: Added support for Microsoft Azure Cloud Storage in the xbcloud binary. (Thanks to Ivan for reporting this issue)
- PXB-2434: Use MySQL page tracking for incremental backup
-
PXB-1741: PS startup displays errors for databases excluded during partial backup
-
PXB-2614: The
xbstreambinary was enhanced to support sparse files on the XFS file system. -
PXB-2608: Upgrade the Vault API to V2 (Thanks to Benedito Marques Magalhaes for reporting this issue)
-
PXB-2543: Fix that adds
IB_EXPORT_CFG_VERSION_V6when exporting a.cfgfile for a table (Thanks to Peng Gao for reporting this issue) -
PXB-2465: Fix for querying the
ps.log_statuswhen group replication channels are items on that list. XtraBackup skips the name if it matches eithergroup_replication_applierorgroup_replication_recovery.(Thanks to Galamb Gerg\u0151 for reporting this issue) -
PXB-2625: The default version of CURL in Debian Buster failed to identify a TLS HTTP2 connection as closed and attempted to reuse the connection. (Thanks to Johan Andersson for reporting this issue)
-
Date
April 26, 2022
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.28-20.0.html#release-highlights","title":"Release Highlights","text":"The log statements structure in Percona XtraBackup has been improved. Now, the error logs have a standard structure. The uniformity of the headers makes it easier to follow an operation\u2019s progress or review the log to diagnose issues. The improved log structure is displayed in the backup, prepare, move-back/copy-back error logs.
Each log statement has the following attributes:
-
Timestamp - a timestamp for when the event occurred in a UTC format.
-
Severity - the severity level of a statement indicates the importance of an event.
-
ID - this identifier is currently not used but may be used in future versions.
-
Context - the name of the module that issued the log statement, such as XtraBackup, InnoDB, or Server.
-
Message - a description of the event generated by the module.
- PXB-2670: Improves the error logging framework in Percona XtraBackup.
-
PXB-1676: The error message after creating a backup was incorrect.
-
PXB-2716: Fix for a compilation error on systems without fallocate. (Thanks to user Bo98 for providing the patch to fix this issue.)
-
PXB-2506: During copy back and move back operations, added a check if the backup is fully prepared. Percona XtraBackup throws an error if the backup is not prepared or was prepared with \u2013apply-log-only.
-
PXB-2662: Percona XtraBackup exited during prepare when creating an incremental backup using page tracking.
-
PXB-2714: Fix for a memory leak on the keyring component.
-
PXB-2697: Undefined symbols in macOS resulted in an error when linked with Percona XtraBackup 8.0.27.
-
PXB-2722: Fix for when via command line, a password, passed using the -p option, was written into the backup tool_command in xtrabackup_info.
-
The Percona XtraBackup installation instructions
-
The Percona XtraBackup downloads
-
The Percona XtraBackup GitHub location
-
To contribute to the documentation, review the Documentation Contribution Guide
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.28-21.0.html#release-highlights","title":"Release Highlights","text":"Percona XtraBackup adds support for the Amazon Key Management Service (KMS) component.
"},{"location":"release-notes/8.0/8.0.28-21.0.html#new-features","title":"New Features","text":"- PXB-2721: Implements support for the Amazon Key Management Service component in Percona XtraBackup.
-
PXB-2761: Fix for a compilation error in GCC 11. (Thanks to user tcoyvwac for providing the patch to fix this issue.)
-
PXB-2422: The extraction of files failed when a file was located in another directory.
-
The Percona XtraBackup installation instructions
-
The Percona XtraBackup downloads
-
The Percona XtraBackup GitHub location
-
To contribute to the documentation, review the Documentation Contribution Guide
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.29-22.0.html#release-highlights","title":"Release Highlights","text":"Percona XtraBackup 8.0.29-22 adds new redo log types to support the changes in the INSTANT algorithm behavior. MySQL 8.0.29 extended the support for ALGORITHM=INSTANT to allow columns to be added to any position in a table and column drops. Older versions of Percona XtraBackup are incompatible with MySQL 8.0.29 because of this new functionality.
Note
MySQL 8.0.29 extended the support of the ALTER TABLE \u2026 DROP COLUMNS statement to use ALGORITHM=INSTANT. These operations only modify the metadata in the data directory and do not affect the physical structure. The ALGORITHM=INSTANT is the default setting for supported DDL operations. Prior to MySQL 8.0.29, an ALGORITHM=INSTANT column could only be added as the last table column. As of MySQL 8.0.29, the column can be added to any table position. The ability to add or drop a column in any position increases the complexity of crash recovery since the redo log does not have the logical positions from the data dictionary. The redo log contains the physical order for the table.
Percona Server for MySQL 8.0.29-21 contains fixes for these issues. Percona XtraBackup 8.0.29 can backup and restore tables that used the INSTANT algorithm in ADD/DROP in Percona Server for MySQL. However, MySQL 8.0.29, if the server contains tables modified by INSTANT ADD/DROP, is unsafe for backups at this time. Percona XtraBackup detects the tables modified by INSTANT ADD/DROP and generates an error. This error lists the affected tables and provides instructions to convert the modified tables to regular tables.
-
The Percona XtraBackup installation instructions
-
The Percona XtraBackup downloads
-
The Percona XtraBackup GitHub location
-
To contribute to the documentation, review the Documentation Contribution Guide
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
For paid support, managed services or consulting services, contact Percona Sales.
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.30-23.0.html#release-highlights","title":"Release highlights","text":"The Take an incremental backup using page tracking feature is Generally Available (GA).
The following features and improvements introduced in Percona XtraBackup 8.0.30-23 are available only in tech preview:
-
Percona XtraBackup adds support for
Zstandard (ZSTD)compression algorithm.ZSTDis a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. -
Percona XtraBackup implements the
Smart memory estimationfeature to compute the memory required to--preparea backup. Percona XtraBackup has extended the crash recovery logic to extract the formula used to allocate memory. Now, in the backup phase, while copying redo log entries, Percona XtraBackup computes the required memory for--prepare. Percona XtraBackup also takes into consideration the number of InnoDB pages to be fetched from the disk. Percona XtraBackup checks the server\u2019s available free memory and uses that memory up to the limit specified in the--use-free-memory-pctoption to run--prepare.
PXB-2669: Implements support for Zstandard (ZSTD) compression algorithm.
PXB-2710: Implements support for Smart Memory Estimation.
PXB-2844: Adjusts the logic to detect multi-threaded slave.
"},{"location":"release-notes/8.0/8.0.30-23.0.html#bug-fixes","title":"Bug fixes","text":"PXB-2681: Percona XtraBackup got stuck during incremental backup prepare of encrypted tables.
PXB-2517: Check if files-from is successfully closed before rsync (Thanks to Garen Chan for the patch.)
PXB-2702: Backup failed when undo truncation log is present (Thanks to Dmitry Smal for his contribution to the initial patch.)
PXB-2729: Percona XtraBackup compilation failed if GTest packages are installed.
PXB-2810: Percona XtraBackup crashed when AIO initialization failed (Thanks to Yan Huang for the patch.)
PXB-2840: Xbcloud attempted to create a non-existing bucket.
PXB-2874: When Percona XtraBackup generated a new master key on --copy/move-back, the master key id was reset back to 1.
PXB-2809: wsrep_sync_wait<>0 caused Lock wait timeout exceeded; try restarting transaction error (Thanks to Frank Well for providing the initial patch.)
Percona XtraBackup 8.0.30 supports Oracle Linux/Red Hat Enterprise Linux 9.
"},{"location":"release-notes/8.0/8.0.30-23.0.html#useful-links","title":"Useful links","text":"-
The Percona XtraBackup GitHub location
-
Contribute to the documentation
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.31-24.0.html#release-highlights","title":"Release highlights","text":"This release adds the ability to use an IAM instance profile when running xbcloud from an EC2 instance. An IAM instance profile does not need the --s3-secret-key nor the --s3-access-key but the profile has a six-hour lifespan.
With this version, qpress/QuickLZ are deprecated.
PXB-2856: Use IAM instance profile when running xbcloud from an EC2 instance.
PXB-2869: A warning is printed on backup that qpress/QuickLZ is deprecated.
"},{"location":"release-notes/8.0/8.0.31-24.0.html#bug-fixes","title":"Bug fixes","text":"PXB-2890: XtraBackup did not compress already compressed tables.
PXB-2928: XtraBackup exits with signal 11 when taking a backup using page tracking.
PXB-2854: Fixed a memory corruption issue in QuickLZ.
PXB-2925: The PXB start_lsn, during the prepare phase, did not match the redo file start LSN when a wait was added in the backup phase after calling recv_find_max_checkpoint and before copying the checkpoint header.
PXB-2964: XtraBackup required a very high open files limit.
PXB-2959: XtraBackup now compiles on platforms with OpenSSL 3.0.
PXB-2971: XtraBackup copied local manifest files which caused node failures when the files overrode settings on another node.
"},{"location":"release-notes/8.0/8.0.31-24.0.html#deprecation-and-removal","title":"Deprecation and removal","text":"Starting with Percona XtraBackup 8.0.31-24, using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either the LZ4 or the Zstandard (ZSTD) compression algorithms.
-
The Percona XtraBackup GitHub location
-
Contribute to the documentation
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.32-25.0.html#release-highlights","title":"Release highlights","text":"This release merges the MySQL 8.0.32 code base. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
Starting with version 8.0.32-25, Percona XtraBackup allows instant columns in the backup for MySQL 8.0.32. This limitation still exists for backing up MySQL 8.0.31 and lower versions.
This limitation does not exist for backing up any version of Percona Server for MySQL.
"},{"location":"release-notes/8.0/8.0.32-25.0.html#useful-links","title":"Useful links","text":"The Percona XtraBackup GitHub location
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.32-26.0.html","title":"Percona XtraBackup 8.0.32-26 (2023-04-04)","text":"Release date April 4, 2023 Install instructions Install Percona XtraBackupPercona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.32-26.0.html#release-highlights","title":"Release highlights","text":"This release includes improvements and bug fixes. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
This release fixes the security vulnerability CVE-2022-25834 with PXB-2977.
This version adds the --estimate-memory parameter to enable/disable the Smart memory estimation feature during the backup phase.
PXB-2882: Removes bitmap code and variables.
PXB-2979: The Smart memory estimation feature parses metadata from the redo log to estimate the required memory to prepare a backup. On write-intensive workloads, this behavior caused the delay of the redo follow thread.
PXB-2980: Adds the --estimate-memory parameter to enable/disable the Smart memory estimation feature during the backup phase.
PXB-2264: When a table with the next auto incremental value 2 was exported using Percona XtraBackup, the table got the next auto incremental value 1 instead of 2.
PXB-2954: During prepare phase Percona XtraBackup uses Serialized Dictionary information (SDI) to create the tabels\u2019s metadata. If there was an orphan table (leftover from a upgrade) without SDI information, Percona XtraBackup failed to prepare a backup.
PXB-2970: Removed unnecessary check for redo log files at the end of backup in case the redo archive was enabled.
PXB-2972: An error after the redo apply phase caused an assertion failure instead of a graceful exit.
PXB-2977: Fixed the security vulnerability CVE-2022-25834.
PXB-2998: Due to a server bug in the upgrade, the server leaves the dictionary tables in a temporary schema instead of the mysql schema. Percona Xtrabackup can now successfully prepare these backup directories.
PXB-2999: Removed the warning message that MLOG_INDEX_LOAD redo log record was found.
PXB-3022: Reduced the memory required for --prepare phase.
The Percona XtraBackup GitHub location
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.33-27.0.html","title":"Percona XtraBackup 8.0.33-27 (2023-05-25)","text":"Release date May 25, 2023 Install instructions Install Percona XtraBackupPercona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.
"},{"location":"release-notes/8.0/8.0.33-27.0.html#release-highlights","title":"Release highlights","text":"This release merges the MySQL 8.0.33 code base. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
"},{"location":"release-notes/8.0/8.0.33-27.0.html#bug-fix","title":"Bug fix","text":"PXB-3026: DWITH_NDB=OFF option did not turn off the DWITH_NDB_CLUSTER_STORAGE_ENGINE and compiled NDB files.
The Percona XtraBackup GitHub location
Download product binaries, packages, and tarballs at Percona Product Downloads
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.33-28.0.html","title":"Percona XtraBackup 8.0.33-28 (2023-07-19)","text":"Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.
"},{"location":"release-notes/8.0/8.0.33-28.0.html#release-highlights","title":"Release highlights","text":"This release includes improvements, features, and bug fixes. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
Percona XtraBackup 8.0.33-28 implements a new data sink that uses the first in, first out (FIF0) method. With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to object storage. This feature is in tech preview.
Percona XtraBackup 8.0.33-28 implements the Dictionary cache feature. Dictionary cache improves the performance of xtrabackup --prepare operation.
PXB-2514: Implements a new data sink that uses FIFO (named pipes).
-
PXB-2955: Implements dictionary cache.
-
PXB-3033: Executes
STOP SLAVEconditionally depending on--lock-ddl. -
PXB-3066: Fixes compilation issues on Debian 12.
-
PXB-2993: Makes Percona XtraBackup compatible with procps-4.
-
PXB-2811: Fix for several Percona XtraBackup compilation warnings.
-
PXB-3075: Percona XtraBackup 8.0.33 could not prepare backups from 8.0.29 and previous versions.
Percona XtraBackup 8.0.33-28 supports Debian 12.
"},{"location":"release-notes/8.0/8.0.33-28.0.html#useful-links","title":"Useful links","text":"Install Percona XtraBackup 8.0
The Percona XtraBackup GitHub location
Download product binaries, packages, and tarballs at Percona Product Downloads
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.4.html","title":"Percona XtraBackup 8.0.4","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.4 on December 10, 2018. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
This release of Percona Xtrabackup is a General Availability release ready for use in a production environment.
"},{"location":"release-notes/8.0/8.0.4.html#please-note-the-following-about-this-release","title":"Please note the following about this release:","text":"-
The deprecated innobackupex has been removed. Use the xtrabackup command to back up your instances:
$ xtrabackup --backup --target-dir=/data/backup -
When migrating from earlier database server versions, backup and restore and using Percona XtraBackup 2.4 and then use
mysql_upgradefrom MySQL 8.0.x -
If using
yumoraptrepositories to install Percona XtraBackup 8.0.4, ensure that you have enabled the new tools repository. You can do this with the percona-release enable tools release command and then install the percona-xtrabackup-80 package.
All Percona software is open-source and free. We are grateful to the community for the invaluable contributions to Percona XtraBackup. We would especially like to highlight the input of Alexey Kopytov who has been actively offering improvements and submitting bug reports for Percona XtraBackup.
"},{"location":"release-notes/8.0/8.0.4.html#new-features","title":"New Features","text":"- Percona XtraBackup 8.0.4 is based on MySQL 8.0.13 and fully supports Percona Server for MySQL 8.0 series and MySQL 8.0 series.
-
PXB-1699: xtrabackup \u2013prepare could fail on backups of MySQL\u00a08.0.13 databases
-
PXB-1704: xtrabackup \u2013prepare could hang while performing insert buffer merge
-
PXB-1668: When the \u2013throttle option was used, the applied value was different from the one specified by the user (off by one error)
-
PXB-1679: PXB could crash when
ALTER TABLE ... TRUNCATE PARTITIONcommand was run during a backup without locking DDL
Percona is glad to announce the release of Percona XtraBackup 8.0.5 on March 4, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.5 introduces the support of undo tablespaces created using the new syntax (CREATE UNDO TABLESPACE) available since MySQL 8.0.14. Percona XtraBackup also supports the binary log encryption introduced in MySQL 8.0.14.
Two new options were added to xbstream. Use the --decompress option with xbstream to decompress individual qpress files. With the --decompress-threads option, specify the number of threads to apply when decompressing. Thanks to Rauli Ikonen for this contribution.
This release of Percona XtraBackup is a General Availability release ready for use in a production environment.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.5.html#please-note-the-following-about-this-release","title":"Please note the following about this release:","text":"-
The deprecated innobackupex has been removed. Use the xtrabackup command to back up your instances:
$ xtrabackup --backup --target-dir=/data/backup -
When migrating from earlier database server versions, backup and restore and using Percona XtraBackup 2.4 and then use
mysql_upgradefrom MySQL 8.0.x -
If using
yumoraptrepositories to install Percona Xtrabackup 8.0.5, ensure that you have enabled the new tools repository. You can do this with the percona-release enable tools release command and then install the percona-xtrabackup-80 package.
-
PXB-1548: Percona XtraBackup enables updating the
ib_buffer_poolfile with the latest pages present in the buffer pool using the--dump-innodb-buffer-pooloption. Thanks to Marcelo Altmann for contribution. -
PXB-1768: Added support for undo tablespaces created with the new MySQL 8.0.14 syntax.
-
PXB-1781: Added support for binary log encryption introduced in MySQL 8.0.14.
-
PXB-1797: For xbstream, two new options were added. The
--decompressoption enables xbstream to decompress individual qpress files. The--decompress-threadsoption controls the number of threads to apply when decompressing. Thanks to Rauli Ikonen for this contribution.
-
Using
--lock-ddl-per-tablecaused the server to scan all records of partitioned tables which could lead to the \u201cout of memory\u201d error. Bugs fixed PXB-1691 and PXB-1698. -
When Percona XtraBackup was started run with the
--slave-info, incorrect coordinates were written to the xtrabackup_slave_info file.. Bug fixed PXB-1737 -
Percona XtraBackup could crash at the prepare stage when making an incremental backup if the variable
innodb-rollback-segmentswas changed after starting the MySQL Server. Bug fixed PXB-1785. -
The full backup could fail when Percona Server for MySQL was started with the
--innodb-encrypt-tablesparameter. Bug fixed PXB-1793.
Other bugs fixed: PXB-1632, PXB-1715, PXB-1770, PXB-1771, PXB-1773.
"},{"location":"release-notes/8.0/8.0.6.html","title":"Percona XtraBackup 8.0.6","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.6 on May 9, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
In version 8.0.6, Percona XtraBackup introduces the support of the MyRocks storage engine with Percona Server for MySQL version 8.0.15-6 or higher.
Percona XtraBackup 8.0.6 enables saving backups to an Amazon S3, MinIO, and Google Cloud Storage (using interoperability mode) when using xbcloud. The following example demonstrates how to use an Amazon S3 storage to make a full backup:
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='s3.amazonaws.com' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n${date -I}-full_backup\n\nAll *Percona* software is open-source and free\n-
The MyRocks storage engine is now supported with Percona XtraBackup. More information in PXB-1754.
-
Amazon S3 is now supported in xbcloud. More information in PXB-1813.
-
Percona XtraBackup could fail to restore the undo tablespace created during or before incremental backup. Bug fixed PXB-1780.
-
A backup could fail if
log_bin_indexwas defined inmy.cnf. Bug fixed PXB-1801. -
When the row format was changed during the backup, xtrabackup could crash during the incremental prepare stage. Bug fixed PXB-1824.
-
During the
preparephase, Percona XtraBackup could freeze and never finish execution. Bug fixed PXB-1819. -
Percona XtraBackup could crash during the
preparestage when making a backup of a host running MySQL Server v8.0.16. Bug fixed PXB-1839.
Other bugs fixed: PXB-1809, PXB-1810, PXB-1832, PXB-1837.
"},{"location":"release-notes/8.0/8.0.7.html","title":"Percona XtraBackup 8.0.7","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.7 on August 7, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
In release 8.0.7, Percona XtraBackup enables making backups of databases that contain the encrypted system tablespace. Encrypted mysql tablespace is now also supported.
Percona XtraBackup 8.0.7 implements the support of the lz4 compression algorithm so that you could make compressed backups using lz4 (\u2013compress=lz4) in addition to the default quicklz method.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.7.html#new-features-and-improvements","title":"New Features and Improvements","text":"-
Add support of the system tablespace encryption. More information in PXB-1649.
-
Implemented the support of the lz4 compression algorithm. More information in PXB-1857.
-
When the encrypted tablespaces feature was enabled, encrypted and compressed tables were not usable on the joiner node (Percona XtraDB Cluster) via SST (State Snapshot Transfer) with the xtrabackup-v2 method. Bug fixed PXB-1867.
-
xbclouddid not update date related fields of the HTTP header when retrying a request. Bug fixed PXB-1874. -
xbclouddid not retry to send the request after receiving the HTTP 408 error (request timeout). Bug fixed PXB-1875. -
xtrabackupdid not accept decimal fractions as values of theinnodb_max_dirty_pages_pctoption. Bug fixed PXB-1807. -
If the user tried to merge an already prepared incremental backup, a misleading error was produced without informing that incremental backups may not be used twice. Bug fixed PXB-1862.
Other bugs fixed: PXB-1493, PXB-1557, PXB-1887, PXB-1870, PXB-1879, PXB-1901.
"},{"location":"release-notes/8.0/8.0.8.html","title":"Percona XtraBackup 8.0.8","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.8 on November 21, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.8.html#new-features-and-improvements","title":"New Features and Improvements","text":"-
Support log archiving feature in PXB 8.0. More information in PXB-1912 and in the Redo Log section of MySQL documentation
-
For the MyRocks storage engine, support the creation of renewable checkpoints (controlled via
--rocksdb-checkpoint-max-ageand--rocksdb-checkpoint-max-count) to minimize the amount of binary logs to apply after the backup was completed. Using renewable checkpoints, Percona XtraBackup only copies the SST files that were created after the previous checkpoint. More information in PXB-1915. -
Two options (\u2013-backup-lock-timeout and -\u2013backup-lock-retry-count) were added to enable the configuring of the timeout for acquiring metadata locks in
FLUSH TABLES WITH READ LOCK,LOCK TABLE FOR BACKUP, andLOCK BINLOG FOR BACKUPstatements. More information in PXB-1914
-
An encrypted table could not be restored when
ADD INDEXorDROP INDEXcommands had been run on the table. Bug fixed PXB-1905 -
In some cases
xtrabackup --preparecould fail to decrypt a table but reported that the operation completed ok. Bug fixed PXB-1936 -
xtrabackup --move-backdid not complete successfully when the encrypted binlog file. Bug fixed PXB-1937. -
Percona XtraBackup could crash during the prepare stage when making an incremental
backup when a multi valued index was being added or dropped for JSON data. Bug fixed PXB-1913.
Other bugs fixed: PXB-1928, PXB-1938, PXB-1951, PXB-1953, PXB-1954.
"},{"location":"release-notes/8.0/8.0.9.html","title":"Percona XtraBackup 8.0.9","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.9 on December 16, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.9.html#bugs-fixed","title":"Bugs Fixed","text":"- Sometime between December 3rd and December 10th, a change was introduced in that caused an incompatibility with our Percona XtraBackup
xbcloudutility. Bug fixed PXB-1978.
This documentation is for the latest release: Percona XtraBackup 8.0.33-28 (Release Notes).
Percona XtraBackup is an open source hot backup utility, for MySQL - based servers, that keeps your database fully available during planned maintenance windows.
Whether it is a 24x7 highly loaded server or a low-transaction-volume environment, Percona XtraBackup is designed to make backups a seamless procedure without disrupting the performance of the server in a production environment. Percona XtraBackup (PXB) is a 100% open source backup solution with a commercial support available for organizations who want to benefit from comprehensive, responsive, and cost-flexible database support for MySQL.
"},{"location":"index.html#supported-storage-engines","title":"Supported storage engines","text":"Percona XtraBackup can back up data from InnoDB, XtraDB, MyISAM, and MyRocks tables on MySQL 8.0 servers as well as Percona Server for MySQL with XtraDB, Percona Server for MySQL 8.0, and Percona XtraDB Cluster 8.0.
Version updates
Version 8.0.6 and later supports the MyRocks storage engine. An incremental backup on the MyRocks storage engine does not determine if an earlier full or incremental backup contains the same files. Percona XtraBackup copies all MyRocks files each time it takes a backup. Percona XtraBackup does not support the TokuDB storage engine. Find more information in Percona TokuBackup.
"},{"location":"index.html#limitations","title":"Limitations","text":"Percona XtraBackup 8.0 does not support making backups of databases created in versions prior to 8.0 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster. As the changes that MySQL 8.0 introduced in data dictionaries, redo log and undo log are incompatible with previous versions, it is currently impossible for Percona XtraBackup 8.0 to also support versions prior to 8.0.
Due to changes in MySQL 8.0.20 released by Oracle at the end of April 2020, Percona XtraBackup 8.0, up to version 8.0.11, is not compatible with MySQL version 8.0.20 or higher, or Percona products that are based on it: Percona Server for MySQL and Percona XtraDB Cluster.
For more information, see Percona XtraBackup 8.x and MySQL 8.0.20
Learn more about other Percona products
Percona Distribution for MySQL 8.0
"},{"location":"404.html","title":"404 - Not Found","text":"We can\u2019t find the page you are looking for. Try using the Search or return to the homepage.
"},{"location":"about-xtrabackup.html","title":"About Percona XtraBackup","text":"Percona XtraBackup is the world\u2019s only open source, free MySQL hot backup software that performs non-blocking backups for InnoDB and XtraDB databases.
With Percona XtraBackup, you can achieve the following benefits:
-
Backups that complete quickly and reliably
-
Uninterrupted transaction processing during backups
-
Savings on disk space and network bandwidth
-
Automatic backup verification
-
Higher uptime due to faster restore time
Percona XtraBackup makes MySQL hot backups for all versions of Percona Server for MySQL, and MySQL. It performs streaming, compressed, and incremental MySQL backups.
Important
Percona XtraBackup 2.4 supports MySQL and Percona Server for MySQL 5.6 and 5.7 databases. Due to changes in the MySQL redo log and data dictionary formats, the Percona XtraBackup 8.0.x versions are only compatible with MySQL 8.0.x, Percona Server for MySQL 8.0.x, and compatible versions.
Percona\u2019s enterprise-grade commercial MySQL Support contracts include support for Percona XtraBackup. We recommend support for critical production deployments. Percona XtraDB Backup supports encryption.
"},{"location":"about-xtrabackup.html#supported-storage-engines","title":"Supported storage engines","text":"Percona XtraBackup can back up data from InnoDB, XtraDB, MyISAM, and MyRocks tables on MySQL 8.0 servers as well as Percona Server for MySQL with XtraDB, Percona Server for MySQL 8.0, and Percona XtraDB Cluster 8.0.
Version updates
Version 8.0.6 and later supports the MyRocks storage engine. An incremental backup on the MyRocks storage engine does not determine if an earlier full or incremental backup contains the same files. Percona XtraBackup copies all MyRocks files each time it takes a backup. Percona XtraBackup does not support the TokuDB storage engine.
See also
Percona TokuBackup
"},{"location":"about-xtrabackup.html#limitations","title":"Limitations","text":"Percona XtraBackup 8.0 does not support making backups of databases created in versions prior to 8.0 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster. As the changes that MySQL 8.0 introduced in data dictionaries, redo log and undo log are incompatible with previous versions, it is currently impossible for Percona XtraBackup 8.0 to also support versions prior to 8.0.
Due to changes in MySQL 8.0.20 released by Oracle at the end of April 2020, Percona XtraBackup 8.0, up to version 8.0.11, is not compatible with MySQL version 8.0.20 or higher, or Percona products that are based on it: Percona Server for MySQL and Percona XtraDB Cluster.
For more information, see Percona XtraBackup 8.x and MySQL 8.0.20
"},{"location":"about-xtrabackup.html#what-are-the-features-of-percona-xtrabackup","title":"What are the features of Percona XtraBackup?","text":"Here is a short list of the Percona XtraBackup features. See the documentation for more.
-
Create hot InnoDB backups without pausing your database
-
Make incremental backups of MySQL
-
Stream compressed MySQL backups to another server
-
Move tables between MySQL servers on-line
-
Create new MySQL replication replicas easily
-
Backup MySQL without adding load to the server
-
Percona XtraBackup performs throttling based on the number of IO operations per second
-
Percona XtraBackup skips secondary index pages and recreates them when a compact backup is prepared
-
Percona XtraBackup can export individual tables even from a full backup, regardless of the InnoDB version
-
Backup locks is a lightweight alternative to
FLUSH TABLES WITH READ LOCKavailable in Percona Server. Percona XtraBackup uses them automatically to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
See also
How Percona XtraBackup works
"},{"location":"about-xtrabackup.html#additional-information","title":"Additional information","text":"The InnoDB tables are locked while copying non-InnoDB data.
"},{"location":"accelerate-backup-process.html","title":"Accelerate the backup process","text":""},{"location":"accelerate-backup-process.html#copy-with-the-parallel-and-compress-threads-options","title":"Copy with the--parallel and --compress-threads options","text":"When making a local or streaming backup with xbstream option, multiple files can be copied at the same time when using the --parallel option. This option specifies the number of threads created by xtrabackup to copy data files.
To take advantage of this option either the multiple tablespaces option must be enabled (innodb_file_per_table) or the shared tablespace must be stored in multiple ibdata files with the innodb_data_file_path option. Having multiple files for the database (or splitting one into many) doesn\u2019t have a measurable impact on performance.
As this feature is implemented at the file level, concurrent file transfer can sometimes increase I/O throughput when doing a backup on highly fragmented data files, due to the overlap of a greater number of random read requests. You should consider tuning the filesystem also to obtain the maximum performance (e.g. checking fragmentation).
If the data is stored on a single file, this option will have no effect.
To use this feature, simply add the option to a local backup, for example:
$ xtrabackup --backup --parallel=4 --target-dir=/path/to/backup\nBy using the xbstream in streaming backups, you can additionally speed up the compression process with the --compress-threads option. This option specifies the number of threads created by xtrabackup for for parallel data compression. The default value for this option is 1.
To use this feature, simply add the option to a local backup, for example:
$ xtrabackup --backup --stream=xbstream --compress --compress-threads=4 --target-dir=./ > backup.xbstream\nBefore applying logs, compressed files will need to be uncompressed.
"},{"location":"accelerate-backup-process.html#the-rsync-option","title":"The--rsync option","text":"In order to speed up the backup process and to minimize the time FLUSH TABLES WITH READ LOCK is blocking the writes, the option --rsync should be used. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. xtrabackup will call the rsync twice, once before the FLUSH TABLES WITH READ LOCK and once during to minimize the time the read lock is being held. During the second rsync call, it will only synchronize the changes to non-transactional data (if any) since the first call performed before the FLUSH TABLES WITH READ LOCK. Note that Percona XtraBackup will use Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. This feature is available in Percona Server for MySQL 5.6+. Percona XtraBackup uses this automatically to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
Note
This option cannot be used together with the --stream option.
Note
As of Percona XtraBackup 8.0.30 the --stats option is deprecated and may be removed in future releases. We no longer support using the --stats option in Percona XtraBackup 8.0.29 and older versions.
The xtrabackup binary can analyze InnoDB data files in read-only mode to give statistics about them. To do this, you should use the --stats option. You can combine this with the --tables option to limit the files to examine. It also uses the --use-memory option.
You can perform the analysis on a running server, with some chance of errors due to the data being changed during analysis. Or, you can analyze a backup copy of the database. Either way, to use the statistics feature, you need a clean copy of the database including correctly sized log files, so you need to execute with --prepare twice to use this functionality on a backup.
The result of running on a backup might look like the following:
Expected output<INDEX STATISTICS>\n table: test/table1, index: PRIMARY, space id: 12, root page 3\n estimated statistics in dictionary:\n key vals: 25265338, leaf pages 497839, size pages 498304\n real statistics:\n level 2 pages: pages=1, data=5395 bytes, data/pages=32%\n level 1 pages: pages=415, data=6471907 bytes, data/pages=95%\n leaf pages: recs=25958413, pages=497839, data=7492026403 bytes, data/pages=91%\nThis can be interpreted as follows:
-
The first line simply shows the table and index name and its internal identifiers. If you see an index named
GEN_CLUST_INDEX, that is the table\u2019s clustered index, automatically created because you did not explicitly create aPRIMARY KEY. -
The estimated statistics in dictionary information is similar to the data that\u2019s gathered through
ANALYZE TABLEinside of InnoDB to be stored as estimated cardinality statistics and passed to the query optimizer. -
The real statistics information is the result of scanning the data pages and computing exact information about the index.
-
The level <X> pages: output means that the line shows information about pages at that level in the index tree. The larger<X>is, the farther it is from the leaf pages, which are level 0. The first line is the root page. -
The
leaf pagesoutput shows the leaf pages, of course. This is where the table\u2019s data is stored. -
The
external pages: output (not shown) shows large external pages that hold values too long to fit in the row itself, such as longBLOBandTEXTvalues. -
The
recsis the real number of records (rows) in leaf pages. -
The
pagesis the page count. -
The
datais the total size of the data in the pages, in bytes. -
The
data/pagesis calculated as (data/ (pages*PAGE_SIZE)) * 100%. It will never reach 100% because of space reserved for page headers and footers.
A more detailed example is posted as a MySQL Performance Blog post.
"},{"location":"analyze-table-statistics.html#script-to-format-output","title":"Script to format output","text":"The following script can be used to summarize and tabulate the output of the statistics information:
Expected outputtabulate-xtrabackup-stats.pl\n\n#!/usr/bin/env perl\nuse strict;\nuse warnings FATAL => 'all';\nmy $script_version = \"0.1\";\n\nmy $PG_SIZE = 16_384; # InnoDB defaults to 16k pages, change if needed.\nmy ($cur_idx, $cur_tbl);\nmy (%idx_stats, %tbl_stats);\nmy ($max_tbl_len, $max_idx_len) = (0, 0);\nwhile ( my $line = <> ) {\n if ( my ($t, $i) = $line =~ m/table: (.*), index: (.*), space id:/ ) {\n $t =~ s!/!.!;\n $cur_tbl = $t;\n $cur_idx = $i;\n if ( length($i) > $max_idx_len ) {\n $max_idx_len = length($i);\n }\n if ( length($t) > $max_tbl_len ) {\n $max_tbl_len = length($t);\n }\n }\n elsif ( my ($kv, $lp, $sp) = $line =~ m/key vals: (\\d+), \\D*(\\d+), \\D*(\\d+)/ ) {\n @{$idx_stats{$cur_tbl}->{$cur_idx}}{qw(est_kv est_lp est_sp)} = ($kv, $lp, $sp);\n $tbl_stats{$cur_tbl}->{est_kv} += $kv;\n $tbl_stats{$cur_tbl}->{est_lp} += $lp;\n $tbl_stats{$cur_tbl}->{est_sp} += $sp;\n }\n elsif ( my ($l, $pages, $bytes) = $line =~ m/(?:level (\\d+)|leaf) pages:.*pages=(\\d+), data=(\\d+) bytes/ ) {\n $l ||= 0;\n $idx_stats{$cur_tbl}->{$cur_idx}->{real_pages} += $pages;\n $idx_stats{$cur_tbl}->{$cur_idx}->{real_bytes} += $bytes;\n $tbl_stats{$cur_tbl}->{real_pages} += $pages;\n $tbl_stats{$cur_tbl}->{real_bytes} += $bytes;\n }\n}\n\nmy $hdr_fmt = \"%${max_tbl_len}s %${max_idx_len}s %9s %10s %10s\\n\";\nmy @headers = qw(TABLE INDEX TOT_PAGES FREE_PAGES PCT_FULL);\nprintf $hdr_fmt, @headers;\n\nmy $row_fmt = \"%${max_tbl_len}s %${max_idx_len}s %9d %10d %9.1f%%\\n\";\nforeach my $t ( sort keys %tbl_stats ) {\n my $tbl = $tbl_stats{$t};\n printf $row_fmt, $t, \"\", $tbl->{est_sp}, $tbl->{est_sp} - $tbl->{real_pages},\n $tbl->{real_bytes} / ($tbl->{real_pages} * $PG_SIZE) * 100;\n foreach my $i ( sort keys %{$idx_stats{$t}} ) {\n my $idx = $idx_stats{$t}->{$i};\n printf $row_fmt, $t, $i, $idx->{est_sp}, $idx->{est_sp} - $idx->{real_pages},\n $idx->{real_bytes} / ($idx->{real_pages} * $PG_SIZE) * 100;\n }\n}\nThe output of the above Perl script, when run against the sample shown in the previously mentioned blog post, will appear as follows:
Expected output TABLE INDEX TOT_PAGES FREE_PAGES PCT_FULL\nart.link_out104 832383 38561 86.8%\nart.link_out104 PRIMARY 498304 49 91.9%\nart.link_out104 domain_id 49600 6230 76.9%\nart.link_out104 domain_id_2 26495 3339 89.1%\nart.link_out104 from_message_id 28160 142 96.3%\nart.link_out104 from_site_id 38848 4874 79.4%\nart.link_out104 revert_domain 153984 19276 71.4%\nart.link_out104 site_message 36992 4651 83.4%\nThe columns are the table and index, followed by the total number of pages in that index, the number of pages not actually occupied by data, and the number of bytes of real data as a percentage of the total size of the pages of real data. The first line in the above output, in which the INDEX column is empty, is a summary of the entire table.
Download DEB packages of the desired series for your architecture from Percona Product Downloads.
The following example downloads Percona XtraBackup 8.0.26-18 release package for Ubuntu 20.04:
$ wget https://downloads.percona.com/downloads/Percona-XtraBackup-LATEST/Percona-XtraBackup-8.0.26-18/binary/debian/focal/x86_64/percona-xtrabackup-80_8.0.26-18-1.focal_amd64.deb\nInstall Percona XtraBackup by using dpkg. Run this command as root or use the sudo command:
$ sudo dpkg -i percona-xtrabackup-80_8.0.26-18-1.focal_amd64.deb\nWarning
When installing packages manually like this, resolve all the dependencies and install missing packages yourself.
"},{"location":"apt-files.html","title":"Files in the DEB package built for Percona XtraBackup 8.0","text":"The following tables show what data each DEB package contains.
percona-xtrabackup-80 percona-xtrabackup-test-80 The test suite for Percona XtraBackup percona-xtrabackup The older version of the Percona XtraBackup"},{"location":"apt-pinning.html","title":"Apt pinning the Percona XtraBackup 8.0 packages","text":"In some cases you might need to pin the selected packages to avoid the upgrades from the distribution repositories.
The pinning takes place in the preference file. To pin a package, set the Pin-Priority to higher numbers.
Make a new file /etc/apt/preferences.d/00percona.pref. For example, add the following to the preference file:
Package:\nPin: release o=Percona Development Team\nPin-Priority: 1001\nFor more information about the pinning, check the official debian wiki.
"},{"location":"apt-repo.html","title":"Use an APT repository to install Percona XtraBackup 8.0","text":"Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page.
Specific information on the supported platforms, products, and versions is described in Percona Release Lifecycle Overview.
Important
To prevent intermittent backup failures, update the curl utility in Debian 10.
"},{"location":"apt-repo.html#install-percona-xtrabackup-through-percona-release","title":"Install Percona XtraBackup through percona-release","text":"Percona XtraBackup, like many other Percona products, is installed with the percona-release package configuration tool.
-
Download a
DEBpackage for percona-release the repository packages from Percona web:$ wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb\n -
Install the downloaded package with dpkg. To do that, run the following commands as root or with sudo:
dpkg -i percona-release_latest.$(lsb_release -sc)_all.debOnce you install this package the Percona repositories should be added. You can check the repository setup in the
/etc/apt/sources.list.d/percona-release.listfile. -
Enable the repository:
percona-release enable-only tools releaseIf Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the
toolsrepository:percona-release enable-only tools. -
Refresh the local cache to update the package information:
$ sudo apt update\n -
Install the
percona-xtrabackup-80package:$ sudo apt install percona-xtrabackup-80\n -
To decompress backups made using
Install theLZ4orZSTDcompression algorithm, install the corresponding package:lz4packageInstall thezstdpackage$ sudo apt install lz4\n$ sudo apt install zstd\n
Note
For AppArmor profile information, see Working with AppArmor.
See also
To install Percona XtraBackup using downloaded deb packages, see Install Percona XtraBackup 8.0.
To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup 8.0
"},{"location":"apt-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.0 on Debian and Ubuntu","text":"To completely uninstall Percona XtraBackup, remove all the installed packages:
$ sudo apt remove percona-xtrabackup-80\nPercona XtraBackup is a set of the following binaries:
-
xtrabackup - a compiled C binary that provides functionality to backup a whole MySQL database instance with MyISAM, InnoDB, and XtraDB tables.
-
xbcrypt - a utility used for encrypting and decrypting backup files.
-
xbstream - a utility that allows streaming and extracting files to / from the xbstream format.
-
xbcloud - a utility used for downloading and uploading full or part of xbstream archive from / to cloud.
The recommended way to take the backup is by using the xtrabackup script. For more information on script options, see xtrabackup.
"},{"location":"binaries-overview.html","title":"Percona XtraBackup binaries overview","text":"Percona XtraBackup is a set of the following binaries:
-
xtrabackup - a compiled C binary that provides functionality to backup a whole MySQL database instance with MyISAM, InnoDB, and XtraDB tables.
-
xbcrypt - a utility used for encrypting and decrypting backup files.
-
xbstream - a utility that allows streaming and extracting files to / from the xbstream format.
-
xbcloud - a utility used for downloading and uploading full or part of xbstream archive from / to cloud.
The recommended way to take the backup is by using the xtrabackup script. For more information on script options, see xtrabackup.
"},{"location":"binary-tarball-names.html","title":"Binary tarball file names available","text":"Download the binary tarballs from Percona Product Downloads.
The following table lists the tarball types available in Linux - Generic. Select the Percona XtraBackup 8.0 version, the software or the operating system, and the type of tarball for your installation. Binary tarballs support all distributions.
After you have downloaded the binary tarballs, extract the tarball in the file location of your choice.
Type Name Operating systems Description Full percona-xtrabackup--Linux.x86_64.glibc2.12.tar.gz Built for CentOS 6 Contains binary files, libraries, test files, and debug symbols Minimal percona-xtrabackup--Linux.x86_64.glibc2.12-minimal.tar.gz Built for CentOS 6 Contains binary files, and libraries but does not include test files, or debug symbols Full percona-xtrabackup--Linux.x86_64.glibc2.17.tar.gz Compatible with any operating system except for CentOS 6 Contains binary files, libraries, test files, and debug symbols Minimal percona-xtrabackup--Linux.x86_64.glibc2.17-minimal.tar.gz Compatible with any operating system except for CentOS 6 Contains binary files, and libraries but does not include test files, or debug symbolsSelect a different software, such as Ubuntu 20.04 (Focal Fossa), for a tarball for that operating system. You can download the packages together or separately.
"},{"location":"binary-tarball.html","title":"Install Percona XtraBackup 8.0 from a binary tarball","text":"Binary tarballs contain precompiled executable files, libraries, and other dependencies and are compressed tar archives. Extract the binary tarballs to any path.
You can download the binary tarballs from the Linux - Generic section on the Percona Product Downloads.
The following command is an example of downloading the full tarball for Linux/Generic:
$ wget https://downloads.percona.com/downloads/Percona-XtraBackup-LATEST/Percona-XtraBackup-8.0.23-16/binary/tarball/percona-xtrabackup-8.0.23-16-Linux-x86_64.glibc2.17.tar.gz\nThe following instructions install Percona XtraBackup 8.0.
"},{"location":"compile-xtrabackup.html#1-install-percona-xtrabackup-from-the-git-source-tree","title":"1. Install Percona XtraBackup from the Git Source Tree","text":"Percona uses the Github revision control system for development. To build the latest Percona Server for MySQL from the source tree, you will need git installed on your system.
You can now fetch the latest Percona XtraBackup 8.0 sources:
$ git clone https://github.com/percona/percona-xtrabackup.git\n$ cd percona-xtrabackup\n$ git checkout 8.0\n$ git submodule update --init --recursive\nThe following packages and tools must be installed to compile Percona XtraBackup from source. These might vary from system to system.
Important
To build **Percona XtraBackup 8.0 from source, you must use cmake version 3. To check which version is currently installed, run cmake --version at a command prompt. If the version is not 3, install cmake3.
This cmake version may be available in your distribution as a separate package cmake3. For more information, see cmake.org
aptCentOS or Red Hat using yum sudo apt install bison pkg-config cmake devscripts debconf \\\ndebhelper automake bison ca-certificates libprocps-dev \\\nlibcurl4-openssl-dev cmake debhelper libaio-dev \\\nlibncurses-dev libssl-dev libtool libz-dev libgcrypt-dev libev-dev libprocps-dev \\\nlsb-release build-essential rsync libdbd-mysql-perl \\\nlibnuma1 socat librtmp-dev libtinfo5 vim-common \\\nliblz4-tool liblz4-1 liblz4-dev zstd python-docutils qpress\nTo install the man pages, install the python3-sphinx package first:
$ sudo apt install python3-sphinx\nPercona Xtrabackup requires GCC version 5.3 or higher. If the version of GCC installed on your system is lower then you may need to install and enable the Developer Toolset on RPM-based distributions to make sure that you use the latest GCC compiler and development tools. Then, install cmake and other dependencies:
$ sudo yum install cmake openssl-devel libaio libaio-devel automake autoconf \\\nbison libtool ncurses-devel libgcrypt-devel libev-devel libcurl-devel zlib-devel \\\nzstd vim-common procps-ng-devel\nTo install the man pages, install the python3-sphinx package first:
$ sudo yum install python3-sphinx\nAt this step, you have cmake run the commands in the CMakeList.txt file to generate the build pipeline, i.e., a native build environment that will be used to compile the source code).
-
Change to the directory where you cloned the Percona XtraBackup repository
$ cd percona-xtrabackup\n -
Create a directory to store the compiled files and then change to that directory:
$ mkdir build\n$ cd build\n -
Run cmake or cmake3. In either case, the options you need to use are the same.
Note
You can build Percona XtraBackup with man pages but this requires python-sphinx package which isn\u2019t available from that main repositories for every distribution. If you installed the python-sphinx package you need to remove the -DWITH_MAN_PAGES=OFF from previous command.
$ cmake -DWITH_BOOST=PATH-TO-BOOST-LIBRARY -DDOWNLOAD_BOOST=ON \\\n-DBUILD_CONFIG=xtrabackup_release -DWITH_MAN_PAGES=OFF -B ..\n-DWITH_BOOST For the -DWITH_BOOST parameter, specify the name of a directory to download the boost library to. This directory is created automatically in your current directory. -DWITH_MAN_PAGES To build Percona XtraBackup man pages, use ON or remove this parameter from the command line (it is ON by default). To install the man pages, install the python3-sphinx package first. -B (--build) Percona XtraBackup is configured to forbid generating the build pipeline for make in the same directory where you store your sources. The -B parameter refers to the directory that contains the source code. In this example, we use the relative path to the parent directory (..). Important
CMake Error at CMakeLists.txt:367 (MESSAGE): Please do not build in-source. Out-of source builds are highly recommended: you can have multiple builds for the same source, and there is an easy way to do cleanup, simply remove the build directory (note that \u2018make clean\u2019 or \u2018make distclean\u2019 does not work)
You can force in-source build by invoking cmake with -DFORCE_INSOURCE_BUILD=1.
To compile the source code in your build directory, use the make command.
-
Change to the
builddirectory (created at Step 2: Generating the build pipeline). -
Run the
makecommand. This command may take a long time to complete.$ make\nTo use all CPU threads and make compilation faster please use:
$ make -j$(nproc --all)\n
The following command installs all Percona XtraBackup binaries xtrabackup and tests to default location on the target system: /usr/local/xtrabackup.
Run make install to install Percona XtraBackup to the default location.
$ sudo make install\nYou may use the DESTDIR parameter with make install to install Percona XtraBackup to another location. Make sure that the effective user is able to write to the destination you choose.
$ sudo make DESTDIR=<DIR_NAME> install\nIn fact, the destination directory is determined by the installation layout (-DINSTALL_LAYOUT) that cmake applies (see Step 2: Generating the build pipeline). In addition to the installation directory, this parameter controls a number of other destinations that you can adjust for your system.
By default, this parameter is set to STANDALONE, which implies the installation directory to be /usr/local/xtrabackup.
See also
MySQL Documentation: -DINSTALL_LAYOUT
"},{"location":"compile-xtrabackup.html#6-run-percona-xtrabackup","title":"6. Run Percona XtraBackup","text":"After Percona XtraBackup is installed on your system, you may run it by using the full path to the xtrabackup command:
$ /usr/local/xtrabackup/bin/xtrabackup\nUpdate your PATH environment variable if you would like to use the command on the command line directly.
$# Setting $PATH on the command line\n$ PATH=$PATH:/usr/local/xtrabackup/bin/xtrabackup\n\n$# Run xtrabackup directly\n$ xtrabackup\nAlternatively, you may consider placing a soft link (using ln -s) to one of the locations listed in your PATH environment variable.
To view the documentation with man, update the MANPATH variable.
All the xtrabackup configuration is done through options, which behave exactly like standard MySQL program options: they can be specified either at the command-line, or through a file such as /etc/my.cnf.
The xtrabackup binary reads the [mysqld] and [xtrabackup] sections from any configuration files, in that order. That is so that it can read its options from your existing MySQL installation, such as the datadir or some of the InnoDB options. If you want to override these, just specify them in the [xtrabackup] section, and because it is read later, it will take precedence.
You don\u2019t need to put any configuration in your my.cnf if you don\u2019t want to. You can simply specify the options on the command-line. Normally, the only thing you might find convenient to place in the [xtrabackup] section of your my.cnf file is the target_dir option to default the directory in which the backups will be placed, for example:
[xtrabackup]\ntarget_dir = /data/backups/mysql/\nThis manual will assume that you do not have any file-based configuration for xtrabackup, so it will always show command-line options being used explicitly. Please see the option and variable reference for details on all the configuration options.
The xtrabackup binary does not accept exactly the same syntax in the my.cnf file as the mysqld server binary does. For historical reasons, the mysqld server binary accepts parameters with a --set-variable=<variable>=<value> syntax, which xtrabackup does not understand. If your my.cnf file has such configuration directives, you should rewrite them in the --variable=value syntax.
The xtrabackup tool requires no special configuration on most systems. However, the storage where the --target-dir is located must behave properly when fsync() is called. In particular, we have noticed that if you mount the NFS volume without the sync option the NFS volume does not sync the data. As a result, if you back up to an NFS volume mounted with the async option, and then try to prepare the backup from a different server that also mounts that volume, the data might appear to be corrupt. You can use the sync mount option to avoid this problem.
Percona XtraBackup documentation is (C)2009-2023 Percona LLC and/or its affiliates and is distributed under the Creative Commons Attribution 4.0 International License.
"},{"location":"create-compressed-backup.html","title":"Create a compressed backup","text":"Percona XtraBackup supports compressed backups. A local or streaming backup can be compressed or decompressed with xbstream.
Note
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either LZ4 or Zstandard (ZSTD) compression algorithms.
To make a compressed backup, use the --compress option along with the --backup and --target-dir options.
By default, the --compress option uses the qpress tool that you can install with the percona-release package configuration tool as follows:
$ sudo percona-release enable tools\n$ sudo apt update\n$ sudo apt install qpress\nNote
Enable the repository: percona-release enable-only tools release.
If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the tools repository: percona-release enable-only tools.
Percona XtraBackup supports the following compression algorithms:
quicklz
To compress files using the quicklz compression algorithm, use --compress option:
$ xtrabackup --backup --compress --target-dir=/data/backup\nlz4
To compress files using the lz4 compression algorithm, set --compress option to lz4:
$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)
The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the Zstandard (ZSTD) compression algorithm. ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.
To compress files using the ZSTD compression algorithm, set --compress option to zstd:
$ xtrabackup --backup --compress=zstd --target-dir=/data/backup\nYou can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.
$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nIf you want to speed up the compression you can use the parallel compression, which can be enabled with --compress-threads option. Following example will use four threads for compression:
$ xtrabackup --backup --compress --compress-threads=4 \\\n--target-dir=/data/compressed/\n...\n170223 13:00:38 [01] Compressing ./test/sbtest1.frm to /tmp/compressed/test/sbtest1.frm.qp\n170223 13:00:38 [01] ...done\n170223 13:00:38 [01] Compressing ./test/sbtest2.frm to /tmp/compressed/test/sbtest2.frm.qp\n170223 13:00:38 [01] ...done\n...\n170223 13:00:39 [00] Compressing xtrabackup_info\n170223 13:00:39 [00] ...done\nxtrabackup: Transaction log of lsn (9291934) to (9291934) was copied.\n170223 13:00:39 completed OK!\nThe next step is to prepare the backup in order to restore it.
"},{"location":"create-full-backup.html","title":"Create a full backup","text":"To create a backup, run xtrabackup with the --backup option. You also need to specify the --target-dir option, which is where the backup will be stored, if the InnoDB data or log files are not stored in the same directory, you might need to specify the location of those, too. If the target directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup will succeed.
xtrabackup will not overwrite existing files, it will fail with operating system error 17, file exists.
To start the backup process run:
$ xtrabackup --backup --target-dir=/data/backups/\nThis will store the backup at /data/backups/. If you specify a relative path, the target directory will be relative to the current directory.
During the backup process, you should see a lot of output showing the data files being copied, as well as the log file thread repeatedly scanning the log files and copying from it. Here is an example that shows the log thread scanning the log in the background, and a file copying thread working on the ibdata1 file:
160906 10:19:17 Finished backing up non-InnoDB tables and files\n160906 10:19:17 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...\nxtrabackup: The latest check point (for incremental): '62988944'\nxtrabackup: Stopping log copying thread.\n.160906 10:19:18 >> log scanned up to (137343534)\n160906 10:19:18 Executing UNLOCK TABLES\n160906 10:19:18 All tables unlocked\n160906 10:19:18 Backup created in directory '/data/backups/'\n160906 10:19:18 [00] Writing backup-my.cnf\n160906 10:19:18 [00] ...done\n160906 10:19:18 [00] Writing xtrabackup_info\n160906 10:19:18 [00] ...done\nxtrabackup: Transaction log of lsn (26970807) to (137343534) was copied.\n160906 10:19:18 completed OK!\nThe last thing you should see is something like the following, where the value of the <LSN> will be a number that depends on your system:
$ xtrabackup: Transaction log of lsn (<SLN>) to (<LSN>) was copied.\nNote
Log copying thread checks the transactional log every second to see if there were any new log records written that need to be copied, but there is a chance that the log copying thread might not be able to keep up with the amount of writes that go to the transactional logs, and will hit an error when the log records are overwritten before they could be read.
After the backup is finished, the target directory will contain files such as the following, assuming you have a single InnoDB table test.tbl1 and you are using MySQL\u2019s innodb_file_per_table option:
$ ls -lh /data/backups/\nThe result should look like this:
Expected outputtotal 182M\ndrwx------ 7 root root 4.0K Sep 6 10:19 .\ndrwxrwxrwt 11 root root 4.0K Sep 6 11:05 ..\n-rw-r----- 1 root root 387 Sep 6 10:19 backup-my.cnf\n-rw-r----- 1 root root 76M Sep 6 10:19 ibdata1\ndrwx------ 2 root root 4.0K Sep 6 10:19 mysql\ndrwx------ 2 root root 4.0K Sep 6 10:19 performance_schema\ndrwx------ 2 root root 4.0K Sep 6 10:19 sbtest\ndrwx------ 2 root root 4.0K Sep 6 10:19 test\ndrwx------ 2 root root 4.0K Sep 6 10:19 world2\n-rw-r----- 1 root root 116 Sep 6 10:19 xtrabackup_checkpoints\n-rw-r----- 1 root root 433 Sep 6 10:19 xtrabackup_info\n-rw-r----- 1 root root 106M Sep 6 10:19 xtrabackup_logfile\nThe backup can take a long time, depending on how large the database is. It is safe to cancel at any time, because xtrabackup does not modify the database.
The next step is to prepare the backup in order to restore it.
"},{"location":"create-gtid-replica.html","title":"How to create a new (or repair a broken) GTID-based replica","text":"MySQL 5.6 introduced the Global Transaction ID (GTID) support in replication. Percona XtraBackup automatically stores the GTID value in the xtrabackup_binlog_info when doing the backup of MySQL and Percona Server for MySQL 5.7 with the GTID mode enabled. This information can be used to create a new (or repair a broken) GTID-based replica.
The following command takes a backup and saves it in the /data/backups/$TIMESTAMP folder:
$ xtrabackup --backup --target-dirs=/data/backups/\nIn the destination folder, there will be a file with the name xtrabackup_binlog_info. This file contains both binary log coordinates and the GTID information.
$ cat xtrabackup_binlog_info\nmysql-bin.000002 1232 c777888a-b6df-11e2-a604-080027635ef5:1-4\nThat information is also printed by xtrabackup after taking the backup:
Expected outputxtrabackup: MySQL binlog position: filename 'mysql-bin.000002', position 1232, GTID of the last change 'c777888a-b6df-11e2-a604-080027635ef5:1-4'\nThe backup will be prepared with the following command on the Source:
$ xtrabackup --prepare --target-dir=/data/backup\nYou need to select the path where your snapshot has been taken, for example /data/backups/2013-05-07_08-33-33. If everything is ok you should get the same OK message. Now, the transaction logs are applied to the data files, and new ones are created: your data files are ready to be used by the MySQL server.
Use rsync or scp to copy the data to the destination server. If you are synchronizing the data directly to the already running replica\u2019s data directory it is advised to stop the MySQL server there.
$ rsync -avprP -e ssh /path/to/backupdir/$TIMESTAMP NewSlave:/path/to/mysql/\nAfter you copy the data over, make sure MySQL has proper permissions to access them.
$ chown mysql:mysql /path/to/mysql/datadir\nSet the gtid_purged variable to the GTID from xtrabackup_binlog_info. Then, update the information about the source node and, finally, start the replica. Run the following commands on the replica if you are using a version before 8.0.22:
# Using the mysql shell\n> SET SESSION wsrep_on = 0;\n> RESET MASTER;\n> SET SESSION wsrep_on = 1;\n> SET GLOBAL gtid_purged='<gtid_string_found_in_xtrabackup_binlog_info>';\n> CHANGE MASTER TO\n MASTER_HOST=\"$masterip\",\n MASTER_USER=\"repl\",\n MASTER_PASSWORD=\"$slavepass\",\n MASTER_AUTO_POSITION = 1;\n> START SLAVE;\nIf you are using version 8.0.22 or later, use START REPLICA instead of START SLAVE. START SLAVE is deprecated as of that release. If you are using version 8.0.21 or earlier, use START SLAVE.
If you are using a version 8.0.23 or later, run the following commands:
# Using the mysql shell\n> SET SESSION wsrep_on = 0;\n> RESET MASTER;\n> SET SESSION wsrep_on = 1;\n> SET GLOBAL gtid_purged='<gtid_string_found_in_xtrabackup_binlog_info>';\n> CHANGE REPLICATION SOURCE TO\n SOURCE_HOST=\"$masterip\",\n SOURCE_USER=\"repl\",\n SOURCE_PASSWORD=\"$slavepass\",\n SOURCE_AUTO_POSITION = 1;\n> START REPLICA;\nIf you are using version 8.0.23 or later, use CHANGE_REPLICATION_SOURCE_TO and the appropriate options. CHANGE_MASTER_TO is deprecated as of that release.
Note
The example above is applicable to Percona XtraDB Cluster. The wsrep_on variable is set to 0 before resetting the source (RESET MASTER). The reason is that Percona XtraDB Cluster will not allow resetting the source if wsrep_on=1.
The following command returns the replica status:
mysql> SHOW REPLICA STATUS\\G\n[..]\nSlave_IO_Running: Yes\nSlave_SQL_Running: Yes\n[...]\nRetrieved_Gtid_Set: c777888a-b6df-11e2-a604-080027635ef5:5\nExecuted_Gtid_Set: c777888a-b6df-11e2-a604-080027635ef5:1-5\nNote
The command SHOW SLAVE STATUS is deprecated. Use SHOW REPLICA STATUS.
We can see that the replica has retrieved a new transaction with number 5, so transactions from 1 to 5 are already on this slave.
We have created a new replica in our GTID based replication environment.
xtrabackup supports incremental backups, which means that they can copy only all the data that has changed since the last backup.
Note
Incremental backups on the MyRocks storage engine do not determine if an earlier full backup or incremental backup contains the same files. Percona XtraBackup copies all the MyRocks files each time it takes a backup.
You can perform many incremental backups between each full backup, so you can set up a backup process such as a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.
Incremental backups work because each InnoDB page contains a log sequence number, or LSN. The LSN is the system version number for the entire database. Each page\u2019s LSN shows how recently it was changed.
An incremental backup copies each page which LSN is newer than the previous incremental or full backup\u2019s LSN. An algorithm finds the pages that match the criteria. The algorithm reads the data pages and checks the page LSN.
"},{"location":"create-incremental-backup.html#use-the-changed-page-tracking-algorithm","title":"Use the changed page tracking algorithm","text":"Percona XtraBackup 8.0.30 removes the algorithm that used the changed page tracking feature in Percona Server for MySQL. Percona Server for MySQL 8.0.27 deprecated the changed page tracking feature.
With PXB 8.0.27 or earlier, another algorithm enabled the Percona Server for MySQL changed page tracking feature. The algorithm generates a bitmap file. The xtrabackup binary uses that bitmap file to read only those pages needed for the incremental backup. This method potentially saves resources. The backup enables the algorithm by default if the xtrabackup binary discovers the bitmap file. You can override the algorithm with --incremental-force-scan which forces a read of all pages even if the bitmap file is available.
To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. Create the full backup with a following command:
$ xtrabackup --backup --target-dir=/data/backups/base\nIf you look at the xtrabackup_checkpoints file, you should see similar content depending on your LSN nuber:
backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 1626007\nlast_lsn = 1626007\ncompact = 0\nrecover_binlog_info = 1\nNow that you have a full backup, you can make an incremental backup based on it. Use the following command:
$ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base\nThe /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 1626007. If you examine the xtrabackup_checkpoints file in this directory, you should see similar content to the following:
backup_type = incremental\nfrom_lsn = 1626007\nto_lsn = 4124244\nlast_lsn = 4124244\ncompact = 0\nrecover_binlog_info = 1\nfrom_lsn is the starting LSN of the backup and for incremental it has to be the same as to_lsn (if it is the last checkpoint) of the previous/base backup.
It\u2019s now possible to use this directory as the base for yet another incremental backup:
$ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1\nThis folder also contains the xtrabackup_checkpoints:
backup_type = incremental\nfrom_lsn = 4124244\nto_lsn = 6938371\nlast_lsn = 7110572\ncompact = 0\nrecover_binlog_info = 1\nNote
In this case you can see that there is a difference between the to_lsn (last checkpoint LSN) and last_lsn (last copied LSN), this means that there was some traffic on the server during the backup process.
The next step is to prepare the backup in order to restore it.
"},{"location":"create-individual-partition-backup.html","title":"Create an individual partitions backup","text":"Percona XtraBackup lets you back up individual partitions because partitions are regular tables with specially formatted names. The only requirement for this feature is to enable the innodb_file_per_table option on the server.
There is one caveat about using this kind of backup: you can not copy back the prepared backup. Restoring partial backups should be done by importing the tables.
There are three ways of specifying which part of the whole data will be backed up: regular expressions ( --tables), enumerating the tables in a file (--tables) or providing a list of databases (--databases).
The regular expression provided to this option will be matched against the fully qualified database name and table name, in the form of database-name.table-name.
If the partition 0 is not backed up, Percona XtraBackup cannot generate a .cfg file. MySQL 8.0 stores the table metadata in partition 0.
For example, this operation takes a back-up of the partition p4 from the table name located in the database imdb:
$ xtrabackup --tables=^imdb[.]name#p#p4 --backup\nIf partition 0 is not backed up, the following errors may occur:
The error messagextrabackup: export option not specified\nxtrabackup: error: cannot find dictionary record of table imdb/name#p#p4\nNote that this option is passed to xtrabackup --tables and is matched against each table of each database, the directories of each database will be created even if they are empty.
xtrabackup supports taking partial backups when the innodb_file_per_table option is enabled. There are three ways to create partial backups:
-
matching the tables names with a regular expression
-
providing a list of table names in a file
-
providing a list of databases
Warning
Do not copy back the prepared backup.
Restoring partial backups should be done by importing the tables, not by using the \u2013copy-back option. It is not recommended to run incremental backups after running a partial backup.
Although there are some scenarios where restoring can be done by copying back the files, this may lead to database inconsistencies in many cases and it is not a recommended way to do it.
For the purposes of this manual page, we will assume that there is a database named test which contains tables named t1 and t2.
Warning
If any of the matched or listed tables is deleted during the backup, xtrabackup will fail.
There are multiple ways of specifying which part of the whole data is backed up:
-
Use the
--tablesoption to list the table names -
Use the
--tables-fileoption to list the tables in a file -
Use the
--databasesoption to list the databases -
Use the
--databases-fileoption to list the databases
\u2013-tables option","text":"The first method involves the xtrabackup \u2013tables option. The option\u2019s value is a regular expression that is matched against the fully-qualified database name and table name using the databasename.tablename format.
To back up only tables in the test database, use the following command:
$ xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \\\n--tables=\"^test[.].*\"\nTo back up only the test.t1 table, use the following command:
$ xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \\\n--tables=\"^test[.]t1\"\n-\u2013tables-file option","text":"The --tables-file option specifies a file that can contain multiple table names, one table name per line in the file. Only the tables named in the file will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching. The table names must be fully-qualified in databasename.tablename format.
$ echo \"mydatabase.mytable\" > /tmp/tables.txt\n$ xtrabackup --backup --tables-file=/tmp/tables.txt\n--databases and -\u2013databases-file options","text":"The ` \u2013databases` option accepts a space-separated list of the databases and tables to backup in the databasename[.tablename] format. In addition to this list, make sure to specify the mysql, sys, and
performance_schema databases. These databases are required when restoring the databases using xtrabackup \u2013copy-back.
Note
Tables processed during the \u2013prepare step may also be added to the backup even if they are not explicitly listed by the parameter if they were created after the backup started.
$ xtrabackup --databases='mysql sys performance_schema test ...'\n--databases-file option","text":"The \u2013databases-file option specifies a file that can contain multiple databases and tables in the databasename[.tablename] format, one element name per line in the file. Names are matched exactly, case-sensitive, with no pattern or regular expression matching.
Note
Tables processed during the \u2013prepare step may also be added to the backup even if they are not explicitly listed by the parameter if they were created after the backup started.
The next step is to prepare the backup in order to restore it.
"},{"location":"dictionary-cache.html","title":"Dictionary cache","text":"Percona XtraBackup is based on how crash recovery works. Percona XtraBackup copies the InnoDB data files, which results in data that is internally inconsistent; then the prepare phase performs crash recovery on the files to make a consistent, usable database again.
The --prepare phase has the following operations:
- Applies the redo log
- Applies the undo log
As a physical operation, the changes from the redo log modifications are applied to a page. In the redo log operation, there is no concept of row or transaction. The redo apply operation does not make the database consistent with a transaction. An uncommitted transaction can be flushed or written to the redo log by the server. Percona XtraBackup applies changes recorded in the redo log.
Percona XtraBackup physically modifies a specific offset of a page within a tablespace (IBD file) when applying a redo log.
As a logical operation, Percona XtraBackup applies the undo log on a specific row. When the redo log completes, XtraBackup uses the undo log to roll back changes from uncommitted transactions during the backup.
"},{"location":"dictionary-cache.html#undo-log","title":"Undo log","text":"There are two types of undo log records:
- INSERT
- UPDATE
An undo log record contains a table_id. Percona XtraBackup uses this table_id to find the table definition, and then uses that information to parse the records on an index page. The transaction rollback reads the undo log records and applies the changes.
After initializing the data dictionary engine and the data dictionary cache, the storage engine can request the table_id and uses this ID to fetch the table schema. An index search tuple (key) is created from the table schema and key fields from the undo log record. The server finds the record using the search tuple (key) and performs the undo operation.
For example, InnoDB uses the table_id (also known as the se_private_id) for a table definition. Percona XtraBackup does not behave like a server and does not have access to the data dictionary. XtraBackup initializes the InnoDB engine and uses the InnoDB table object (dict_table_t) when needed. XtraBackup relies on Serialized Dictionary Information (SDI) that is stored in the tablespace. SDI is a JSON representation of a table.
Prior to Percona XtraBackup 8.0.33-28, XtraBackup loads the SDI from every .IBD file and all tables into cache as non-evictable. Making the tables non-evictable essentially disables the LRU cache. Every table remains in memory until the operation ends.
This method has the following issues:
- Unnecessary IO operations from reading SDI pages to load the tables that are not required for rollback
- All tables loaded into the cache increases the time required in the
--preparephase - Unnecessary tables can lead to out-of-memory errors
- Huge number of tables and IBD files may cause an exit in the
--preparephase
Percona XtraBackup 8.0.33-28 implements a new design and tables are loaded as evictable. XtraBackup scans the B-tree index of the data dictionary tables mysql.indexes and mysql.index_partitions to establish a relationship between the table_id and the tablespace(space_id). XtraBackup uses this relationship during transaction rollback. XtraBackup does not load user tables unless there is a transaction rollback on them.
A background thread or a Percona XtraBackup main thread handles the cache eviction when the cache size limit is reached.
This new design provides the following benefits during the --prepare phase:
- Uses less memory
- Uses less IO
- Improves the time taken in the
--preparephase - Completes successfully, even if the
--preparephase has a huge number of tables - Completes the Percona XtraDB Cluster SST process faster
Docker allows you to run applications in a lightweight unit called a container.
You can run Percona XtraBackup in a Docker container without installing the product. All required libraries are available in the container. Being a lightweight execution environment, Docker containers enable creating configurations where each program runs in a separate container. You may run Percona Server for MySQL in one container and Percona XtraBackup in another. Docker images offer a range of options.
Create a Docker container based on a Docker image. Docker images for Percona XtraBackup are hosted publicly on Docker Hub.
$ sudo docker create ... percona/percona-xtrabackup --name xtrabackup ...\nThis section demonstrates how to back up data on a Percona Server for MySQL running in another Dockers container.
"},{"location":"docker.html#1-install-docker","title":"1. Install Docker","text":"Your operating system may already provide a package for docker. However, the versions of Docker provided by your operating system are likely to be outdated.
Use the installation instructions for your operating system available from the Docker site to set up the latest version of docker.
"},{"location":"docker.html#2-connect-to-a-percona-server-for-mysql-container","title":"2. Connect to a Percona Server for MySQL container","text":"Percona XtraBackup works in combination with a database server. When running a Docker container for Percona XtraBackup, you can make backups for a database server either installed on the host machine or running in a separate Docker container.
To set up a database server on a host machine or in Docker container, follow the documentation of the supported product that you intend to use with Percona XtraBackup.
See also
Docker volumes as container persistent data storage
More information about containers
$ sudo docker run -d --name percona-server-mysql \\\n-e MYSQL_ROOT_PASSWORD=root percona/percona-server:8.0\nAs soon as Percona Server for MySQL runs, add some data to it. Now, you are ready to make backups with Percona XtraBackup.
Important
When running Percona XtraBackup from a container and connecting to a MySQLserver container, we recommend using the \u2013user root option in the Docker command.
"},{"location":"docker.html#3-create-a-docker-container-from-percona-xtrabackup-image","title":"3. Create a Docker container from Percona XtraBackup image","text":"You can create a Docker container based on Percona XtraBackup image with either docker create or the docker run command. docker create creates a Docker container and makes it available for starting later.
Docker downloads the Percona XtraBackup image from the Docker Hub. If it is not the first time you use the selected image, Docker uses the image available locally.
$ sudo docker create --name percona-xtrabackup --volumes-from percona-server-mysql \\\npercona/percona-xtrabackup \\\nxtrabackup --backup --datadir=/var/lib/mysql/ --target-dir=/backup \\\n--user=root --password=mysql\nWith parameter name you give a meaningful name to your new Docker container so that you could easily locate it among your other containers.
The volumes-from flag refers to Percona Server for MySQL and indicates that you intend to use the same data as the Percona Server for MySQL container.
Run the container with exactly the same parameters that were used when the container was created:
$ sudo docker start -ai percona-xtrabackup\nThis command starts the percona-xtrabackup container, attaches to its input/output streams, and opens an interactive shell.
The docker run is a shortcut command that creates a Docker container and then immediately runs it.
$ sudo docker run --name percona-xtrabackup --volumes-from percona-server-mysql \\\npercona/percona-xtrabackup\nxtrabackup --backup --data-dir=/var/lib/mysql --target-dir=/backup --user=root --password=mysql\nPercona XtraBackup supports encrypting and decrypting local and streaming backups with xbstream option adding another layer of protection. The encryption is implemented using the libgcrypt library from GnuPG.
To make an encrypted backup the following options need to be specified (options --encrypt-key and --encrypt-key-file are mutually exclusive, i.e. just one of them needs to be provided):
-
--encrypt -
--encrypt-key -
--encrypt-key-file
Both the --encrypt-key option and --encrypt-key-file option can be used to specify the encryption key. An encryption key can be generated with a command like openssl rand -base64 24
U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\nThis value then can be used as the encryption key
"},{"location":"encrypt-backups.html#the-encrypt-key-option","title":"The--encrypt-key option","text":"Example of the xtrabackup command using the --encrypt-key should look like this:
$ xtrabackup --backup --encrypt=AES256 --encrypt-key=\"U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\" --target-dir=/data/backup\n--encrypt-key-file option","text":"Use the --encrypt-key-file option as follows:
$ xtrabackup --backup --encrypt=AES256 --encrypt-key-file=/data/backups/keyfile --target-dir=/data/backup\nNote
Depending on the text editor that you use to make the KEYFILE, the editor can automatically insert the CRLF (end of line) character. This will cause the key size to grow and thus making it invalid. The suggested way to create the file is by using the command line: echo -n \u201cU2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\u201d > /data/backups/keyfile.
Two new options are available for encrypted backups that can be used to speed up the encryption process. These are --encrypt-threads and --encrypt-chunk-size. By using the --encrypt-threads option multiple threads can be specified to be used for encryption in parallel. Option --encrypt-chunk-size can be used to specify the size (in bytes) of the working encryption buffer for each encryption thread (default is 64K).
Backups can be decrypted with The xbcrypt binary. The following one-liner can be used to encrypt the whole folder:
$ for i in `find . -iname \"*\\.xbcrypt\"`; do xbcrypt -d --encrypt-key-file=/root/secret_key --encrypt-algo=AES256 < $i > $(dirname $i)/$(basename $i .xbcrypt) && rm $i; done\nPercona XtraBackup --decrypt option has been implemented that can be used to decrypt the backups:
$ xtrabackup --decrypt=AES256 --encrypt-key=\"U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\" --target-dir=/data/backup/\nPercona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should remove the \\*.xbcrypt files.
Note
--parallel can be used with --decrypt option to decrypt multiple files simultaneously.
When the files are decrypted, the backup can be prepared.
"},{"location":"encrypt-backups.html#prepare-encrypted-backups","title":"Prepare encrypted backups","text":"After the backups have been decrypted, they can be prepared in the same way as the standard full backups with the --prepare option:
$ xtrabackup --prepare --target-dir=/data/backup/\nxtrabackup offers the --copy-back option to restore a backup to the server\u2019s datadir:
$ xtrabackup --copy-back --target-dir=/data/backup/\nIt will copy all the data-related files back to the server\u2019s datadir, determined by the server\u2019s my.cnf configuration file. You should check the last line of the output for a success message:
150318 11:08:13 xtrabackup: completed OK!\nInnoDB supports data encryption for InnoDB tables stored in file-per-table tablespaces. This feature provides an at-rest encryption for physical tablespace data files.
For an authenticated user or application to access an encrypted tablespace, InnoDB uses the master encryption key to decrypt the tablespace key. The master encryption key is stored in a keyring. xtrabackup supports two keyring plugins: keyring_file, and keyring_vault. These plugins are installed into the plugin directory.
Percona XtraBackup 8.0.25-17 adds support for the keyring_file component, which is part of the component-based infrastructure MySQL which extends the server capabilities. The component is stored in the plugin directory.
See a comparison of keyring components and keyring plugins for more information.
Percona XtraBackup 8.0.27-19 adds support for the Key Management Interoperability Protocol (KMIP) which enables the communication between the key management system and encrypted database server.
Percona XtraBackup 8.0.28-21 adds support for the Amazon Key Management Service (AWS KMS). AWS KMS is cloud-based encryption and key management service. The keys and functionality can be used for other AWS services or your applications that use AWS. No configuration is required to back up a server with AWS KMS-enabled encryption.
"},{"location":"encrypted-innodb-tablespace-backups.html#use-keyring_file-plugin","title":"Usekeyring_file plugin","text":"In order to back up and prepare a database containing encrypted InnoDB tablespaces, specify the path to a keyring file as the value of the --keyring-file-data option.
$ xtrabackup --backup --target-dir=/data/backup/ --user=root \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nAfter xtrabackup takes the backup, the following message confirms the action:
Confirmation messagextrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\nWarning
xtrabackup does not copy the keyring file into the backup directory. To prepare the backup, you must copy the keyring file manually.
"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring_file-plugin","title":"Prepare the backup with thekeyring_file plugin","text":"To prepare the backup specify the keyring-file-data.
$ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nAfter xtrabackup takes the backup, the following message confirms the action:
Confirmation messageInnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\nThe backup is now prepared and can be restored with the --copy-back option. In case the keyring has been rotated, you must restore the keyring used when the backup was taken and prepared.
keyring_vault plugin","text":"Keyring vault plugin settings are described here.
"},{"location":"encrypted-innodb-tablespace-backups.html#create-a-backup-with-the-keyring_vault-plugin","title":"Create a backup with thekeyring_vault plugin","text":"The following command creates a backup in the /data/backup directory:
$ xtrabackup --backup --target-dir=/data/backup --user=root\nAfter xtrabackup completes the action, the following message confirms the action:
Confirmation messagextrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\nkeyring_vault plugin","text":"To prepare the backup, xtrabackup must access the keyring. xtrabackup does not communicate with the MySQL server or read the default my.cnf configuration file. Specify the keyring settings in the command line:
$ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-vault-config=/etc/vault.cnf\nReview using the keyring vault plugin for a description of keyring vault plugin settings.
After xtrabackup completes the action, the following message confirms the action:
Confirmation messageInnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\nThe backup is now prepared and can be restored with the --copy-back option:
$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql\nkeyring_file component","text":"A component is not loaded with the --early_plugin_load option. The server uses a manifest to load the component and the component has its own configuration file. See the MySQL documentation on the component installation for more information.
An example of a manifest and a configuration file follows:
An example of ./bin/mysqld.my:
{\n\"components\": \"file://component_keyring_file\"\n}\nAn example of /lib/plugin/component_keyring_file.cnf:
{\n\"path\": \"/var/lib/mysql-keyring/keyring_file\", \"read_only\": false\n}\nFor more information, see Keyring Component Installation and Using the keyring_file File-Based Keyring Plugin.
With the appropriate privilege, you can SELECT on the performance_schema.keyring_component_status table to view the attributes and status of the installed keyring component when in use.
The component has no special requirements for backing up a database that contains encrypted InnoDB tablespaces.
$ xtrabackup --backup --target-dir=/data/backup --user=root\nAfter xtrabackup completes the action, the following message confirms the action:
Confirmation messagextrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\nWarning
xtrabackup does not copy the keyring file into the backup directory. To prepare the backup, you must copy the keyring file manually.
"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring_file-component","title":"Prepare the backup with thekeyring_file component","text":"xtrabackup reads the keyring_file component configuration from xtrabackup_component_keyring_file.cnf. You must specify the keyring_file data path if the keyring-file-data is not located in the attribute PATH from the xtrabackup_component_keyring_file.cnf.
The following is an example of adding the location for the keyring-file-data:
$ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nxtrabackup attempts to read xtrabackup_component_keyring_file.cnf. You can assign another keyring file component configuration by passing the --component-keyring-file-config option.
After xtrabackup completes preparing the backup, the following message confirms the action:
Confirmation messageInnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\nThe backup is prepared. To restore the backup use the --copy-back option. If the keyring has been rotated, you must restore the specific keyring used to take and prepare the backup.
keyring_file","text":"The process of taking incremental backups with InnoDB tablespace encryption is similar to taking the Incremental Backups with unencrypted tablespace. The keyring-file component should not used in production or for regulatory compliance.
To make an incremental backup, begin with a full backup. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. First you need to create a full backup with the following command:
$ xtrabackup --backup --target-dir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nIn order to prepare the backup, you must make a copy of the keyring file yourself. xtrabackup does not copy the keyring file into the backup directory. Restoring the backup after the keyring has been changed causes errors like ERROR 3185 (HY000): Can't find master key from keyring, please check keyring plugin is loaded. when the restore process tries accessing an encrypted table.
If you look at the xtrabackup_checkpoints file, you should see the output similar to the following:
backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 7666625\nlast_lsn = 7666634\ncompact = 0\nrecover_binlog_info = 1\nNow that you have a full backup, you can make an incremental backup based on it. Use a command such as the following:
$ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nTo prepare the backup, you must copy the keyring file manually. xtrabackup does not copy the keyring file into the backup directory.
If the keyring has not been rotated you can use the same as the one you\u2019ve backed-up with the base backup. If the keyring has been rotated, or you have upgraded the plugin to a component, you must back up the keyring file. Otherwise, you cannot prepare the backup.
The /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 7666625. If you examine the xtrabackup_checkpoints file in this directory, you should see the output similar to the following:
backup_type = incremental\nfrom_lsn = 7666625\nto_lsn = 8873920\nlast_lsn = 8873929\ncompact = 0\nrecover_binlog_info = 1\nUse this directory as the base for yet another incremental backup:
$ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nThe --prepare step for incremental backups is not the same as for normal backups. In normal backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing a backup, because transactions that were uncommitted at the time of your backup may be in progress, and it\u2019s likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.
If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups are useless. After transactions have been rolled back, further incremental backups cannot be applied.
Beginning with the full backup you created, you can prepare it and then apply the incremental differences to it. Recall that you have the following backups:
/data/backups/base\n/data/backups/inc1\n/data/backups/inc2\nTo prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nInnoDB: Shutdown completed; log sequence number 7666643\nInnoDB: Number of pools: 1\n160401 12:31:11 completed OK!\nTo apply the first incremental backup to the full backup, you should use the following command:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nThe backup should be prepared with the keyring file and type that was used when backup was being taken. This means that if the keyring has been rotated, or you have upgraded from a plugin to a component between the base and incremental backup that you must use the keyring that was in use when the first incremental backup has been taken.
Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:
$ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n--apply-log-only when merging all incremental backups except the last one. That\u2019s why the previous line does not contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase. The backup is now prepared and can be restored with --copy-back option. In case the keyring has been rotated you\u2019ll need to restore the keyring which was used to take and prepare the backup.
Percona XtraBackup 8.0.27-19 adds support for the Key Management Interoperability Protocol (KMIP) which enables the communication between the key management system and encrypted database server.
Percona XtraBackup has no special requirements for backing up a database that contains encrypted InnoDB tablespaces.
Percona XtraBackup performs the following actions:
-
Connects to the MySQL server
-
Pulls the configuration
-
Connects to the KMIP server
-
Fetches the necessary keys from the KMIP server
-
Stores the KMIP server configuration settings in the
xtrabackup_component_keyring_kmip.cnffile in the backup directory
When preparing the backup, Percona XtraBackup connects to the KMIP server with the settings from the xtrabackup_component_keyring_kmip.cnf file.
While the described restore method works, this method requires access to the same keyring that the server is using. It may not be possible if the backup is prepared on a different server or at a much later time, when keys in the keyring are purged, or, in the case of a malfunction, when the keyring vault server is not available at all.
The --transition-key=<passphrase> option should be used to make it possible for xtrabackup to process the backup without access to the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and will use it to encrypt tablespace keys of tablespaces that are being backed up.
The following example illustrates how the backup can be created in this case:
$ xtrabackup --backup --user=root -p --target-dir=/data/backup \\\n--transition-key=MySecretKey\nIf --transition-key is specified without a value, xtrabackup will ask for it.
xtrabackup scrapes --transition-key so that its value is not visible in the ps command output.
The same passphrase should be specified for the prepare command:
$ xtrabackup --prepare --target-dir=/data/backup \\\n--transition-key=MySecretKey\nThere are no --keyring-vault...,``\u2013keyring-file\u2026``, or --component-keyring-file-config options here, because xtrabackup does not talk to the keyring in this case.
When restoring a backup you will need to generate a new master key. Here is the example for keyring_file plugin or component:
$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nIn case of keyring_vault, it will look like this:
$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--keyring-vault-config=/etc/vault.cnf\nxtrabackup will generate a new master key, store it in the target keyring vault server and re-encrypt the tablespace keys using this key.
"},{"location":"encrypted-innodb-tablespace-backups.html#make-a-backup-with-a-stored-transition-key","title":"Make a backup with a stored transition key","text":"Finally, there is an option to store a transition key in the keyring. In this case, xtrabackup will need to access the same keyring file or vault server during prepare and copy-back but does not depend on whether the server keys have been purged.
In this scenario, the three stages of the backup process look as follows.
- Backup
$ xtrabackup --backup --user=root -p --target-dir=/data/backup \\\n--generate-transition-key\n-
Prepare
keyring_filevariant:
$ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nkeyring_vaultvariant:
$ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-vault-config=/etc/vault.cnf\n -
Copy-back
-
keyring_filevariant:$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --keyring-file-data=/var/lib/mysql-keyring/keyring\n -
keyring_vaultvariant:$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --keyring-vault-config=/etc/vault.cnf\n
MySQL 8.0.29 extended the support for ALTER TABLE \u2026 ALGORITHM=INSTANT to allow users to add a column in any table position or drop any column. As part of this update, the redo log format has changed for all server DML operations. This updated redo log format has a design flaw that can cause data corruption for tables with INSTANT ADD/DROP COLUMNS.
The corruption happens when the crash recovery occurs. InnoDB applies redo logs at startup. Xtrabackup copies the redo log during the backup and applies the log as part of the --prepare step to bring the backup to a consistent state.
Percona fixed the corruption issue and several other issues with the INSTANT ADD/DROP column feature in the upcoming Percona Server for MySQL 8.0.29.
For more details, see the following:
-
PS-8291,
-
PS-8292
-
PS-8303
Percona XtraBackup 8.0.32-25 allows instant columns in the backup for MySQL 8.0.32. This limitation remains for backing up MySQL 8.0.31 and lower versions. This limitation does not exist for backing up Percona Server for any version.
Percona XtraBackup 8.0.29 is able to take backups of Percona Server for MySQL 8.0.29 with tables that have INSTANT ADD/DROP COLUMNS. However, the issues in MySQL 8.0.29 make this version unsafe to take backups.
It is impossible for XtraBackup to deal with the corrupted redo log generated by MySQL 8.0.29 and, for this reason, XtraBackup 8.0.29 version will not take backups if it detects tables with INSTANT ADD/DROP COLUMNS and generates an error listing the affected tables and providing instructions to convert them to regular tables.
Please avoid ALTER ADD/DROP COLUMN without an explicit ALGORITHM=INPLACE. The default ALGORITHM is INSTANT, so ALTER TABLE without the ALGORITHM keyword uses the newly introduced INSTANT algorithm.
If you already have such tables (information below on how to find such tables), users are advised to do OPTIMIZE TABLE against these tables before taking backups.
To find tables with INSTANT ADD/DROP COLUMNS run the following command:
mysql> SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLES WHERE TOTAL_ROW_VERSIONS > 0;\n+---------+\n| NAME |\n+---------+\n| test/t1 |\n| test/t2 |\n| test/t3 |\n+---------+\n3 rows in set (0.02 sec)\nIf this query shows an empty result set, you have no issues. Percona Xtrabackup takes the backup of your MySQL 8.0.29 server. If the results are a list of tables, please run OPTIMIZE TABLE on the list before taking the backup.
"},{"location":"error-message-instant.html#percona-xtrabackup-error-message","title":"Percona XtraBackup Error Message","text":"If Percona XtraBackup detects that MySQL 8.0.29 server has tables with instant add/drop columns, it aborts with the following error message
2022-07-01T15:18:35.127689+05:30 0 [ERROR] [MY-011825] [Xtrabackup] Found tables with row versions due to INSTANT ADD/DROP columns\n2022-07-01T15:18:35.127714+05:30 0 [ERROR] [MY-011825] [Xtrabackup] This feature is not stable and will cause backup corruption.\n2022-07-01T15:18:35.127723+05:30 0 [ERROR] [MY-011825] [Xtrabackup] Tables found:\n2022-07-01T15:18:35.127730+05:30 0 [ERROR] [MY-011825] [Xtrabackup] test/t1\n2022-07-01T15:18:35.127737+05:30 0 [ERROR] [MY-011825] [Xtrabackup] test/t2\n2022-07-01T15:18:35.127744+05:30 0 [ERROR] [MY-011825] [Xtrabackup] test/t3\n2022-07-01T15:18:35.127752+05:30 0 [ERROR] [MY-011825] [Xtrabackup] Please run OPTIMIZE TABLE or ALTER TABLE ALGORITHM=COPY on all listed tables to fix this issue.\nThe option, ALGORITHM=INSTANT is the default value in MySQL 8.0.29. If you do not specify an algorithm, all ALTER TABLE ADD/DROP COLUMN statements use this algorithm.
The INSTANT algorithm is unstable at this point.
Percona XtraBackup refuses to take backups from MySQL 8.0.29 tables that have used this algorithm. Percona Server users do not suffer the same limitations as the corruption issues are fixed in the upcoming release of Percona Server for MySQL 8.0.29
"},{"location":"faq.html","title":"Frequently asked questions","text":""},{"location":"faq.html#does-percona-xtrabackup-80-support-making-backups-of-databases-in-versions-prior-to-80","title":"Does Percona XtraBackup 8.0 support making backups of databases in versions prior to 8.0?","text":"Percona XtraBackup 8.0 does not support making backups of databases created in versions prior to 8.0 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster. As the changes that MySQL 8.0 introduced in data dictionaries, redo log and undo log are incompatible with previous versions, it is currently impossible for Percona XtraBackup 8.0 to also support versions prior to 8.0.
Due to changes in MySQL 8.0.20 released by Oracle at the end of April 2020, Percona XtraBackup 8.0, up to version 8.0.11, is not compatible with MySQL version 8.0.20 or higher, or Percona products that are based on it: Percona Server for MySQL and Percona XtraDB Cluster.
For more information, see Percona XtraBackup 8.x and MySQL 8.0.20
"},{"location":"faq.html#why-will-innobackupex-not-run-in-percona-xtrabackup-80","title":"Why willinnobackupex not run in Percona XtraBackup 8.0?","text":"innobackupex has been removed from Percona XtraBackup 8.0 in favor of xtrabackup.
"},{"location":"faq.html#are-you-aware-of-any-web-based-backup-management-tools-commercial-or-not-built-around-percona-xtrabackup","title":"Are you aware of any web-based backup management tools (commercial or not) built around Percona XtraBackup?","text":"Zmanda Recovery Manager is a commercial tool that uses Percona XtraBackup for Non-Blocking Backups:
\u201cZRM provides support for non-blocking backups of MySQL using Percona XtraBackup. ZRM with *Percona XtraBackup provides resource utilization management by providing throttling based on the number of IO operations per second. Percona XtraBackup based backups also allow for table level recovery even though the backup was done at the database level (needs the recovery database server to be Percona Server for MySQL with XtraDB).\u201d*
"},{"location":"faq.html#xtrabackup-binary-fails-with-a-floating-point-exception","title":"xtrabackup binary fails with a floating point exception","text":"In most of the cases this is due to not having installed the required libraries (and version) by xtrabackup. Installing the GCC suite with the supporting libraries and recompiling xtrabackup solves the issue. See Compiling and Installing from Source Code for instructions on the procedure.
"},{"location":"faq.html#how-xtrabackup-handles-the-ibdataib_log-files-on-restore-if-they-arent-in-mysql-datadir","title":"How xtrabackup handles the ibdata/ib_log files on restore if they aren\u2019t in mysql datadir?","text":"In case the ibdata and ib_log files are located in different directories outside the datadir, you will have to put them in their proper place after the logs have been applied.
This usually happens when database being backed up contains large amount of files and Percona XtraBackup can\u2019t open all of them to create a successful backup. In order to avoid this error the operating system should be configured appropriately so that Percona XtraBackup can open all its files. On Linux, this can be done with the ulimit command for specific backup session or by editing the /etc/security/limits.conf to change it globally (NOTE: the maximum possible value that can be set up is 1048576 which is a hard-coded constant in the Linux kernel).
To prevent creating corrupted backups when running DDL operations, Percona XtraBackup aborts if it detects that redo logging is disabled. In this case, the following error is printed:
[FATAL] InnoDB: An optimized (without redo logging) DDL operation has been performed. All modified pages may not have been flushed to the disk yet.\nPercona XtraBackup will not be able to take a consistent backup. Retry the backup operation.\nNote
Redo logging is disabled during a sorted index build. To avoid this error, Percona XtraBackup can use metadata locks on tables while they are copied:
-
To block all DDL operations, use the
--lock-ddloption that issuesLOCK TABLES FOR BACKUP. -
If
LOCK TABLES FOR BACKUPis not supported, you can block DDL for each table before XtraBackup starts to copy it and until the backup is completed using the--lock-ddl-per-tableoption.
Note
As of Percona XtraBackup 8.0.15, the \u2013lock-ddl-per-table option is deprecated. Use the --lock-ddl option.
The FLUSH TABLES WITH READ LOCK option does the following with a global read lock:
-
Closes all open tables
-
Locks all tables for all databases
Release the lock with UNLOCK TABLES.
Note
FLUSH TABLES WITH READ LOCK does not prevent inserting rows into the log tables.
To ensure consistent backups, use the FLUSH TABLES WITH READ LOCK option before taking a non-InnoDB file backup. The option does not affect long-running queries.
Long-running queries with FLUSH TABLES WITH READ LOCK enabled can leave the server in a read-only mode until the queries finish. Killing the FLUSH TABLES WITH READ LOCK does not help if the database is in either the Waiting for table flush or Waiting for master to send event state. To return to normal operation, you must kill any long-running queries.
Note
All described in this section has no effect when backup locks are used. Percona XtraBackup will use Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. This feature is available in Percona Server for MySQL 5.6+. Percona XtraBackup uses this automatically to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
In order to prevent this from happening two things have been implemented:
-
xtrabackup waits for a good moment to issue the global lock
-
xtrabackup kills all queries or only the SELECT queries which prevent the global lock from being acquired
You should issue a global lock when no long queries are running. Waiting to issue the global lock for extended period of time is not a good method. The wait can extend the time needed for backup to take place. The \u2013ftwrl-wait-timeout option can limit the waiting time. If it cannot issue the lock during this time, xtrabackup stops the option, exits with an error message, and backup is not be taken.
The default value for this option is zero (0) value which turns off the option.
Another possibility is to specify the type of query to wait on. In this case --ftwrl-wait-query-type. Possible values are all and update. When all is used xtrabackup will wait for all long running queries (execution time longer than allowed by --ftwrl-wait-threshold) to finish before running the FLUSH TABLES WITH READ LOCK. When update is used xtrabackup will wait on UPDATE/ALTER/REPLACE/INSERT queries to finish.
The time needed for a specific query to complete is hard to predict. We assume that the long-running queries will not finish in a timely manner. Other queries which run for a short time finish quickly. xtrabackup uses the value of \u2013ftwrl-wait-threshold option to specify the long-running queries and will block a global lock. In order to use this option xtrabackup user should have PROCESS and SUPER privileges.
The second option is to kill all the queries which prevent from acquiring the global lock. In this case, all queries which run longer than FLUSH TABLES WITH READ LOCK are potential blockers. Although all queries can be killed, additional time can be specified for the short running queries to finish using the --kill-long-queries-timeout option. This option specifies the time for queries to complete, after the value is reached, all the running queries will be killed. The default value is zero, which turns this feature off.
The --kill-long-query-type option can be used to specify all or only SELECT queries that are preventing global lock from being acquired. In order to use this option xtrabackup user should have PROCESS and SUPER privileges.
-
--ftwrl-wait-timeout(seconds) - how long to wait for a good moment. Default is 0, not to wait. -
--ftwrl-wait-query-type- which long queries should be finished beforeFLUSH TABLES WITH READ LOCKis run. Default is all. -
--ftwrl-wait-threshold(seconds) - how long query should be running before we consider it long running and potential blocker of global lock. -
--kill-long-queries-timeout(seconds) - how many time we give for queries to complete afterFLUSH TABLES WITH READ LOCKis issued before start to kill. Default if0, not to kill. -
--kill-long-query-type- which queries should be killed oncekill-long-queries-timeouthas expired.
Running the xtrabackup with the following options will cause xtrabackup to spend no longer than 3 minutes waiting for all queries older than 40 seconds to complete.
$ xtrabackup --backup --ftwrl-wait-threshold=40 \\\n--ftwrl-wait-query-type=all --ftwrl-wait-timeout=180 \\\n--kill-long-queries-timeout=20 --kill-long-query-type=all \\\n--target-dir=/data/backups/\nAfter FLUSH TABLES WITH READ LOCK is issued, xtrabackup will wait for 20 seconds for lock to be acquired. If lock is still not acquired after 20 seconds, it will kill all queries which are running longer that the FLUSH TABLES WITH READ LOCK.
Each table with the CSV Storage Engine has .CSM file which contains the metadata of it.
Each table with the CSV Storage engine has .CSV file which contains the data of it (which is a standard Comma Separated Value file).
Files with the .exp extension are created by Percona XtraBackup per each InnoDB tablespace when the --export option is used on prepare. See restore individual tables.
For each table, the server will create a file with the .frm extension containing the table definition (for all storage engines).
A finalized version of the product which is made available to the general public. It is the final stage in the software release cycle.
"},{"location":"glossary.html#ibd","title":".ibd","text":"On a multiple tablespace setup ([innodb_file_per_table] enabled), MySQL will store each newly created table on a file with a .ibd extension.
Each table using the MERGE storage engine, besides of a .frm file, will have .MRG file containing the names of the MyISAM tables associated with it.
Each MyISAM table has .MYD (MYData) file which contains the data on it.
Each MyISAM table has .MYI (MYIndex) file which contains the table\u2019s indexes.
MySQL stores options of a database (like charset) in a file with a .opt extension in the database directory.
Each partitioned table has .par file which contains metadata about the partitions.
The file contains the triggers associated with a table, for example, \\mytable.TRG. With the .TRN file, they represent all the trigger definitions.
The file contains the names of triggers that are associated with a table, for example, \\mytable.TRN. With the .TRG file, they represent all the trigger definitions.
The process of copying data or tables to be stored in a different location.
"},{"location":"glossary.html#compression","title":"compression","text":"The method that produces backups in a reduced size.
"},{"location":"glossary.html#configuration-file","title":"configuration file","text":"The file that contains the server startup options.
"},{"location":"glossary.html#crash","title":"crash","text":"An unexpected shutdown which does not allow the normal server shutdown cleanup activities.
"},{"location":"glossary.html#crash-recovery","title":"crash recovery","text":"The actions that occur when MySQL is restarted after a crash.
"},{"location":"glossary.html#data-dictionary","title":"data dictionary","text":"The metadata for the tables, indexes, and table columns stored in the InnoDB system tablespace.
"},{"location":"glossary.html#datadir","title":"datadir","text":"The directory in which the database server stores its data files. Most Linux distribution use /var/lib/mysql by default.
A backup that contains the complete source data from an instance.
"},{"location":"glossary.html#ibdata","title":"ibdata","text":"The default prefix for tablespace files. For example, ibdata1 is a 10MB auto-extensible file that MySQL creates for a shared tablespace by default.
A backup stores data from a specific point in time.
"},{"location":"glossary.html#innodb","title":"InnoDB","text":"Storage engine which provides ACID-compliant transactions and foreign key support, among others improvements over MyISAM. It is the default engine for MySQL as of the 8.0 series.
"},{"location":"glossary.html#innodb_buffer_pool_size","title":"innodb_buffer_pool_size","text":"The size in bytes of the memory buffer to cache data and indexes of InnoDB\u2019s tables. This aims to reduce disk access to provide better performance.
[mysqld] innodb_buffer_pool_size=8MB
"},{"location":"glossary.html#innodb_data_home_dir","title":"innodb_data_home_dir","text":"The directory (relative to datadir) where the database server stores the files in a shared tablespace setup. This option does not affect the location of innodb\\_file\\_per\\_table. For example:
[mysqld] innodb_data_home_dir = ./
"},{"location":"glossary.html#innodb_data_file_path","title":"innodb_data_file_path","text":"Specifies the names, sizes and location of shared tablespace files:
[mysqld] innodb_data_file_path=ibdata1:50M;ibdata2:50M:autoextend
"},{"location":"glossary.html#innodb_file_per_table","title":"innodb_file_per_table","text":"By default, InnoDB creates tables and indexes in a file-per-tablespace. If the innodb_file_per_table variable is disabled, you can enable the variable in your configuration file:
[mysqld] innodb_file_per_table or start the server with --innodb_file_per_table.
Specifies the location of the InnoDB log files:
[mysqld] innodb_log_group_home=/var/lib/mysql
"},{"location":"glossary.html#logical-backup","title":"logical backup","text":"A backup which contains a set of SQL statements. The statements can be used to recreate the databases.
"},{"location":"glossary.html#lsn","title":"LSN","text":"Each InnoDB page contains a log sequence number(LSN). The LSN is the system version number for the database. Each page\u2019s LSN shows how recently it was changed.
"},{"location":"glossary.html#mycnf","title":"my.cnf","text":"The database server\u2019s main configuration file. Most Linux distributions place it as /etc/mysql/my.cnf or /etc/my.cnf, but the location and name depends on the particular installation. Note that this method is not the only way of configuring the server, some systems rely on the command options.
The MySQL default storage engine until version 5.5. It doesn\u2019t fully support transactions but in some scenarios may be faster than InnoDB. Each table is stored on disk in 3 files: .frm, .MYD, .MYI.
A backup that copies the data files.
"},{"location":"glossary.html#point-in-time-recovery","title":"point in time recovery","text":"This method restores the data into the state it was at any selected point of time.
"},{"location":"glossary.html#prepared-backup","title":"prepared backup","text":"A consistent set of backup data that is ready to be restored.
"},{"location":"glossary.html#restore","title":"restore","text":"Copies the database backups taken using the backup command to the original location or a different location. A restore returns data that has been either lost, corrupted, or stolen to the original condition at a specific point in time.
"},{"location":"glossary.html#tech-preview","title":"Tech preview","text":"A tech preview item can be a feature, a variable, or a value within a variable. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment and also use an alternative backup method for redundancy. A tech preview item is included in a release for users to provide feedback. The item is either updated and released as general availability(GA) or removed if not useful. The functionality can change from tech preview to GA.
"},{"location":"glossary.html#xbcrypt","title":"xbcrypt","text":"To support the encryption and the decryption of the backups, a new tool xbcrypt was introduced to Percona XtraBackup. This utility has been modeled after the xbstream binary to perform encryption and decryption outside Percona XtraBackup.
"},{"location":"glossary.html#xbstream","title":"xbstream","text":"To support simultaneous compression and streaming, Percona XtraBackup uses the xbstream format. For more information see xbstream
"},{"location":"glossary.html#xtradb","title":"XtraDB","text":"Percona XtraDB is an enhanced version of the InnoDB storage engine, designed to better scale on modern hardware. Percona XtraDB includes features which are useful in a high performance environment. It is fully backward-compatible, and is a drop-in replacement for the standard InnoDB storage engine. For more information, see The Percona XtraDB Storage Engine.
"},{"location":"glossary.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.
ZSTD compression algorithm was implemented in Percona XtraBackup 8.0.30-23.
Percona XtraBackup is based on InnoDB\u2019s crash-recovery functionality. It copies your InnoDB data files, which results in data that is internally inconsistent; but then it performs crash recovery on the files to make them a consistent, usable database again.
This works because InnoDB maintains a redo log, also called the transaction log. This contains a record of every change to InnoDB data. When InnoDB starts, it inspects the data files and the transaction log, and performs two steps. It applies committed transaction log entries to the data files, and it performs an undo operation on any transactions that modified data but did not commit.
Percona XtraBackup 8.0.30-23 adds --register-redo-log-consumer parameter. The --register-redo-log-consumer parameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.
Percona XtraBackup works by remembering the LSN when it starts, and then copying away the data files. It takes some time to do this, so if the files are changing, then they reflect the state of the database at different points in time. At the same time, Percona XtraBackup runs a background process that watches the transaction log files, and copies changes from it. Percona XtraBackup needs to do this continually because the transaction logs are written in a round-robin fashion, and can be reused after a while. Percona XtraBackup needs the transaction log records for every change to the data files since it began execution.
Percona XtraBackup uses Backup locks
where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. This feature is available in Percona Server for MySQL 5.6+. MySQL 8.0 allows acquiring an instance level backup lock via the LOCK INSTANCE FOR BACKUP statement.
Locking is only done for MyISAM and other non-InnoDB tables after Percona XtraBackup finishes backing up all InnoDB/XtraDB data and logs. Percona XtraBackup uses this automatically to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.
Important
The BACKUP_ADMIN privilege is required to query the performance_schema_log_status for either LOCK INSTANCE FOR BACKUP or LOCK TABLES FOR BACKUP.
xtrabackup tries to avoid backup locks and FLUSH TABLES WITH READ LOCK when the instance contains only InnoDB tables. In this case, xtrabackup obtains binary log coordinates from performance_schema.log_status. FLUSH TABLES WITH READ LOCK is still required in MySQL 8.0 when xtrabackup is started with the --slave-info. The log_status table in Percona Server for MySQL 8.0 is extended to include the relay log coordinates, so no locks are needed even with the --slave-info option.
See also
MySQL Documentation: LOCK INSTANCE FOR BACKUP
When backup locks are supported by the server, xtrabackup first copies InnoDB data, runs the LOCK TABLES FOR BACKUP and then copies the MyISAM tables. Once this is done, the backup of the files will begin. It will backup .frm, .MRG, .MYD, .MYI, .CSM, .CSV, .sdi and .par files.
After that xtrabackup will use LOCK BINLOG FOR BACKUP to block all operations that might change either binary log position or Exec_Master_Log_Pos or Exec_Gtid_Set (i.e. source binary log coordinates corresponding to the current SQL thread state on a replication replica) as reported by SHOW MASTER/SLAVE STATUS. xtrabackup will then finish copying the REDO log files and fetch the binary log coordinates. After this is completed xtrabackup will unlock the binary log and tables.
Finally, the binary log position will be printed to STDERR and xtrabackup will exit returning 0 if all went OK.
Note that the STDERR of xtrabackup is not written in any file. You will have to redirect it to a file, e.g., xtrabackup OPTIONS 2> backupout.log.
It will also create the following files in the directory of the backup.
During the prepare phase, Percona XtraBackup performs crash recovery against the copied data files, using the copied transaction log file. After this is done, the database is ready to restore and use.
The backed-up MyISAM and InnoDB tables will be eventually consistent with each other, because after the prepare (recovery) process, InnoDB\u2019s data is rolled forward to the point at which the backup completed, not rolled back to the point at which it started. This point in time matches where the FLUSH TABLES WITH READ LOCK was taken, so the MyISAM data and the prepared InnoDB data are in sync.
The xtrabackup offers many features not mentioned in the preceding explanation. The functionality of each tool is explained in more detail further in this manual. In brief, though, the tools enable you to do operations such as streaming and incremental backups with various combinations of copying the data files, copying the log files, and applying the logs to the data.
"},{"location":"how-xtrabackup-works.html#restoring-a-backup","title":"Restoring a backup","text":"To restore a backup with xtrabackup you can use the --copy-back or --move-back options.
xtrabackup will read from the my.cnf the variables datadir, innodb_data_home_dir, innodb_data_file_path, innodb_log_group_home_dir and check that the directories exist.
It will copy the MyISAM tables, indexes, etc. (.MRG, .MYD, .MYI, .CSM, .CSV, .sdi, and par files) first, InnoDB tables and indexes next and the log files at last. It will preserve file\u2019s attributes when copying them, you may have to change the files\u2019 ownership to mysql before starting the database server, as they will be owned by the user who created the backup.
Alternatively, the --move-back option may be used to restore a backup. This option is similar to --copy-back with the only difference that instead of copying files it moves them to their target locations. As this option removes backup files, it must be used with caution. It is useful in cases when there is not enough free disk space to hold both data files and their backup copies.
We recommend that you install Percona XtraBackup 8.0 from the official Percona software repositories using the appropriate package manager for your system:
-
Use an APT repo to install Percona XtraBackup
-
Use a YUM repo to install Percona XtraBackup
Percona also provides the following methods:
-
Use DEB downloaded packages to install Percona XtraBackup
-
Use RPM downloaded packages to install Percona XtraBackup
-
Install Percona XtraBackup from a Binary Tarball with all the required files and binaries for a manual installation
-
Compile and install Percona XtraBackup from source code
-
Run Percona XtraBackup in a Docker container
Before installing Percona XtraBackup with any of these methods, we recommend that you review the release notes and Server version and backup version comparison.
"},{"location":"lock-options.html","title":"lock-ddl-per-table option improvements","text":"To block DDL statements on an instance, Percona Server implemented LOCK TABLES FOR BACKUP. Percona XtraBackup uses this lock for the duration of the backup. This lock does not affect DML statements.
Percona XtraBackup has also implemented --lock-ddl-per-table, which blocks DDL statements by using metadata locks (MDL).
The following procedures describe a simplified backup operation when using --lock-ddl-per-table:
-
Parse and copy all redo logs after the checkpoint mark
-
Fork a dedicated thread to continue following new redo log entries
-
List the tablespaces required to copy
-
Iterate through the list. The following steps occur with each listed tablespace:
-
Query INFORMATION_SCHEMA.INNODB_TABLES to find which tables belong to the tablespace ID and take an MDL on the underlying table or tables in case there is a shared tablespace.
-
Copy the tablespace
.ibdfiles.
The backup process may encounter a redo log event, generated by the bulk load operations, which notifies backup tools that data file changes have been omitted from the redo log. This event is an MLOG_INDEX_LOAD. If this event is found by the redo follow thread, the backup continues and assumes the backup is safe because the MDL protects tablespaces already copied and the MLOG_INDEX_LOAD event is for a tablespace that is not copied.
These assumptions may not be correct and may lead to inconsistent backups.
"},{"location":"lock-options.html#-lock-ddl-per-table-redesign","title":"--lock-ddl-per-table redesign","text":"Implemented in Percona XtraBackup version 8.0.22-15.0, the --lock-ddl-per-table has been redesigned to minimize inconsistent backups.
The following procedure reorders the steps:
-
The MDL lock acquired at the beginning of the backup
-
Scan the redo logs. An
MLOG_INDEX_LOADevent may be recorded if aCREATE INDEXstatement has occurred before the backup starts. At this time, the backup process is safe and can parse and accept the event. -
After the first scan has completed, the follow redo log thread is initiated. This thread stops the backup process if an
MLOG_INDEX_LOADevent is found. -
Gather the tablespace list to copy
-
Copy the
.ibdfiles.
The following improvements have been added:
-
If the
.ibdfile belongs to a temporary table, theSELECTquery is skipped. -
For a FullText Index, an MDL is acquired on the base table.
-
A
SELECTquery that acquires an MDL does not retrieve any data.
Warning
The lock-ddl-per-table variable is deprecated in Percona Server for MySQL 8.0. Use --lock-ddl instead of this variable.
Percona XtraBackup is an open-source command-line utility. Command-line tools have limited interaction with the background operations and the logs provide the progress of an operation or more information about errors.
In earlier versions, the error logs did not have a standard structure. Notice in the following examples the variance in the log statements:
- The backup log statement header has the name of the module,
xtrabackup, which generated the statement but no timestamp:
$ xtrabackup: recognized client arguments: --parallel=4 --target-dir=/data/backups/ --backup=1\n./bin/xtrabackup version 8.0.27-19 based on MySQL server 8.0.27 Linux (x86_64) (revision id: b0f75188ca3)\n- The copy-back log statement has a timestamp but no module name. The timestamp is a mix of UTC and the local timezone.
220322 19:05:13 [01] Copying undo_001 to /data/backups/undo_001\n- The following prepare log statements do not have header information, which makes diagnosing an issue more difficult.
Completed space ID check of 1008 files.\nInitializing buffer pool, total size = 128.000000M, instances = 1, chunk size =128.000000M\nCompleted initialization of buffer pool\nIf the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().\nStarting in Percona XtraBackup 8.0.28-20, changes have been made to improve the log statements. The improved log structure is displayed in the backup, prepare, move-back/copy-back error logs.
Each log statement has the following attributes:
-
Timestamp - a timestamp for when the event occurred in a UTC format.
-
Severity - the severity level of a statement indicates the importance of an event.
-
ID - this identifier is currently not used but may be used in future versions.
-
Context - the name of the module that issued the log statement, such as XtraBackup, InnoDB, or Server.
-
Message - a description of the event generated by the module.
An example of a prepare log that is generated with the improved structure. The uniformity of the headers makes it easier to follow an operation\u2019s progress or review the log to diagnose issues.
2022-03-22T19:15:36.142247+05:30 0 [Note] [MY-011825] [Xtrabackup] This target seems to be not prepared yet.\n2022-03-22T19:15:36.142792+05:30 0 [Note] [MY-013251] [InnoDB] Number of pools: 1\n2022-03-22T19:15:36.149212+05:30 0 [Note] [MY-011825] [Xtrabackup] xtrabackup_logfile detected: size=8388608, start_lsn=(33311656)\n2022-03-22T19:15:36.149998+05:30 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration for recovery:\n2022-03-22T19:15:36.150023+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .\n2022-03-22T19:15:36.150036+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend\n2022-03-22T19:15:36.150078+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = .\n2022-03-22T19:15:36.150095+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 1\n2022-03-22T19:15:36.150111+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 8388608\n2022-03-22T19:15:36.151667+05:30 0 [Note] [MY-011825] [Xtrabackup] inititialize_service_handles suceeded\n2022-03-22T19:15:36.151903+05:30 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration for recovery:\n2022-03-22T19:15:36.151926+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .\n2022-03-22T19:15:36.151954+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend\n2022-03-22T19:15:36.151976+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = .\n2022-03-22T19:15:36.151991+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 1\n2022-03-22T19:15:36.152004+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 8388608\n2022-03-22T19:15:36.152021+05:30 0 [Note] [MY-011825] [Xtrabackup] Starting InnoDB instance for recovery.\n2022-03-22T19:15:36.152035+05:30 0 [Note] [MY-011825] [Xtrabackup] Using 104857600 bytes for buffer pool (set by --use-memory parameter)\nPercona XtraBackup includes a saved buffer pool dump into a backup to enable reducing the warm up time. It restores the buffer pool state from ib_buffer_pool file after restart. Percona XtraBackup discovers ib_buffer_pool and backs it up automatically.
If the buffer restore option is enabled in my.cnf buffer pool will be in the warm state after backup is restored.
There are options specific to back up from a replication replica.
"},{"location":"make-backup-in-replication-env.html#the-slave-info-option","title":"The--slave-info option","text":"This option is useful when backing up a replication replica server. It prints the binary log position and name of the source server. It also writes this information to the xtrabackup_slave_info file as a CHANGE MASTER statement.
This option is useful for setting up a new replica for this source. You can start a replica server with this backup and issue the statement saved in the xtrabackup_slave_info file. More details of this procedure can be found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.
--safe-slave-backup option","text":"In order to assure a consistent replication state, this option stops the replication SQL thread and waits to start backing up until Slave_open_temp_tables in SHOW STATUS is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds (defaults to 300 seconds). The replication SQL thread will be restarted when the backup finishes.
Note
Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.
As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.
Using this option is always recommended when taking backups from a replica server.
Warning
Make sure your replica is a true replica of the source before using it as a source for backup. A good tool to validate a replica is pt-table-checksum.
"},{"location":"page-tracking.html","title":"Take an incremental backup using page tracking","text":"Percona XtraBackup 8.0.27 adds support for page tracking functionality for an incremental backup.
To create an incremental backup with page tracking, Percona XtraBackup uses the MySQL mysqlbackup component. This component provides a list of pages modified since the last backup, and Percona XtraBackup copies only those pages. This operation removes the need to scan the pages in the database. If the majority of pages have not been modified, the page tracking feature can improve the speed of incremental backups.
To start using the page tracking functionality, do the following:
-
Install the
mysqlbackupcomponent and enable it on the server:$ INSTALL COMPONENT \"file://component_mysqlbackup\";\n -
Check whether the
mysqlbackupcomponent is installed successfully:$ SELECT COUNT(1) FROM mysql.component WHERE component_urn='file://component_mysqlbackup';\n
You can enable the page tracking functionality for the full and incremental backups with the --page-tracking option.
The option has the following benefits:
-
Resets page tracking to the start of the backup. This reset allows the next incremental backup to use page tracking.
-
Allows the use of page tracking for an incremental backup if the page tracking data is available from the backup\u2019s start checkpoint LSN.
Percona XtraBackup processes a list of all the tracked pages in memory. If Percona XtraBackup does not have enough available memory to process this list, the process throws an error and exits. For example, if an incremental backup uses 200GB, Percona XtraBackup can use an additional 100MB of memory to process and store the page tracking data.
The examples of creating full and incremental backups using the --page-tracking option:
$ xtrabackup --backup --target-dir=$FULL_BACK --page-tracking\n$ xtrabackup --backup --target-dir=$INC_BACKUP \n--incremental-basedir=$FULL_BACKUP --page-tracking\nAfter enabling the functionality, the next incremental backup finds changed pages using page tracking.
The first full backup using page tracking, Percona XtraBackup may have a delay. The following is an example of the message:
Expected outputxtrabackup: pagetracking: Sleeping for 1 second, waiting for checkpoint lsn 17852922 /\nto reach to page tracking start lsn 21353759\nEnable page tracking before creating the first backup to avoid this delay. This method ensures that the page tracking log sequence number (LSN) is higher than the checkpoint LSN of the server.
"},{"location":"page-tracking.html#start-page-tracking-manually","title":"Start page tracking manually","text":"After the mysqlbackup component is loaded and active on the server, you can start page tracking manually with the following option:
$ SELECT mysqlbackup_page_track_set(true);\nCheck the LSN value starting from which changed pages are tracked with the following option:
$ SELECT mysqlbackup_page_track_get_start_lsn();\nTo stop page tracking, use the following command:
$ SELECT mysqlbackup_page_track_set(false);\nWhen you start page tracking, it creates a file under the server\u2019s datadir to collect data about changed pages. This file grows until you stop the page tracking. If you stop the server and then restart it, page tracking creates a new file but also keeps the old one. The old file continues to grow until you stop the page tracking explicitly.
If you purge the page tracking data, you should create a full backup afterward. To purge the page tracking data, do the following steps:
$ SELECT mysqlbackup_page_track_set(false);\n$ SELECT mysqlbackup_page_track_purge_up_to(9223372036854775807);\n/* Specify the LSN up to which you want to purge page tracking data. /\n9223372036854775807 is the highest possible LSN which purges all page tracking files.*/\n$ SELECT mysqlbackup_page_track_set(true);\nIf the index is built in place using an exclusive algorithm and then is added to a table after the last LSN checkpoint, you may generate a bad incremental backup using page tracking. For more details see PS-8032.
"},{"location":"page-tracking.html#uninstall-the-mysqlbackup-component","title":"Uninstall the mysqlbackup component","text":"To uninstall the mysqlbackup component, use the following statement:
$ UNINSTALL COMPONENT \"file://component_mysqlbackup\"\nWe will be referring to permissions to the ability of a user to access and perform changes on the relevant parts of the host\u2019s filesystem, starting/stopping services and installing software.
By privileges, we refer to the abilities of a database user to perform different kinds of actions on the database server.
There are many ways for checking the permission on a file or directory. For example, ls -ls /path/to/file or stat /path/to/file | grep Access will do the job:
$ stat /etc/mysql | grep Access\nAccess: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)\nAccess: 2011-05-12 21:19:07.129850437 -0300\n$ ls -ld /etc/mysql/my.cnf\n-rw-r--r-- 1 root root 4703 Apr 5 06:26 /etc/mysql/my.cnf\nAs in this example, my.cnf is owned by root and not writable for anyone else. Assuming that you do not have root\u2018s password, you can check what permissions you have on these types of files with sudo -l:
$ sudo -l\nPassword:\nYou may run the following commands on this host:\n(root) /usr/bin/\n(root) NOPASSWD: /etc/init.d/mysqld\n(root) NOPASSWD: /bin/vi /etc/mysql/my.cnf\n(root) NOPASSWD: /usr/local/bin/top\n(root) NOPASSWD: /usr/bin/ls\n(root) /bin/tail\nBeing able to execute with sudo scripts in /etc/init.d/, /etc/rc.d/ or /sbin/service is the ability to start and stop services.
Also, If you can execute the package manager of your distribution, you can install or remove software with it. If not, having rwx permission over a directory will let you do a local installation of software by compiling it there. This is a typical situation in many hosting companies\u2019 services.
There are other ways for managing permissions, such as using PolicyKit, * Extended ACLs or SELinux*, which may be preventing or allowing your access. You should check them in that case.
See also
Connection and privileges needed
"},{"location":"point-in-time-recovery.html","title":"Point-in-time recovery","text":"Recovering up to particular moment in database\u2019s history can be done with xtrabackup and the binary logs of the server.
Note that the binary log contains the operations that modified the database from a point in the past. You need a full datadir as a base, and then you can apply a series of operations from the binary log to make the data match what it was at the point in time you want.
$ xtrabackup --backup --target-dir=/path/to/backup\n$ xtrabackup --prepare --target-dir=/path/to/backup\nFor more details on these procedures, see Creating a backup and Preparing a backup.
Now, suppose that some time has passed, and you want to restore the database to a certain point in the past, having in mind that there is the constraint of the point where the snapshot was taken.
To find out what is the situation of binary logging in the server, execute the following queries:
mysql> SHOW BINARY LOGS;\n+------------------+-----------+\n| Log_name | File_size |\n+------------------+-----------+\n| mysql-bin.000001 | 126 |\n| mysql-bin.000002 | 1306 |\n| mysql-bin.000003 | 126 |\n| mysql-bin.000004 | 497 |\n+------------------+-----------+\nand
mysql> SHOW MASTER STATUS;\n+------------------+----------+--------------+------------------+\n| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |\n+------------------+----------+--------------+------------------+\n| mysql-bin.000004 | 497 | | |\n+------------------+----------+--------------+------------------+\nThe first query will tell you which files contain the binary log and the second one which file is currently being used to record changes, and the current position within it. Those files are stored usually in the datadir (unless other location is specified when the server is started with the --log-bin= option).
To find out the position of the snapshot taken, see the xtrabackup_binlog_info at the backup\u2019s directory:
$ cat /path/to/backup/xtrabackup_binlog_info\nmysql-bin.000003 57\nThis will tell you which file was used at moment of the backup for the binary log and its position. That position will be the effective one when you restore the backup:
$ xtrabackup --copy-back --target-dir=/path/to/backup\nAs the restoration will not affect the binary log files (you may need to adjust file permissions, see Restoring a Backup), the next step is extracting the queries from the binary log with mysqlbinlog starting from the position of the snapshot and redirecting it to a file
$ mysqlbinlog /path/to/datadir/mysql-bin.000003 /path/to/datadir/mysql-bin.000004 \\\n--start-position=57 > mybinlog.sql\nNote that if you have multiple files for the binary log, as in the example, you have to extract the queries with one process, as shown above.
Inspect the file with the queries to determine which position or date corresponds to the point-in-time wanted. Once determined, pipe it to the server. Assuming the point is 11-12-25 01:00:00:
$ mysqlbinlog /path/to/datadir/mysql-bin.000003 /path/to/datadir/mysql-bin.000004 \\\n--start-position=57 --stop-datetime=\"11-12-25 01:00:00\" | mysql -u root -p\nand the database will be rolled forward up to that Point-In-Time.
"},{"location":"prepare-compressed-backup.html","title":"Prepare the backup","text":"Before you can prepare the backup you\u2019ll need to uncompress all the files. Percona XtraBackup has implemented --decompress option that can be used to decompress the backup.
$ xtrabackup --decompress --target-dir=/data/compressed/\nNote
--parallel can be used with --decompress option to decompress multiple files simultaneously.
Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory you should use --remove-original option. Even if they\u2019re not removed these files will not be copied/moved over to the datadir if --copy-back or --move-back are used.
When the files are uncompressed you can prepare the backup with the --prepare option:
$ xtrabackup --prepare --target-dir=/data/compressed/\nInnoDB: Starting shutdown...\nInnoDB: Shutdown completed; log sequence number 9293846\n170223 13:39:31 completed OK!\nNow the files in /data/compressed/ are ready to be used by the server.
The next step is to restore the backup.
"},{"location":"prepare-full-backup.html","title":"Prepare a full backup","text":"After making a backup with the --backup option, you need to prepare it in order to restore it. Data files are not point-in-time consistent until they are prepared, because they were copied at different times as the program ran, and they might have been changed while this was happening.
If you try to start InnoDB with these data files, it will detect corruption and stop working to avoid running on damaged data. The --prepare step makes the files perfectly consistent at a single instant in time, so you can run InnoDB on them.
You can run the prepare operation on any machine; it does not need to be on the originating server or the server to which you intend to restore. You can copy the backup to a utility server and prepare it there.
Note that Percona XtraBackup 8.0 can only prepare backups of MySQL 8.0, Percona Server for MySQL 8.0, and Percona XtraDB Cluster 8.0 databases. Releases prior to 8.0 are not supported.
During the prepare operation, xtrabackup boots up a kind of modified embedded InnoDB (the libraries xtrabackup was linked against). The modifications are necessary to disable InnoDB standard safety checks, such as complaining about the log file not being the right size. This warning is not appropriate for working with backups. These modifications are only for the xtrabackup binary; you do not need a modified InnoDB to use xtrabackup for your backups.
The prepare step uses this \u201cembedded InnoDB\u201d to perform crash recovery on the copied data files, using the copied log file. The prepare step is very simple to use: you simply run xtrabackup with the --prepare option and tell it which directory to prepare, for example, to prepare the previously taken backup run:
$ xtrabackup --prepare --target-dir=/data/backups/\nWhen this finishes, you should see an InnoDB shutdown with a message such as the following, where again the value of LSN will depend on your system:
InnoDB: Shutdown completed; log sequence number 137345046\n160906 11:21:01 completed OK!\nAll following prepares will not change the already prepared data files, you\u2019ll see that output says:
Expected outputxtrabackup: This target seems to be already prepared.\nxtrabackup: notice: xtrabackup_logfile was already used to '--prepare'.\nIt is not recommended to interrupt xtrabackup process while preparing backup because it may cause data files corruption and backup will become unusable. Backup validity is not guaranteed if prepare process was interrupted.
Note
If you intend the backup to be the basis for further incremental backups, you should use the --apply-log-only option when preparing the backup, or you will not be able to apply incremental backups to it. See the documentation on preparing incremental backups for more details.
After preparing the backup, you can restore the backup.
"},{"location":"prepare-incremental-backup.html","title":"Prepare an incremental backup","text":"The --prepare step for incremental backups is not the same as for full backups. In full backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing an incremental backup, because transactions that were uncommitted at the time of your backup may be in progress, and it\u2019s likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.
Warning
If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups will be useless. After transactions have been rolled back, further incremental backups cannot be applied.
Beginning with the full backup you created, you can prepare it, and then apply the incremental differences to it. Recall that you have the following backups:
/data/backups/base\n/data/backups/inc1\n/data/backups/inc2\nTo prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base\nThe output should end with text similar to the following:
Expected outputInnoDB: Shutdown completed; log sequence number 1626007\n161011 12:41:04 completed OK!\nThe log sequence number should match the to_lsn of the base backup, which you saw previously.
Warning
This backup is actually safe to restore as-is now, even though the rollback phase has been skipped. If you restore it and start MySQL, InnoDB will detect that the rollback phase was not performed, and it will do that in the background, as it usually does for a crash recovery upon start. It will notify you that the database was not shut down normally.
To apply the first incremental backup to the full backup, run the following command:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1\nThis applies the delta files to the files in /data/backups/base, which rolls them forward in time to the time of the incremental backup. It then applies the redo log as usual to the result. The final data is in /data/backups/base, not in the incremental directory. You should see an output similar to:
incremental backup from 1626007 is enabled.\nxtrabackup: cd to /data/backups/base\nxtrabackup: This target seems to be already prepared with --apply-log-only.\nxtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(4124244)\n...\nxtrabackup: page size for /tmp/backups/inc1/ibdata1.delta is 16384 bytes\nApplying /tmp/backups/inc1/ibdata1.delta to ./ibdata1...\n...\n161011 12:45:56 completed OK!\nAgain, the LSN should match what you saw from your earlier inspection of the first incremental backup. If you restore the files from /data/backups/base, you should see the state of the database as of the first incremental backup.
Warning
Percona XtraBackup does not support using the same incremental backup directory to prepare two copies of backup. Do not run --prepare with the same incremental backup directory (the value of \u2013incremental-dir) more than once.
Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:
$ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2\nNote
--apply-log-only should be used when merging the incremental backups except the last one. That\u2019s why the previous line does not contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase.
Once prepared incremental backups are the same as the full backups, and they can be restored in the same way.
"},{"location":"prepare-individual-partitions-backup.html","title":"Prepare an individual partitions backup","text":"For preparing partial backups, the procedure is analogous to restoring individual tables. Apply the logs and use xtrabackup --export:
$ xtrabackup --apply-log --export /mnt/backup/2012-08-28_10-29-09\nYou may see warnings in the output about tables that do not exist. This happens because InnoDB-based engines stores its data dictionary inside the tablespace files. xtrabackup removes the missing tables (those that haven\u2019t been selected in the partial backup) from the data dictionary in order to avoid future warnings or errors.
The next step is to restore the backup.
"},{"location":"prepare-partial-backup.html","title":"Prepare a partial backup","text":"The procedure is analogous to restoring individual tables: apply the logs and use the --export option:
$ xtrabackup --prepare --export --target-dir=/path/to/partial/backup\nWhen you use the --prepare option on a partial backup, you will see warnings about tables that don\u2019t exist. This is because these tables exist in the data dictionary inside InnoDB, but the corresponding .ibd files don\u2019t exist. They were not copied into the backup directory. These tables will be removed from the data dictionary, and when you restore the backup and start InnoDB, they will no longer exist and will not cause any errors or warnings to be printed to the log file.
Could not find any file associated with the tablespace ID: 5
Use --innodb-directories to find the tablespace files. If that fails then use -\u2013innodb-force-recovery=1 to ignore this and to permanently lose all changes to the missing tablespace(s).
The next step is to restore restore the partition from the backup.
"},{"location":"privileges.html","title":"Connection and privileges needed","text":"Percona XtraBackup needs to be able to connect to the database server and perform operations on the server and the datadir when creating a backup, when preparing in some scenarios and when restoring it. In order to do so, there are privileges and permission requirements on its execution that must be fulfilled.
Privilege refers to the operations that a system user is permitted to do in the database server. They are set at the database server and only apply to users in the database server.
Permissions are those which permits a user to perform operations on the system, like reading, writing or executing on a certain directory or start/stop a system service. They are set at a system level and only apply to system users.
When xtrabackup is used, there are two actors involved: the user invoking the program - a system user - and the user performing action in the database server - a database user. Note that these are different users in different places, even though they may have the same username.
All the invocations of xtrabackup in this documentation assume that the system user has the appropriate permissions, and you are providing the relevant options for connecting the database server - besides the options for the action to be performed - and the database user has adequate privileges.
"},{"location":"privileges.html#connect-to-the-server","title":"Connect to the server","text":"The database user used to connect to the server and its password are specified by the --user and --password option:
$ xtrabackup --user=DVADER --password=14MY0URF4TH3R --backup \\\n--target-dir=/data/bkps/\nIf you don\u2019t use the --user option, Percona XtraBackup will assume the database user whose name is the system user executing it.
According to your system, you may need to specify one or more of the following options to connect to the server:
Option Description -port Use this port when connecting to the database with TCP/IP -socket Use this socket when connecting to the local database. -host Use this host when connecting to the database server with TCP/IPThese options are passed to the mysql child process without alteration, see mysql --help for details.
Note
In case of multiple server instances, the correct connection parameters (port, socket, host) must be specified in order for xtrabackup to talk to the correct server.
"},{"location":"privileges.html#privileges-needed","title":"Privileges needed","text":"Once connected to the server, in order to perform a backup you need READ and EXECUTE permissions at a filesystem level in the server\u2019s datadir.
The database user needs the following privileges to back up tables or databases:
-
RELOADandLOCK TABLES(unless the--no-lockoption is specified) in order to runFLUSH TABLES WITH READ LOCKandFLUSH ENGINE LOGSprior to start copying the files, and requires this privilege when Backup Locks are used -
BACKUP_ADMINprivilege is needed to query the performance_schema.log_status table, and runLOCK INSTANCE FOR BACKUP,LOCK BINLOG FOR BACKUP, orLOCK TABLES FOR BACKUP. -
REPLICATION CLIENTin order to obtain the binary log position, -
CREATE TABLESPACEin order to import tables (see Restoring Individual Tables), -
PROCESSin order to runSHOW ENGINE INNODB STATUS(which is mandatory), and optionally to see all threads which are running on the server (see FLUSH TABLES WITH READ LOCK option), -
SUPERin order to start/stop the replication threads in a replication environment, use XtraDB Changed Page Tracking for Incremental Backups and for handling FLUSH TABLES WITH READ LOCK, -
CREATEprivilege in order to create the PERCONA_SCHEMA.xtrabackup_history database and table, -
ALTERprivilege in order to upgrade the PERCONA_SCHEMA.xtrabackup_history database and table, -
INSERTprivilege in order to add history records to the PERCONA_SCHEMA.xtrabackup_history table, -
SELECTprivilege in order to use--incremental-history-nameor--incremental-history-uuidin order for the feature to look up theinnodb_to_lsnvalues in the PERCONA_SCHEMA.xtrabackup_history table. -
SELECTprivilege on the keyring_component_status table to view the attributes and status of the installed keyring component when in use. -
SELECTprivilege on the replication_group_members table to validate if the instance is part of group replication cluster.
A SQL example of creating a database user with the minimum privileges required to take full backups would be:
mysql> CREATE USER 'bkpuser'@'localhost' IDENTIFIED BY 's3cr%T';\nmysql> GRANT BACKUP_ADMIN, PROCESS, RELOAD, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'bkpuser'@'localhost';\nmysql> GRANT SELECT ON performance_schema.log_status TO 'bkpuser'@'localhost';\nmysql> GRANT SELECT ON performance_schema.keyring_component_status TO bkpuser@'localhost';\nmysql> GRANT SELECT ON performance_schema.replication_group_members TO bkpuser@'localhost';\nmysql> FLUSH PRIVILEGES;\nTo query the privileges that your database user has been granted at the console of the server execute:
mysql> SHOW GRANTS;\nor for a particular user with:
mysql> SHOW GRANTS FOR 'db-user'@'host';\nIt will display the privileges using the same format as for the GRANT statement.
Note that privileges may vary across versions of the server. To list the exact list of privileges that your server support (and a brief description of them) execute:
mysql> SHOW PRIVILEGES;\nSee also
Permissions needed
"},{"location":"quickstart-overview.html","title":"Quickstart Guide for Percona XtraBackup","text":"Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.
"},{"location":"quickstart-overview.html#install-percona-xtrabackup","title":"Install Percona XtraBackup:","text":"You can install Percona XtraBackup using different methods:
-
Use the Percona Repositories
-
Use APT
-
Use YUM
-
Use binary tarballs
-
Use Docker
Percona Server for MySQL (PS) is a freely available, fully compatible, enhanced, and open source drop-in replacement for any MySQL database. It provides superior and optimized performance, greater scalability and availability, enhanced backups, increased visibility, and instrumentation. Percona Server for MySQL is trusted by thousands of enterprises to provide better performance and concurrency for their most demanding workloads.
Install Percona Server for MySQL.
"},{"location":"quickstart-overview.html#for-high-availability","title":"For high availability","text":"Percona XtraDB Cluster (PXC) is a 100% open source, enterprise-grade, highly available clustering solution for MySQL multi-master setups based on Galera. PXC helps enterprises minimize unexpected downtime and data loss, reduce costs, and improve performance and scalability of your database environments supporting your critical business applications in the most demanding public, private, and hybrid cloud environments.
Install Percona XtraDB Cluster.
"},{"location":"quickstart-overview.html#for-monitoring-and-management","title":"For Monitoring and Management","text":"Percona Monitoring and Management (PMM) monitors and provides actionable performance data for MySQL variants, including Percona Server for MySQL, Percona XtraDB Cluster, Oracle MySQL Community Edition, Oracle MySQL Enterprise Edition, and MariaDB. PMM captures metrics and data for the InnoDB, XtraDB, and MyRocks storage engines, and has specialized dashboards for specific engine details.
Install PMM and connect your MySQL instances to it.
"},{"location":"release-notes.html","title":"Percona XtraBackup 8.0 release notes index","text":"-
Percona XtraBackup 8.0.33-28 (2023-07-19)
-
Percona XtraBackup 8.0.33-27 (2023-05-25)
-
Percona XtraBackup 8.0.32-26 (2023-04-04)
-
Percona XtraBackup 8.0.32-25 (2023-02-27)
-
Percona XtraBackup 8.0.31-24 (2023-02-07)
-
Percona XtraBackup 8.0.30-23 (2022-11-14)
-
Percona XtraBackup 8.0.29-22 (2022-07-19)
-
Percona XtraBackup 8.0.28-21 (2022-05-25)
-
Percona XtraBackup 8.0.28-20.0 (2022-04-26)
-
Percona Xtrabackup 8.0.27-19.0 (2022-02-02)
-
Percona XtraBackup 8.0.26-18.0 (2021-09-02)
-
Percona Xtrabackup 8.0.25-17.0 (2021-06-03)
-
Percona XtraBackup 8.0.23-16.0 (2021-03-22)
-
Percona Xtrabackup 8.0.22-15.0 (2020-12-14)
-
Percona XtraBackup 8.0.14 (2020-08-31)
-
Percona Xtrabackup 8.0.13 (2020-06-17)
-
Percona XtraBackup 8.0.12 (2020-05-27)
-
Percona Xtrabackup 8.0.11 (2020-04-13)
-
Percona XtraBackup 8.0.10 (2020-03-16)
-
Percona Xtrabackup 8.0.9 (2019-12-16)
-
Percona XtraBackup 8.0.8 (2019-11-21)
-
Percona Xtrabackup 8.0.7 (2019-08-07)
-
Percona Xtrabackup 8.0.6 (2019-05-09)
-
Percona XtraBackup 8.0.5 (2019-03-04)
-
Percona Xtrabackup 8.0.4 (2018-12-10)
-
Percona XtraBackup 8.0-3-rc1 (2018-10-31)
Warning
Backup needs to be prepared before it can be restored.
For convenience, xtrabackup binary has the --copy-back option to copy the backup to the datadir of the server:
$ xtrabackup --copy-back --target-dir=/data/backups/\nIf you don\u2019t want to save your backup, you can use the --move-back option which will move the backed up data to the datadir.
If you don\u2019t want to use any of the above options, you can additionally use rsync or cp to restore the files.
Note
The datadir must be empty before restoring the backup. Also, it\u2019s important to note that MySQL server needs to be shut down before restore is performed. You cannot restore to a datadir of a running mysqld instance (except when importing a partial backup).
Example of the rsync command that can be used to restore the backup can look like this:
$ rsync -avrP /data/backup/ /var/lib/mysql/\nYou should check that the restored files have the correct ownership and permissions.
As files\u2019 attributes will be preserved, in most cases you will need to change the files\u2019 ownership to mysql before starting the database server, as they will be owned by the user who created the backup:
$ chown -R mysql:mysql /var/lib/mysql\nData is now restored, and you can start the server.
"},{"location":"restore-individual-partitions.html","title":"Restore the partition from the backup","text":"Restoring should be done by importing the tables in the partial backup to the server.
First step is to create new table in which data will be restored.
mysql> CREATE TABLE `name_p4` (\n`id` int(11) NOT NULL AUTO_INCREMENT,\n`name` text NOT NULL,\n`imdb_index` varchar(12) DEFAULT NULL,\n`imdb_id` int(11) DEFAULT NULL,\n`name_pcode_cf` varchar(5) DEFAULT NULL,\n`name_pcode_nf` varchar(5) DEFAULT NULL,\n`surname_pcode` varchar(5) DEFAULT NULL,\nPRIMARY KEY (`id`)\n) ENGINE=InnoDB AUTO_INCREMENT=2812744 DEFAULT CHARSET=utf8\nNote
Generate a .cfg metadata file by running FLUSH TABLES ... FOR EXPORT. The command can only be run on a table, not on the individual table partitions. The file is located in the table schema directory and is used for schema verification when importing the tablespace.
To restore the partition from the backup, the tablespace must be discarded for that table:
mysql> ALTER TABLE name_p4 DISCARD TABLESPACE;\nThe next step is to copy the .ibd file from the backup to the MySQL data directory:
cp /mnt/backup/2012-08-28_10-29-09/imdb/name#P#p4.ibd /var/lib/mysql/imdb/name_p4.ibd\nNote
Make sure that the copied files can be accessed by the user running MySQL.
The last step is to import the tablespace:
mysql> ALTER TABLE name_p4 IMPORT TABLESPACE;\nPercona XtraBackup can export a table that is contained in its own .ibd file. With Percona XtraBackup, you can export individual tables from any InnoDB database, and import them into Percona Server for MySQL with XtraDB or MySQL 8.0. The source doesn\u2019t have to be XtraDB or MySQL 8.0, but the destination does. This method only works on individual .ibd files.
The following example exports and imports the following table:
CREATE TABLE export_test (\na int(11) DEFAULT NULL\n) ENGINE=InnoDB DEFAULT CHARSET=latin1;\nTo generate an .ibd file in the target directory, create the table using the innodb_file_per_table mode:
$ find /data/backups/mysql/ -name export_test.*\n/data/backups/mysql/test/export_test.ibd\nDuring the --prepare step, add the --export option to the command. For example:
$ xtrabackup --prepare --export --target-dir=/data/backups/mysql/\nWhen restoring an encrypted InnoDB tablespace table, add the keyring file:
$ xtrabackup --prepare --export --target-dir=/tmp/table \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\nThe following files are the only files required to import the table into a server running Percona Server for MySQL with XtraDB or MySQL 8.0. If the server uses InnoDB Tablespace Encryption, add the .cfp file, which contains the transfer key and an encrypted tablespace key.
The files are located in the target directory:
/data/backups/mysql/test/export_test.ibd\n/data/backups/mysql/test/export_test.cfg\nOn the destination server running Percona Server for MySQL with XtraDB or MySQL 8.0, create a table with the same structure, and then perform the following steps:
-
Run the
Error messageALTER TABLE test.export_test DISCARD TABLESPACE;command. If you see the following error message:ERROR 1809 (HY000): Table 'test/export_test' in system tablespace\nenable
innodb_file_per_tableoption on the server and create the table again.$ set global innodb_file_per_table=ON;\n -
Copy the exported files to the
test/subdirectory of the destination server\u2019s data directory. -
Run
ALTER TABLE test.export_test IMPORT TABLESPACE;The table is imported, and you can run a
SELECTto see the imported data.
Restoring should be done by restoring individual tables in the partial backup to the server.
It can also be done by copying back the prepared backup to a \u201cclean\u201d datadir (in that case, make sure to include the mysql database) to the datadir you are moving the backup to. A system database can be created with the following:
$ sudo mysql --initialize --user=mysql\nOnce you start the server, you may see mysql complaining about missing tablespaces:
Expected output2021-07-19T12:42:11.077200Z 1 [Warning] [MY-012351] [InnoDB] Tablespace 4, name 'test1/t1', file './d2/test1.ibd' is missing!\n2021-07-19T12:42:11.077300Z 1 [Warning] [MY-012351] [InnoDB] Tablespace 4, name 'test1/t1', file './d2/test1.ibd' is missing!\nIn order to clean the orphan database from the data dictionary, you must manually create the missing database directory and then DROP this database from the server.
Example of creating the missing database:
$ mkdir /var/lib/mysql/test1/d2\nExample of dropping the database from the server:
mysql> DROP DATABASE d2;\nQuery OK, 2 rows affected (0.5 sec)\nA MySQL change to a feature, such as the structure of a redo log record, can cause older versions of Percona XtraBackup to fail. To ensure that you can back up and restore your data, use a Percona XtraBackup version that is equal to or above your source server version.
See also
How XtraBackup works
Percona XtraBackup 8.0.21 adds the --no-server-version-check option. Before the backup starts, XtraBackup compares the source system version to the Percona XtraBackup version. If the source system version is greater than the XtraBackup version, XtraBackup stops the backup and returns an error message. This comparison prevents a failed backup or a corrupted backup due to source system changes.
The parameter checks for the following scenarios:
-
The source system and the PXB version are the same, the backup proceeds
-
The source system is less than the PXB version, the backup proceeds
-
The source system is greater than the PXB version, and the parameter is not overridden, the backup is stopped and returns an error message
-
The source system is greater than the PXB version, and the parameter is overridden, the backup proceeds
Explicitly adding the --no-server-version-check parameter, like the example, overrides the parameter and the backup proceeds.
$ xtrabackup --backup --no-server-version-check --target-dir=$mysql/backup1\nWhen you override the parameter, the following events can happen:
-
Backup fails
-
Creates a corrupted backup
-
Backup successful
Data is, by far, the most valuable part of a system. Having a backup done systematically and available for a rapid recovery in case of failure is admittedly essential to a system. However, it is not common practice because of its costs, infrastructure needed or even the boredom associated to the task. Percona XtraBackup is designed to solve this problem.
You can have almost real-time backups in 6 simple steps by setting up a replication environment with Percona XtraBackup.
"},{"location":"set-up-replication.html#all-the-things-you-will-need","title":"All the things you will need","text":"Setting up a replica for replication with Percona XtraBackup is a straightforward procedure. In order to keep it simple, here is a list of the things you need to follow the steps without hassles:
Source
A system with a MySQL-based server installed, configured and running. This system will be called Source, as it is where your data is stored and the one to be replicated. We will assume the following about this system:
-
the MySQL server is able to communicate with others by the standard TCP/IP port;
-
the SSH server is installed and configured;
-
you have a user account in the system with the appropriate permissions;
-
you have a MySQL\u2019s user account with appropriate privileges.
-
server has binlogs enabled and server-id set up to 1.
Replica
Another system, with a MySQL-based server installed on it. We will refer to this machine as Replica and we will assume the same things we did about Source, except that the server-id on Replica is 2.
Percona XtraBackup
The backup tool we will use. It should be installed in both computers for convenience.
Note
It is not recommended to mix MySQL variants (Percona Server, MySQL) in your replication setup. This may produce incorrect xtrabackup_slave_info file when adding a new replica.
The 8.0.23 version deprecates the CHANGE_MASTER_TO command. In that version and later, use the CHANGE_REPLICATION_SOURCE_TO and the appropriate options instead.
The 8.0.22 version deprecates theSTART SLAVE command. In that version or later, use START REPLICA instead.
Source and prepare it","text":"At the Source, issue the following to a shell:
$ xtrabackup --backup --user=yourDBuser --password=MaGiCdB1 --target-dir=/path/to/backupdir\nAfter this is finished you should get:
Expected outputxtrabackup: completed OK!\nThis will make a copy of your MySQL data dir to the /path/to/backupdir directory. You have told Percona XtraBackup to connect to the database server using your database user and password, and do a hot backup of all your data in it (all MyISAM, InnoDB tables and indexes in them).
In order for snapshot to be consistent you need to prepare the data on the source:
$ xtrabackup --prepare --target-dir=/path/to/backupdir\nYou need to select path where your snapshot has been taken. If everything is ok you should get the same OK message. Now the transaction logs are applied to the data files, and new ones are created: your data files are ready to be used by the MySQL server.
Percona XtraBackup knows where your data is by reading your my.cnf file. If you have your configuration file in a non-standard place, you should use the flag --defaults-file =/location/of/my.cnf.
If you want to skip writing the username and password every time you want to access MySQL, you can set it up in .mylogin.cnf as follows:
mysql_config_editor set --login-path=client --host=localhost --user=root --password\nFor more information, see MySQL Configuration Utility.
This statement provides root access to MySQL.
"},{"location":"set-up-replication.html#2-copy-backed-up-data-to-the-replica","title":"2. Copy backed up data to the Replica","text":"On the Source, use rsync or scp to copy the data from the Source to the Replica. If you are syncing the data directly to replica\u2019s data directory, we recommend that you stop the mysqld there.
$ rsync -avpP -e ssh /path/to/backupdir Replica:/path/to/mysql/\nAfter data has been copied, you can back up the original or previously installed MySQL datadir (NOTE: Make sure mysqld is shut down before you move the contents of its datadir, or move the snapshot into its datadir.). Run the following commands on the Replica:
$ mv /path/to/mysql/datadir /path/to/mysql/datadir_bak\nand move the snapshot from the Source in its place:
$ xtrabackup --move-back --target-dir=/path/to/mysql/backupdir\nAfter you copy data over, make sure the Replica MySQL has the proper permissions to access them.
$ chown mysql:mysql /path/to/mysql/datadir\nIf the ibdata and iblog files are located in directories outside the datadir, you must put these files in their proper place after the logs have been applied.
"},{"location":"set-up-replication.html#3-configure-the-sources-mysql-server","title":"3. Configure the Source\u2019s MySQL server","text":"On the source, run the following command to add the appropriate grant. This grant allows the replica to be able to connect to source:
mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'$replicaip'\nIDENTIFIED BY '$replicapass';\nAlso make sure that firewall rules are correct and that the Replica can connect to the Source. Run the following command on the Replica to test that you can run the mysql client on Replica, connect to the Source, and authenticate.
mysql> mysql --host=Source --user=repl --password=$replicapass\nVerify the privileges.
mysql> SHOW GRANTS;\nCopy the my.cnf file from the Source to the Replica:
$ scp user@Source:/etc/mysql/my.cnf /etc/mysql/my.cnf\nand change the following options in /etc/mysql/my.cnf:
server-id=2\nand start/restart mysqld on the Replica.
In case you\u2019re using init script on Debian-based system to start mysqld, be sure that the password for debian-sys-maint user has been updated, and it\u2019s the same as that user\u2019s password on the Source. Password can be seen and updated in /etc/mysql/debian.cnf.
On the Replica, review the content of the file xtrabackup_binlog_info, it will be something like:
$ cat /var/lib/mysql/xtrabackup_binlog_info\nThe results should resemble the following:
Expected outputSource-bin.000001 481\nThe term master is deprecated. Do the following on a MySQL console and use the username and password you\u2019ve set up in STEP 3 :
-
Version 8.0.23 or later, use the
CHANGE_REPLICATION_SOURCE_TOstatement -
Before 8.0.23, use the
CHANGE MASTERstatement
CHANGE REPLICATION SOURCE TO\nSOURCE_HOST='$sourceip',\nSOURCE_USER='repl',\nSOURCE_PASSWORD='$replicapass',\nSOURCE_LOG_FILE='Source-bin.000001',\nSOURCE_LOG_POS=481;\nStart the replica:
START REPLICA;\nThe term slave is deprecated. Do the following:
- Version 8.0.22 or later, use
START REPLICA - Before version 8.0.22, use
START SLAVE
On the Replica, check that everything went OK with:
SHOW REPLICA STATUS\\G\nThe result shows the status:
Expected outputSlave_IO_Running: Yes\nSlave_SQL_Running: Yes\nSeconds_Behind_Master: 13\nBoth IO and SQL threads need to be running. The Seconds_Behind_Master means the SQL currently being executed has a current_timestamp of 13 seconds ago. It is an estimation of the lag between the Source and the Replica. Note that at the beginning, a high value could be shown because the Replica has to \u201ccatch up\u201d with the Source.
You can use this procedure with slight variation to add new replicas to a source. We will use Percona XtraBackup to clone an already configured replica. We will continue using the previous scenario for convenience, but we will add a NewReplica to the plot.
At the Replica, do a full backup:
$ xtrabackup --user=yourDBuser --password=MaGiCiGaM \\\n--backup --slave-info --target-dir=/path/to/backupdir\nBy using the --slave-info Percona XtraBackup creates additional file called xtrabackup_slave_info.
Apply the logs:
$ xtrabackup --prepare --use-memory=2G --target-dir=/path/to/backupdir/\nNote
In the prepare phase, the --use-memory parameter speeds up the process if the amount of RAM assigned to the option is available. Use the parameter only in the prepare phase. In the other phases the parameter makes the application lazy allocate this memory (reserve) but does not affect database pages.
Copy the directory from the Replica to the NewReplica (NOTE: Make sure mysqld is shut down on the NewReplica before you copy the contents the snapshot into its datadir.):
rsync -avprP -e ssh /path/to/backupdir NewReplica:/path/to/mysql/datadir\nFor example, to set up a new user, user2, you add another grant on the Source:
> GRANT REPLICATION SLAVE ON *.* TO 'user2'@'$newreplicaip'\nIDENTIFIED BY '$replicapass';\nOn the NewReplica, copy the configuration file from the Replica:
$ scp user@Replica:/etc/mysql/my.cnf /etc/mysql/my.cnf\nMake sure you change the server-id variable in /etc/mysql/my.cnf to 3 and disable the replication on start:
skip-slave-start\nserver-id=3\nAfter setting server_id, start mysqld.
Fetch the master_log_file and master_log_pos from the file xtrabackup_slave_info, execute the statement for setting up the source and the log file for the NewReplica:
> CHANGE MASTER TO\n MASTER_HOST='$Sourceip',\n MASTER_USER='repl',\n MASTER_PASSWORD='$replicapass',\n MASTER_LOG_FILE='Source-bin.000001',\n MASTER_LOG_POS=481;\nThe term master is deprecated. Do the following and then start the replica:
-
Version 8.0.23 or later, use the
CHANGE_REPLICATION_SOURCE_TOstatement -
Before 8.0.23, use the
CHANGE MASTERstatement
The term slave is deprecated. Do the following:
- Version 8.0.22 or later, use
START REPLICA. - Before version 8.0.22, use
START SLAVE
> START REPLICA;\nIf both IO and SQL threads are running when you check the NewReplica, server is replicating the Source.
See also
How to create a new (or repair a broken) GTID based slave
"},{"location":"smart-memory-estimation.html","title":"Smart memory estimation","text":"The Smart memory estimation is tech preview feature. Before using Smart memory estimation in production, we recommend that you test restoring production from physical backups in your environment and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the Smart memory estimation feature. With this feature, Percona XtraBackup computes the memory required for prepare phase, while copying redo log entries during the backup phase. Percona XtraBackup also considers the number of InnoDB pages to be fetched from the disk.
Percona XtraBackup performs the backup procedure in two steps:
-
Creates a backup
To create a backup, Percona XtraBackup copies your InnoDB data files. While copying the files, Percona XtraBackup runs a background process that watches the InnoDB redo log, also called the transaction log, and copies changes from it.
-
Prepares a backup
During the
preparephase, Percona XtraBackup performs crash recovery against the copied data files using the copied transaction log file. Percona XtraBackup reads all the redo log entries into memory, categorizes them by space id and page id, reads the relevant pages into memory, and checks the log sequence number (LSN) on the page and on the redo log record. If the redo log LSN is more recent than the page LSN, Percona XtraBackup applies the redo log changes to the page.To
preparea backup, Percona Xtrabackup uses InnoDB Buffer Pool memory. Percona Xtrabackup reserves memory to load 256 pages into the buffer pool. The remaining memory is used for hashing/categorizing the redo log entries.The available memory is controlled by the
--use-memoryoption. If the available memory on the buffer pool is insufficient, the work is performed in multiple batches. After the batch is processed, the memory is freed to release space for the next batch. This process greatly impacts performance as an InnoDB page holds data from multiple rows. If a change on a page happens in different batches, that page is fetched and evicted numerous times.
In the prepare phase, Percona XtraBackup checks the server\u2019s available free memory and uses that memory up to the limit specified in the --use-free-memory-pct option to run --prepare. Due to backward compatibility, the default value for the --use-free-memory-pct option is 0 (zero), which defines the option as disabled. For example, if you set --use-free-memory-pct=50, then 50% of the free memory is used to prepare a backup.
Starting with Percona XtraBackup 8.0.32-26, you can enable or disable the memory estimation during the backup phase with the --estimate-memory option. The default value is OFF. Enable the memory estimation with --estimate-memory=ON:
$ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\nIn the prepare phase, enable the --use-free-memory-pct option by specifying the percentage of free memory to be used to prepare a backup. The --use-free-memory-pct value must be larger than 0.
For example:
$ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\nThe examples of how Smart memory estimation can improve the time spent on prepare in different versions of Percona XtraBackup:
We back up 16, 32, and 64 tables using sysbench. Each set contains 1M rows. In the backup phase, we enable Smart memory estimation with --estimate-memory=ON. In the prepare phase, we set --use-free-memory-pct=50, and Percona XtraBackup uses 50% of the free memory to prepare a backup. The backup is run on an ec2 c4.8xlarge instance (36 vCPUs / 60GB memory / General Purpose SSD (gp2)).
We back up 16, 32, and 64 tables using sysbench. Each set contains 1M rows. In the prepare phase, we set --use-free-memory-pct=50, and Percona XtraBackup uses 50% of the free memory to prepare a backup. The backup is run on an ec2 c4.8xlarge instance (36 vCPUs / 60GB memory / General Purpose SSD (gp2)).
During each --backup, the following sysbench is run:
sysbench --db-driver=mysql --db-ps-mode=disable --mysql-user=sysbench --mysql-password=sysbench --table_size=1000000 --tables=${NUM_OF_TABLES} --threads=24 --time=0 --report-interval=1 /usr/share/sysbench/oltp_write_only.lua run\nThe following table shows the backup details (all measurements are in Gigabytes):
Used memory Size of XtraBackup log Size of backup 16 tables 3.375 0.7 4.7 32 tables 8.625 2.6 11 64 tables 18.5 5.6 22-
Used memory - the amount of memory required by Percona XtraBackup with
--use-free-memory-pct=50 -
Size of XtraBackup log - the size of Percona XtraBackup log file (redo log entries copied during the backup)
-
Size of backup - the size of the resulting backup folder
Prepare executed without Smart memory estimation uses the default of 128MB for the buffer pool.
The results are the following:
Note
The following results are based on tests in a specific environment. Your results may vary.
-
16 tables result - prepare time dropped to ~5.7% of the original time. An improvement in recovery time of about 17x.
-
32 tables result - prepare time dropped to ~8,2% of the original time. An improvement in recovery time of about 12x.
-
64 tables result - prepare time dropped to ~9.9% of the original time. An improvement in recovery time of about 10x.
Percona XtraBackup supports storing the backups history on the server. This feature was implemented in Percona XtraBackup 2.2. Storing backup history on the server was implemented to provide users with additional information about backups that are being taken. Backup history information will be stored in the PERCONA_SCHEMA.XTRABACKUP_HISTORY table.
To use this feature the following options are available:
-
--history= : This option enables the history feature and allows the user to specify a backup series name that will be placed within the history record. -
--incremental-history-name= : This option allows an incremental backup to be made based on a specific history series by name. xtrabackup will search the history table looking for the most recent (highestto_lsn) backup in the series and take theto_lsnvalue to use as it\u2019s starting lsn. This is mutually exclusive with--incremental-history-uuid,--incremental-basedirand--incremental-lsnoptions. If no valid LSN can be found (no series by that name) xtrabackup will return with an error. -
--incremental-history-uuid= : Allows an incremental backup to be made based on a specific history record identified by UUID. xtrabackup will search the history table looking for the record matching UUID and take theto_lsnvalue to use as it\u2019s starting LSN. This options is mutually exclusive with--incremental-basedir,--incremental-lsnand--incremental-history-nameoptions. If no valid LSN can be found (no record by that UUID or missingto_lsn), xtrabackup will return with an error.Note
Backup that\u2019s currently being performed will NOT exist in the xtrabackup_history table within the resulting backup set as the record will not be added to that table until after the backup has been taken.
If you want access to backup history outside of your backup set in the case of some catastrophic event, you will need to either perform a
mysqldump, partial backup orSELECT* on the history table after xtrabackup completes and store the results with you backup set.For the necessary privileges, see Permissions and Privileges Needed.
"},{"location":"store-backup-history.html#percona_schemaxtrabackup_history-table","title":"PERCONA_SCHEMA.XTRABACKUP_HISTORY table","text":"This table contains the information about the previous server backups. Information about the backups will only be written if the backup was taken with
Column Name Description uuid Unique backup id name User provided name of backup series. There may be multiple entries with the same name used to identify related backups in a series. tool_name Name of tool used to take backup tool_command Exact command line given to the tool with --password and --encryption_key obfuscated tool_version Version of tool used to take backup ibbackup_version Version of the xtrabackup binary used to take backup server_version Server version on which backup was taken start_time Time at the start of the backup end_time Time at the end of the backup lock_time Amount of time, in seconds, spent calling and holding locks for--historyoption.FLUSH TABLES WITH READ LOCKbinlog_pos Binlog file and position at end ofFLUSH TABLES WITH READ LOCKinnodb_from_lsn LSN at beginning of backup which can be used to determine prior backups innodb_to_lsn LSN at end of backup which can be used as the starting lsn for the next incremental partial Is this a partial backup, ifNthat means that it\u2019s the full backup incremental Is this an incremental backup format Description of result format (xbstream) compact Is this a compact backup compressed Is this a compressed backup encrypted Is this an encrypted backup"},{"location":"store-backup-history.html#limitations","title":"Limitations","text":"-
--historyoption must be specified only on the command line and not within a configuration file in order to be effective. -
--incremental-history-nameand--incremental-history-uuidoptions must be specified only on the command line and not within a configuration file in order to be effective.
Percona XtraBackup supports streaming mode. Streaming mode sends a backup to
STDOUTin the xbstream format instead of copying the files to the backup directory.This method allows you to use other programs to filter the output of the backup, providing greater flexibility for storage of the backup. For example, compression is achieved by piping the output to a compression utility. One of the benefits of streaming backups and using Unix pipes is that the backups can be automatically encrypted.
To use the streaming feature, you must use the
--stream, providing the format of the stream (xbstream) and where to store the temporary files:$ xtrabackup --stream=xbstream --target-dir=/tmp\nxtrabackup uses xbstream to stream all of the data files to
STDOUT, in a specialxbstreamformat. After it finishes streaming all of the data files toSTDOUT, it stops xtrabackup and streams the saved log file too.When compression is enabled, xtrabackup compresses the output data, except for the meta and non-InnoDB files which are not compressed, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:
quicklzNote
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either
LZ4or Zstandard (ZSTD) compression algorithms.The resulting files have the
qpressarchive format. Every\\*.qpfile produced by xtrabackup is essentially a one-file qpress archive and can be extracted and uncompressed by theqpress file archiverwhich is available from Percona Software repositories.lz4To compress files using the
lz4compression algorithm, set--compressoption tolz4:$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the
Zstandard (ZSTD)compression algorithm.ZSTDis a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. To compress files using theZSTDcompression algorithm, use the--compress=zstdoption. The resulting files have the\\*.zstformat.You can specify
ZSTDcompression level with the--compress-zstd-level(=#)option. The defaul value is1.$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nTo decompress files, use the
--decompressoption.Using xbstream as a stream option, backups can be copied and compressed in parallel. This option can significantly improve the speed of the backup process. In case backups were both compressed and encrypted, they must be decrypted before they are uncompressed.
Task Command Stream the backup into an archived namedbackup.xbstream$ xtrabackup --backup --stream=xbstream --target-dir=./ > backup.xbstreamStream the backup into a compressed archive namedbackup.xbstream$ xtrabackup --backup --stream=xbstream --compress --target-dir=./ > backup.xbstreamEncrypt the backup$ xtrabackup --backup --stream=xbstream ./ > backup.xbstream gzip -`` | openssl des3 -salt -k \u201cpassword\u201d backup.xbstream.gz.des3Unpack the backup to the current directory$ xbstream -x < backup.xbstreamSend the backup compressed directly to another host and unpack it$ xtrabackup --backup --compress --stream=xbstream --target-dir=./ | ssh user@otherhost \"xbstream -x\"Send the backup to another server usingnetcatOn the destination host:$ nc -l 9999 | cat - > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | nc desthost 9999Send the backup to another server using a one-liner$ ssh user@desthost \u201c( nc -l 9999 > /data/backups/backup.xbstream & )\u201d && xtrabackup --backup --stream=xbstream ./ | nc desthost 9999Throttle the throughput to 10MB/sec using the pipe viewer tool$ xtrabackup --backup --stream=xbstream ./ | pv -q -L10m ssh user@desthost \u201ccat - > /data/backups/backup.xbstream\u201dChecksum the backup during the streaming On the destination host:$ nc -l 9999 | tee >(sha1sum > destination_checksum) > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | tee >(sha1sum > source_checksum) | nc desthost 9999Compare the checksums on the source host:$ cat source_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbadCompare the checksums on the destination host:$ cat destination_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbadParallel compression with parallel copying backup$ xtrabackup --backup --compress --compress-threads=8 --stream=xbstream --parallel=4 --target-dir=./ > backup.xbstreamImportant
The streamed backup must be prepared before restoration. Streaming mode does not prepare the backup.
"},{"location":"throttling-backups.html","title":"Throttling backups","text":"Although xtrabackup does not block your database\u2019s operation, any backup can add load to the system being backed up. On systems that do not have much spare I/O capacity, it might be helpful to throttle the rate at which xtrabackup reads and writes data. You can do this with the
--throttleoption. This option limits the number of chunks copied per second. The chunk +size is 10 MB.The image below shows how throttling works when
--throttleis set to1.When specified with the
--backupoption, this option limits the number of pairs of read-and-write operations per second that xtrabackup will perform. If you are creating an incremental backup, then the limit is the number of read I/O operations per second.By default, there is no throttling, and xtrabackup reads and writes data as quickly as it can. If you set too strict of a limit on the IOPS, the backup might be so slow that it will never catch up with the transaction logs that InnoDB is writing, so the backup might never complete.
"},{"location":"toolkit-version-check.html","title":"Percona Toolkit version checking","text":"Some Percona software contains \u201cversion checking\u201d functionality which is a feature that enables Percona software users to be notified of available software updates to improve your environment security and performance. Alongside this, the version check functionality also provides Percona with information relating to which software versions you are running, coupled with the environment confirmation which the software is running within. This helps enable Percona to focus our development effort accordingly based on trends within our customer community.
The purpose of this document is to articulate the information that is collected, as well as to provide guidance on how to disable this functionality if desired.
"},{"location":"toolkit-version-check.html#usage","title":"Usage","text":"Version Check was implemented in Percona Toolkit 2.1.4, and was enabled by default in version 2.2.1. Currently, it is supported as a
--[no]version-checkoption by a number of tools in Percona Toolkit, Percona XtraBackup, and Percona Monitoring and Management (PMM).When launched with Version Check enabled, the tool that supports this feature connects to a Percona\u2019s version check service via a secure HTTPS channel. It compares the locally installed version for possible updates, and also checks versions of the following software:
-
Operating System
-
Percona Monitoring and Management (PMM)
-
MySQL
-
Perl
-
MySQL driver for Perl (DBD::mysql)
-
Percona Toolkit
Then it checks for and warns about versions with known problems if they are identified as running in the environment.
Each version check request is logged by the server. Stored information consists of the checked system unique ID followed by the software name and version. The ID is generated either at installation or when the version checking query is submitted for the first time.
Note
Prior to version 3.0.7 of Percona Toolkit, the system ID was calculated as an MD5 hash of a hostname, and starting from Percona Toolkit 3.0.7 it is generated as an MD5 hash of a random number. Percona XtraBackup continues to use hostname-based MD5 hash.
As a result, the content of the sent query is as follows:
Expected output
"},{"location":"toolkit-version-check.html#disabling-version-check","title":"Disabling version check","text":"85624f3fb5d2af8816178ea1493ed41a;DBD::mysql;4.044\nc2b6d625ef3409164cbf8af4985c48d3;MySQL;MySQL Community Server (GPL) 5.7.22-log\n85624f3fb5d2af8816178ea1493ed41a;OS;Manjaro Linux\n85624f3fb5d2af8816178ea1493ed41a;Percona::Toolkit;3.0.11-dev\n85624f3fb5d2af8816178ea1493ed41a;Perl;5.26.2\nAlthough the version checking feature does not collect any personal information, you might prefer to disable this feature, either one time or permanently. To disable it one time, use
--no-version-checkoption when invoking the tool from a Percona product which supports it. Here is a simple example which shows running pt-diskstats tool from the Percona Toolkit with version checking turned off:pt-diskstats --no-version-check\nDisabling version checking permanently can be done by placing
no-version-checkoption into the configuration file of a Percona product (see correspondent documentation for exact file name and syntax). For example, in case of Percona Toolkit this can be done in a global configuration file/etc/percona-toolkit/percona-toolkit.conf:# Disable Version Check for all tools:\nno-version-check\nIn case of Percona XtraBackup this can be done in its configuration file in a similar way:
"},{"location":"toolkit-version-check.html#frequently-asked-questions","title":"Frequently asked questions","text":""},{"location":"toolkit-version-check.html#why-is-this-functionality-enabled-by-default","title":"Why is this functionality enabled by default?","text":"[xtrabackup]\nno-version-check\nWe believe having this functionality enabled improves security and performance of environments running Percona Software and it is good choice for majority of the users.
"},{"location":"toolkit-version-check.html#why-not-rely-on-operating-systems-built-in-functionality-for-software-updates","title":"Why not rely on Operating System\u2019s built in functionality for software updates?","text":"In many environments the Operating Systems repositories may not carry the latest version of software and newer versions of software often installed manually, so not being covered by operating system wide check for updates.
"},{"location":"toolkit-version-check.html#why-do-you-send-more-information-than-just-the-version-of-software-being-run-as-a-part-of-version-check-service","title":"Why do you send more information than just the version of software being run as a part of version check service?","text":"Compatibility problems can be caused by versions of various components in the environment, for example problematic versions of Perl, DBD or MySQL could cause operational problems with Percona Toolkit.
"},{"location":"trademark-policy.html","title":"Trademark policy","text":"This Trademark Policy is to ensure that users of Percona-branded products or services know that what they receive has really been developed, approved, tested and maintained by Percona. Trademarks help to prevent confusion in the marketplace, by distinguishing one company\u2019s or person\u2019s products and services from another\u2019s.
Percona owns a number of marks, including but not limited to Percona, XtraDB, Percona XtraDB, XtraBackup, Percona XtraBackup, Percona Server, and Percona Live, plus the distinctive visual icons and logos associated with these marks. Both the unregistered and registered marks of Percona are protected.
Use of any Percona trademark in the name, URL, or other identifying characteristic of any product, service, website, or other use is not permitted without Percona\u2019s written permission with the following three limited exceptions.
First, you may use the appropriate Percona mark when making a nominative fair use reference to a bona fide Percona product.
Second, when Percona has released a product under a version of the GNU General Public License (\u201cGPL\u201d), you may use the appropriate Percona mark when distributing a verbatim copy of that product in accordance with the terms and conditions of the GPL.
Third, you may use the appropriate Percona mark to refer to a distribution of GPL-released Percona software that has been modified with minor changes for the sole purpose of allowing the software to operate on an operating system or hardware platform for which Percona has not yet released the software, provided that those third party changes do not affect the behavior, functionality, features, design or performance of the software. Users who acquire this Percona-branded software receive substantially exact implementations of the Percona software.
Percona reserves the right to revoke this authorization at any time in its sole discretion. For example, if Percona believes that your modification is beyond the scope of the limited license granted in this Policy or that your use of the Percona mark is detrimental to Percona, Percona will revoke this authorization. Upon revocation, you must immediately cease using the applicable Percona mark. If you do not immediately cease using the Percona mark upon revocation, Percona may take action to protect its rights and interests in the Percona mark. Percona does not grant any license to use any Percona mark for any other modified versions of Percona software; such use will require our prior written permission.
Neither trademark law nor any of the exceptions set forth in this Trademark Policy permit you to truncate, modify or otherwise use any Percona mark as part of your own brand. For example, if XYZ creates a modified version of the Percona Server, XYZ may not brand that modification as \u201cXYZ Percona Server\u201d or \u201cPercona XYZ Server\u201d, even if that modification otherwise complies with the third exception noted above.
In all cases, you must comply with applicable law, the underlying license, and this Trademark Policy, as amended from time to time. For instance, any mention of Percona trademarks should include the full trademarked name, with proper spelling and capitalization, along with attribution of ownership to Percona LLC and/or its affiliates. For example, the full proper name for XtraBackup is Percona XtraBackup. However, it is acceptable to omit the word \u201cPercona\u201d for brevity on the second and subsequent uses, where such omission does not cause confusion.
In the event of doubt as to any of the conditions or exceptions outlined in this Trademark Policy, please contact trademarks@percona.com for assistance and we will do our very best to be helpful.
"},{"location":"update-curl-utility.html","title":"Update curl utility","text":""},{"location":"update-curl-utility.html#update-the-curl-utility-in-debian-10","title":"Update the curl utility in Debian 10","text":"The default curl version, 7.64.0, in Debian 10 has known issues when attempting to reuse an already closed connection. This issue directly affects
xbcloudand users may see intermittent backup failures.For more details, see curl #3750 or curl #3763.
Follow these steps to upgrade curl to version 7.74.0:
-
Edit the
/etc/apt/sources.listto add the following:deb http://ftp.de.debian.org/debian buster-backports main\n -
Refresh the
aptsources:$ sudo apt update\n -
Install the version from
buster-backports:$ sudo apt install curl/buster-backports\n -
Verify the version number:
Expected output$ curl --version\ncurl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0\n
One way to verify if the backup is consistent is by setting up the replication and running pt-table-checksum. This can be used to verify any type of backups, but before setting up replication, backup should be prepared and be able to run (this means that incremental backups should be merged to full backups, encrypted backups decrypted etc.).
"},{"location":"varify-backup.html#set-up-the-replication","title":"Set up the replication","text":"How to set up a replica for replication in 6 simple steps with Percona XtraBackup guide provides a detailed instructions on how to take the backup and set up the replication.
For checking the backup consistency you can use either the original server where the backup was taken, or another test server created by using a different backup method (such as cold backup, mysqldump or LVM snapshots) as the source server in the replication setup.
"},{"location":"varify-backup.html#use-pt-table-checksum","title":"Use pt-table-checksum","text":"This tool is part of the Percona Toolkit. It performs an online replication consistency check by executing checksum queries on the source, which produces different results on replicas that are inconsistent with the source.
After you confirmed that replication has been set up successfully, you can install or download pt-table-checksum. This example shows downloading the latest version of pt-table-checksum:
$ wget percona.com/get/pt-table-checksum\nNote
In order for pt-table-checksum to work correctly
libdbd-mysql-perlwill need to be installed on Debian/Ubuntu systems orperl-DBD-MySQLon RHEL/CentOS. If you installed the percona-toolkit package from the Percona repositories package manager should install those libraries automatically.After this command has been run, pt-table-checksum will be downloaded to your current working directory.
Running the pt-table-checksum on the source will create
perconadatabase with thechecksumstable which will be replicated to the replicas as well. Example of the pt-table-checksum will look like this:$ ./pt-table-checksum\nThe results are similar to the following:
Expected outputTS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE\n04-30T11:31:50 0 0 633135 8 0 5.400 exampledb.aka_name\n04-30T11:31:52 0 0 290859 1 0 2.692 exampledb.aka_title\nChecksumming exampledb.user_info: 16% 02:27 remain\nChecksumming exampledb.user_info: 34% 01:58 remain\nChecksumming exampledb.user_info: 50% 01:29 remain\nChecksumming exampledb.user_info: 68% 00:56 remain\nChecksumming exampledb.user_info: 86% 00:24 remain\n04-30T11:34:38 0 0 22187768 126 0 165.216 exampledb.user_info\n04-30T11:38:09 0 0 0 1 0 0.033 mysql.time_zone_name\n04-30T11:38:09 0 0 0 1 0 0.052 mysql.time_zone_transition\n04-30T11:38:09 0 0 0 1 0 0.054 mysql.time_zone_transition_type\n04-30T11:38:09 0 0 8 1 0 0.064 mysql.user\nIf all the values in the
DIFFScolumn are 0 that means that backup is consistent with the current setup.In case backup was not consistent pt-table-checksum should spot the difference and point to the table that does not match. Following example shows adding new user on the backed up replica in order to simulate the inconsistent backup:
mysql> grant usage on exampledb.* to exampledb@localhost identified by 'thisisnewpassword';\nIf we run the pt-table-checksum now difference should be spotted
$ ./pt-table-checksum\nThe results are similar to the following:
Expected outputTS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE\n04-30T11:31:50 0 0 633135 8 0 5.400 exampledb.aka_name\n04-30T11:31:52 0 0 290859 1 0 2.692 exampledb.aka_title\nChecksumming exampledb.user_info: 16% 02:27 remain\nChecksumming exampledb.user_info: 34% 01:58 remain\nChecksumming exampledb.user_info: 50% 01:29 remain\nChecksumming exampledb.user_info: 68% 00:56 remain\nChecksumming exampledb.user_info: 86% 00:24 remain\n04-30T11:34:38 0 0 22187768 126 0 165.216 exampledb.user_info\n04-30T11:38:09 0 0 0 1 0 0.033 mysql.time_zone_name\n04-30T11:38:09 0 0 0 1 0 0.052 mysql.time_zone_transition\n04-30T11:38:09 0 0 0 1 0 0.054 mysql.time_zone_transition_type\n04-30T11:38:09 1 0 8 1 0 0.064 mysql.user\nThis output shows that source and the replica aren\u2019t in consistent state and that the difference is in the
mysql.usertable.More information on different options that pt-table-checksum provides can be found in the pt-table-checksum documentation.
"},{"location":"work-with-apparmor.html","title":"Work with AppArmor","text":"The Linux Security Module implements mandatory access controls (MAC) with AppArmor. Debian and Ubuntu systems install AppArmor by default. AppArmor uses profiles which define which files and permissions are needed for application.
Percona XtraBackup does not have a profile and is not confined by AppArmor.
For a list of common AppArmor commands, see Percona Server for MySQL - AppArmor
"},{"location":"work-with-apparmor.html#develop-a-profile","title":"Develop a profile","text":"Download the profile from:
https://github.com/percona/percona-xtrabackup/tree/8.0/packaging/percona/apparmor/apparmor.d
The following profile sections should be updated with your system information, such as location of the backup destination directory.
Expected output# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\nMove the updated file:
$ sudo mv usr.sbin.xtrabackup /etc/apparmor.d/\nInstall the profile with the following command:
$ sudo apparmor_parser -r -T -W /etc/apparmor.d/usr.sbin.xtrabackup\nRun the backup as usual.
No additional AppArmor-related actions are required to restore a backup.
"},{"location":"work-with-selinux.html","title":"Work with SELinux","text":"Percona XtraBackup is installed as an unconfined process running in an undefined domain. SELinux allows unconfined processes almost all access and the processes only use Discretionary Access Control (DAC) rules.
You find the current state of the Percona XtraBackup file with the following command:
Expected output$ ls -Z /usr/bin | grep xtrabackup\n-rwxr-xr-x. root root system_u:object_r:bin_t:s0 xtrabackup\nThe SELinux context is the following:
-
user (root)
-
role (object_r)
-
type (bin_t)
-
level (s0)
The unconfined domain supports the network-facing services, which are protected by SELinux. These domains are not exposed. In this configuration, SELinux protects against remote intrusions but local intrusions, which require local access, are not confined.
Percona XtraBackup works locally. The service is not network-facing and cannot be exploited externally. The service interacts only with the local user, who provides the parameters. Percona XtraBackup requires access to the
"},{"location":"work-with-selinux.html#confine-xtrabackup","title":"Confine XtraBackup","text":"target-dirlocation.You can modify your security configuration to confine Percona XtraBackup. The first question is where to store the backup files. The service requires read and write access to the selected location.
You can use either of the following methods:
-
Allow Percona XtraBackup to write to any location. The user provides any path to the
target-dirparameter. -
Allow Percona XtraBackup to write to a specific location, such as /backups or the user\u2019s home directory.
The first option opens the entire system to read and write. Select the second option to harden your security.
"},{"location":"work-with-selinux.html#install-selinux-tools","title":"Install SELinux tools","text":"To work with policies, you must install the SELinux tools. To find which package provides the
semanagecommand and install the package. The following is an example on CentOS 7.
The result should list the packages. Expected output$ yum provides *bin/semanage\n...\npolicycoreutils-python-2.5-34.el7.x86_64 : SELinux policy core python utilities\n...\nTo install missing packages, run the following:
$ sudo yum install -y policycoreutils-python\nThe following is an example on CentOS 8:
The result should list the missing packages. Expected output$ yum provides *bin/semanage\n...\npolicycoreutils-python-utils-2.8-16.1.el8.noarch : SELinux policy core python utilities\nRun the following to install the missing packages:
"},{"location":"work-with-selinux.html#create-a-policy","title":"Create a policy","text":"$ sudo yum install -y policycoreutils-python-utils\nUse a modular approach to create an SELinux policy. Create a policy module to manage XtraBackup. You must create a
.tefile for type enforcement, and an optional.fcfile for the file contexts.Use
$ ps -efZ | grep xtrabackupto verify the service is not confined by SELinux.Create the
xtrabackup.fcfile and add content. This file defines the security contexts./usr/bin/xtrabackup -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcrypt -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbstream -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcloud -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/backups(/.*)? system_u:object_r:xtrabackup_data_t:s0\nNote
If you are using the
/backupsdirectory you must have the last line. If you are storing the backups in the user\u2019s home directory, you can omit this line.Download the
xtrabackup.tefile from the following location:https://github.com/percona/percona-xtrabackup/tree/8.0/packaging/percona/selinx
Note
In the file, the sections in bold should be modified for your system. The fc file can also be downloaded from the same location.
Compile the policy module:
$ make -f /usr/share/selinux/devel/Makefile xtrabackup.pp\nInstall the module:
$ semodule -i xtrabackup.pp\nTag the PXB binaries with the proper SELinux tags, such as
xtrabackup_exec_t.$ restorecon -v /usr/bin/*\nIf you store your backups at
/backups, restore the tag in that location:$ restorecon -v /backups\nNote
Remember to add the standard Linux DAC permissions for this directory.
Perform the backup in the standard way.
"},{"location":"working-with-binary-logs.html","title":"Work with binary logs","text":"The
"},{"location":"working-with-binary-logs.html#find-the-binary-log-position","title":"Find the binary log position","text":"xtrabackupbinary integrates with thelog_status table. This integration enablesxtrabackupto print out the backup\u2019s corresponding binary log position, so that you can use this binary log position to provision a new replica or perform point-in-time recovery.You can find the binary log position corresponding to a backup after the backup has been taken. If your backup is from a server with binary logging enabled,
Expected output during the backup stagextrabackupcreates a file namedxtrabackup_binlog_infoin the target directory. This file contains the binary log file name and position of the exact point when the backup was taken.210715 14:14:59 Backup created in directory '/backup/'\nMySQL binlog position: filename 'binlog.000002', position '156'\n. . .\n210715 14:15:00 completed OK!\nNote
As of Percona XtraBackup 8.0.26-18.0, xtrabackup no longer creates the
"},{"location":"working-with-binary-logs.html#point-in-time-recovery","title":"Point-in-time recovery","text":"xtrabackup_binlog_pos_innodbfile. This change is because MySQL and Percona Server no longer update the binary log information on global transaction system section ofibdata. You should rely onxtrabackup_binlog_inforegardless of the storage engine in use.To perform a point-in-time recovery from an
xtrabackupbackup, you should prepare and restore the backup, and then replay binary logs from the point shown in thextrabackup_binlog_infofile.A more detailed procedure is found here.
"},{"location":"working-with-binary-logs.html#set-up-a-new-replication-replica","title":"Set up a new replication replica","text":"To set up a new replica, you should prepare the backup, and restore it to the data directory of your new replication replica. If you are using version 8.0.22 or earlier, in your
CHANGE MASTER TOcommand, use the binary log filename and position shown in thextrabackup_binlog_infofile to start replication.If you are using 8.0.23 or later, use the CHANGE_REPLICATION_SOURCE_TO and the appropriate options.
CHANGE_MASTER_TOis deprecated.A more detailed procedure is found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.
"},{"location":"xbcloud-azure.html","title":"Use the xbcloud binary with Microsoft Azure Cloud Storage","text":"Implemented in Percona XtraBackup 8.0.27-19, the xbcloud binary adds support for the Microsoft Azure Cloud Storage using the REST API.
"},{"location":"xbcloud-azure.html#options","title":"Options","text":"The following are the options, environment variables, and descriptions for uploading a backup to Azure using the REST API. The environment variables are recognized by xbcloud, which maps them automatically to the corresponding parameters:
Option name Environment variables Description --azure-storage-account=name AZURE_STORAGE_ACCOUNT An Azure storage account is a unique namespace to access and store your Azure data objects. --azure-container-name=name AZURE_CONTAINER_NAME A container name is a valid DNS name that conforms to the Azure naming rules --azure-access-key=name AZURE_ACCESS_KEY A generated key that can be used to authorize access to data in your account using the Shared Key authorization. --azure-endpoint=name AZURE_ENDPOINT The endpoint allows clients to securely access data --azure-tier-class=name AZURE_STORAGE_CLASS Cloud tier can decrease the local storage required while maintaining the performance. When enabled, this feature has the following categories: Hot - Frequently accessed or modified data Cool - Infrequently accessed or modified data Archive - Rarely accessed or modified dataTest your Azure applications with the Azurite open-source emulator. For testing purposes, the xbcloud binary adds the
"},{"location":"xbcloud-azure.html#usage","title":"Usage","text":"--azure-development-storageoption that uses the defaultaccess_keyandstorage accountof azurite andtestcontainerfor the container. You can overwrite these options, if needed.All the available options for xbcloud, such as parallel, max-retries, and others, can be used. For more information, see the xbcloud binary overview.
"},{"location":"xbcloud-azure.html#examples","title":"Examples","text":"An example of an xbcloud backup.
$ xtrabackup --backup --stream=xbstream | \nxbcloud put backup_name --azure-storage-account=pxbtesting --azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\nAn example of restoring a backup from xbcloud.
$ xbcloud get backup_name --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure --parallel=10 2>download.log | xbstream -x -C restore\nAn example of deleting a backup from xbcloud.
$ xbcloud delete backup_name --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\nAn example of using a shortcut restore.
"},{"location":"xbcloud-binary-fifo-datasink.html","title":"FIFO data sink","text":"$ xbcloud get azure://operator-testing/bak22 ...\nThe feature is in tech preview.
Percona XtraBackup 8.0.33-28 implements a new data sink that uses the first in, first out (FIFO) method. With the
"},{"location":"xbcloud-binary-fifo-datasink.html#fifo-data-sink-options","title":"FIFO data sink options","text":"FIFOdata sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to an object storage.Percona XtraBackup 8.0.33-28 implements the following options:
--fifo-streams=#- specifies the number of FIFO files to use for parallel data stream. To disable FIFO data sink and send stream to STDOUT, set--fifo-streams=1. The default value is1(STDOUT) to ensure the backward compatibility. The--fifo-streamsvalue must match on both the XtraBackup and xbcloud sides.--fifo-dir=name- specifies a directory to write Named Pipe.--fifo-timeout=#- specifies the number of seconds to wait for the other end to open the stream for reading. The default value is60seconds.
To use FIFO data sink, you can either run two commands in separate terminal sessions or run xtrabackup in the background.
For example, run the following commands in separate terminal sessions:
$ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo\n$ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\nRun xtrabackup in the background with the following commands:
"},{"location":"xbcloud-binary-fifo-datasink.html#stream-to-an-object-storage","title":"Stream to an object storage","text":"$ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo &\n$ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\nWhen taking a backup, you can save the files locally or stream the files to either a different server or an object storage.
Before Percona XtraBackup 8.0.33-28, XtraBackup streams data to an object storage writing to STDOUT (a pipe) and using
xbcloudto read from STDIN (standard input device).XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then each copy thread writes the data chunks to a pipe (STDOUT). An xbcloud read thread reads each chunk of STDIN data. Then xbcloud uploads the chunks to an object storage using an async request, and adds a callback to an event handler list. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).
The streaming capacity for XtraBackup using STDOUT is 1.8G. This capacity is sufficient for streaming data using Wide Area Network (WAN) into, for example, Amazon Web Services or Google Cloud Platform.
Starting with Percona XtraBackup 8.0.33-28, when you stream backups to Amazon S3 compatible storage using LAN with a streaming capacity of 10Gbps, XtraBackup can use multiple FIFO streams to stream the backups faster.
XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then multiple FIFO files are created to store the data from XtraBackup. Each XtraBackup copy thread writes the data chunks to a specific FIFO file. Xbcloud reads from the FIFO streams and uploads data to an object storage using an async request. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).
"},{"location":"xbcloud-binary-fifo-datasink.html#performance-improvement","title":"Performance improvement","text":"Consider an example of using a FIFO data sink compared to the STDOUT method.
The database has 1TB of data in multiple tables. The link speed between the source server and destination server using MinIO is ~ 9.2 Gbps.
Both STDOUT and FIFO data sink scenarios push 1TB of data from two servers.
For the FIFO data sink we configure 8 parallel streams with
--fifo-streams=8both for XtraBackup and xbcloud.The results are the following:
- The
STDOUTmethod takes 01:25:24 to push 1TB of data using 239 MBps (1.8 Gbps). - The
FIFOmethod, using 8 streams, takes 00:16:01 to push 1TB of data using 1.15 GBps (9.2 Gbps).
The purpose of xbcloud is to download from the cloud and upload to the cloud the full or part of an xbstream archive. xbcloud will not overwrite the backup with the same name. xbcloud accepts input via a pipe from xbstream so that it can be invoked as a pipeline with xtrabackup to stream directly to the cloud without needing a local storage.
Note
In a Bash shell, the
$?parameter returns the exit code from the last binary. If you use pipes, the ${PIPESTATUS[x]} array parameter returns the exit code for each binary in the pipe string.$ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ | xbcloud put [options] full_backup\n...\n$ ${PIPESTATUS[x]}\n0 0\n$ true | false\n$ echo $?\n1\n\n# with PIPESTATUS\n$ true | false\n$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}\n0 1\nThe xbcloud binary stores each chunk as a separate object with a name
backup_name/database/table.ibd.NNN..., whereNNN...is a 0-padded serial number of chunk within a file. The size of chunk produced by xtrabackup and xbstream changed to 10M.To adjust the chunk size use
--read-buffer-size. To adjust the chunk size for encrypted files, use--read-buffer-sizeand--encrypt-chunk-size.xbcloud has three essential operations: put, get, and delete. With these operations, backups are created, stored, retrieved, restored, and deleted. xbcloud operations clearly map to similar operations within the AWS Amazon S3 API.
The Exponential Backoff feature was implemented in Percona XtraBackup 8.0.26-18. During the backup, a chunk fails to upload or download operation; this feature adds an exponential backoff, a sleep time, and retries the operation. This feature increases the chances of completing a backup or a restore operation.
The FIFO datasink feature was implemented in Percona XtraBackup 8.0.33-28. With the
FIFOdata sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to object storage.Important
To prevent intermittent backup failures, update the curl utility in Debian 10.
"},{"location":"xbcloud-binary-overview.html#supported-cloud-storage-types","title":"Supported cloud storage types","text":"The following cloud storage types are supported:
-
OpenStack Object Storage (Swift) - see Using the xbcloud binary with Swift
-
Amazon Simple Storage (S3) - see Using the xbcloud binary with Amazon S3
-
Azure Cloud Storage - see Using the xbcloud binary with Microsoft Azure Cloud Storage
-
Google Cloud Storage (gcs) - see Using the xbcloud binary with Google Cloud Storage
-
MinIO - see Using the xbcloud binary with MinIO
In addition to OpenStack Object Storage (Swift), which has been the only option for storing backups in a cloud storage until Percona XtraBackup 2.4.14, xbcloud supports Amazon S3, MinIO, and Google Cloud Storage. Other Amazon S3-compatible storages, such as Wasabi or Digital Ocean Spaces, are also supported.
See also
OpenStack Object Storage(\u201cSwift\u201d)
Amazon Simple Storage Service
MinIO
Google Cloud Storage
Wasabi
Digital Ocean Spaces
"},{"location":"xbcloud-binary-overview.html#usage","title":"Usage","text":"The following sample command creates a full backup:
$ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ --extra-lsndirk=/storage/backups/| xbcloud \\\nput [options] full_backup\nAn incremental backup only includes the changes since the last backup. The last backup can be either a full or incremental backup.
The following sample command creates an incremental backup:
$ xtrabackup --backup --stream=xbstream --incremental-basedir=/storage/backups \\\n--target-dir=/storage/inc-backup | xbcloud put [options] inc_backup\nTo prepare an incremental backup, you must first download the full backup with the following command:
$ xtrabackup get [options] full_backup | xbstream -xv -C /tmp/full-backup\nYou must prepare the full backup:
$ xtrabackup --prepare --apply-log-only --target-dir=/tmp/full-backup\nAfter the full backup has been prepared, download the incremental backup:
xbcloud get [options] inc_backup | xbstream -xv -C /tmp/inc-backup\nThe downloaded backup is prepared by running the following command:
$ xtrabackup --prepare --target-dir=/tmp/full-backup --incremental-dir=/tmp/inc-backup\nYou do not need the full backup to restore only a specific database. You can specify only the tables to be restored:
An example of the code:xbcloud get [options] ibdata1 sakila/payment.ibd /tmp/partial/partial.xbs\n
"},{"location":"xbcloud-binary-overview.html#supplying-parameters","title":"Supplying parameters","text":"xbstream -xv -C /tmp/partial < /tmp/partial/partial.xbs\nEach storage type has mandatory parameters that you can supply on the command line, in a configuration file, and via environment variables.
"},{"location":"xbcloud-binary-overview.html#configuration-files","title":"Configuration files","text":"The parameters the values of which do not change frequently can be stored in
my.cnfor in a custom configuration file. The following example is a template of configuration options under the[xbcloud]group:[xbcloud]\nstorage=s3\ns3-endpoint=http://localhost:9000/\ns3-access-key=minio\ns3-secret-key=minio123\ns3-bucket=backupsx\ns3-bucket-lookup=path\ns3-api-version=4\nNote
If you explicitly use a parameter on the command line and in a configuration file, xbcloud uses the value provided on the command line.
"},{"location":"xbcloud-binary-overview.html#environment-variables","title":"Environment variables","text":"If you explicitly use a parameter on the command line, in a configuration file, and the corresponding environment variable contains a value, xbcloud uses the value provided on the command line or in the configuration file.
"},{"location":"xbcloud-binary-overview.html#shortcuts","title":"Shortcuts","text":"For all operations (put, get, and delete), you can use a shortcut to specify the storage type, bucket name, and backup name as one parameter instead of using three distinct parameters (\u2013storage, \u2013s3-bucket, and backup name per se).
Note
Use the following format:
storage-type://bucket-name/backup-nameIn this example s3 refers to a storage type, operator-testing is a bucket name, and bak22 is the backup name.
$ xbcloud get s3://operator-testing/bak22 ...\nThis shortcut expands as follows:
$ xbcloud get --storage=s3 --s3-bucket=operator-testing bak22 ...\nYou can supply the mandatory parameters on the command line, configuration files, and in environment variables.
"},{"location":"xbcloud-binary-overview.html#additional-parameters","title":"Additional parameters","text":"xbcloud accepts additional parameters that you can use with any storage type. The
--md5parameter computes the MD5 hash value of the backup chunks. The result is stored in files that following thebackup_name.md5pattern.$ xtrabackup --backup --stream=xbstream \\\n--parallel=8 2>backup.log | xbcloud put s3://operator-testing/bak22 \\\n--parallel=8 --md5 2>upload.log\nYou may use the
--headerparameter to pass an additional HTTP header with the server side encryption while specifying a customer key.An example of using the
--headerfor AES256 encryption.$ xtrabackup --backup --stream=xbstream --parallel=4 | \\\nxbcloud put s3://operator-testing/bak-enc/ \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key: CuStoMerKey=\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key-MD5: CuStoMerKeyMd5==\" \\\n--parallel=8\nThe
"},{"location":"xbcloud-binary-overview.html#incremental-backups","title":"Incremental backups","text":"--headerparameter is also useful to set the access control list (ACL) permissions:--header=\"x-amz-acl: bucket-owner-full-controlFirst, you need to make the full backup on which the incremental one is going to be based:
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/storage/backups/ \\\n--target-dir=/storage/backups/ | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup\nThen you can make the incremental backup:
"},{"location":"xbcloud-binary-overview.html#preparing-incremental-backups","title":"Preparing incremental backups","text":"$ xtrabackup --backup --incremental-basedir=/storage/backups \\\n--stream=xbstream --target-dir=/storage/inc_backup | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup\nTo prepare a backup you first need to download the full backup:
$ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup | xbstream -xv -C /storage/downloaded_full\nOnce you download the full backup it should be prepared:
$ xtrabackup --prepare --apply-log-only --target-dir=/storage/downloaded_full\nAfter the full backup has been prepared you can download the incremental backup:
$ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup | xbstream -xv -C /storage/downloaded_inc\nOnce the incremental backup has been downloaded you can prepare it by running:
"},{"location":"xbcloud-binary-overview.html#partial-download-of-the-cloud-backup","title":"Partial download of the cloud backup","text":"$ xtrabackup --prepare --apply-log-only \\\n--target-dir=/storage/downloaded_full \\\n--incremental-dir=/storage/downloaded_inc\n\n$ xtrabackup --prepare --target-dir=/storage/downloaded_full\nIf you do not want to download the entire backup to restore the specific database you can specify only the tables you want to restore:
"},{"location":"xbcloud-exbackoff.html","title":"Exponential backoff","text":"$ xbcloud get --swift-container=test_backup\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ full_backup \\\nibdata1 sakila/payment.ibd \\\n> /storage/partial/partial.xbs\n\n$ xbstream -xv -C /storage/partial < /storage/partial/partial.xbs\nThis feature was implemented in Percona XtraBackup 8.0.26-18.0 in the xbcloud binary.
Exponential backoff increases the chances for the completion of a backup or a restore operation. For example, a chunk upload or download may fail if you have an unstable network connection or other network issues. This feature adds an exponential backoff, or sleep, time and then retries the upload or download.
When a chunk upload or download operation fails, xbcloud checks the reason for the failure. This failure can be a CURL error or an HTTP error, or a client-specific error. If the error is listed in the Retriable errors list, xbcloud pauses for a calculated time before retrying the operation until that time reaches the
--max-backoffvalue.The operation is retried until the
--max-retriesvalue is reached. If the chunk operation fails on the last retry, xbcloud aborts the process.The default values are the following:
-
\u2013max-backoff = 300000 (5 minutes)
-
\u2013max-retries = 10
You can adjust the number of retries by adding the
--max-retriesparameter and adjust the maximum length of time between retries by adding the--max-backoffparameter to an xbcloud command.Since xbcloud does multiple asynchronous requests in parallel, a calculated value, measured in milliseconds, is used for
"},{"location":"xbcloud-exbackoff.html#retriable-errors","title":"Retriable errors","text":"max-backoff. This algorithm calculates how many milliseconds to sleep before the next retry. A number generated is based on the combining the power of two (2), the number of retries already attempted and adds a random number between 1 and 1000. This number avoids network congestion if multiple chunks have the same backoff value. If the default values are used, the final retry attempt to be approximately 17 minutes after the first try. The number is no longer calculated when the milliseconds reach the--max-backoffsetting. At that point, the retries continue by using the--max-backoffsetting until themax-retriesparameter is reached.We retry for the following CURL operations:
-
CURLE_GOT_NOTHING
-
CURLE_OPERATION_TIMEOUT
-
CURLE_RECV_ERROR
-
CURLE_SEND_ERROR
-
CURLE_SEND_FAIL_REWIND
-
CURLE_PARTIAL_FILE
-
CURLE_SSL_CONNECT_ERROR
We retry for the following HTTP operation status codes:
-
503
-
500
-
504
-
408
Each cloud provider may return a different CURL error or an HTTP error, depending on the issue. Add new errors by setting the following variables
--curl-retriable-errorsor--http-retriable-errorson the command line or inmy.cnfor in a custom configuration file under the [xbcloud] section. You can add multiple errors using a comma-separated list of codes.The error handling is enhanced when using the
--verboseoutput. This output specifies which error caused xbcloud to fail and what parameter a user must add to retry on this error.The following is an example of a verbose output:
Expected output
"},{"location":"xbcloud-exbackoff.html#example","title":"Example","text":"210701 14:34:23 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210701 14:34:23 /work/pxb/ins/8.0/bin/xbcloud: Curl error (52) Server returned nothing (no headers, no data) is not configured as retriable. You can allow it by adding --curl-retriable-errors=52 parameter\nThe following example adjusts the maximum number of retries and the maximum time between retries.
xbcloud [options] --max-retries=5 --max-backoff=10000\nThe following list describes the process using
--max-backoff=10000:-
The chunk
xtrabackup_logfile.00000000000000000006fails to upload the first time and sleeps for 2384 milliseconds. -
The same chunk fails for the second time and the sleep time is increased to 4387 milliseconds.
-
The same chunk fails for the third time and the sleep time is increased to 8691 milliseconds.
-
The same chunk fails for the fourth time. The
max-backoffparameter has been reached. All retries sleep the same amount of time after reaching the parameter. -
The same chunk is successfully uploaded.
"},{"location":"xbcloud-gcs.html","title":"Use the xbcloud binary with Google Cloud Storage","text":""},{"location":"xbcloud-gcs.html#create-a-full-backup-with-google-cloud-storage","title":"Create a full backup with Google Cloud Storage","text":"210702 10:07:05 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:05 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 2384 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:23 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:23 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 4387 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:52 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:07:52 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 8691 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:08:47 /work/pxb/ins/8.0/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:08:47 /work/pxb/ins/8.0/bin/xbcloud: Sleeping for 10000 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:10:12 /work/pxb/ins/8.0/bin/xbcloud: successfully uploaded chunk: backup3/xtrabackup_logfile.00000000000000000006, size: 8388660\nThe support for Google Cloud Storage is implemented using the interoperability mode. This mode was especially designed to interact with cloud services compatible with Amazon S3.
See also
Cloud Storage Interoperability
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=google \\\n--google-endpoint=`storage.googleapis.com` \\\n--google-access-key='YOUR-ACCESSKEYID' \\\n--google-secret-key='YOUR-SECRETACCESSKEY' \\\n--google-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\nThe following options are available when using Google Cloud Storage:
-
\u2013google-access-key =
-
\u2013google-secret-key =
-
\u2013google-bucket =
-
\u2013google-storage-class=name
-
STANDARD
-
NEARLINE
-
COLDLINE
-
ARCHIVE
- Who am I?
- What can I do?
Note
The Google storage class name options are the following:
See also
Google storage classes - the default Google storage class depends on the storage class of the bucket
"},{"location":"xbcloud-iam-profile.html","title":"Use the xbcloud binary with an IAM instance profile","text":"This feature is tech preview. Before using this feature in production, we recommend that you test restoring from physical backups in your environment and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.31-24 adds the ability to use the IAM instance profile when running xbcloud from an EC2 instance.
An authentication system has two elements:
A role defines \u201cwhat can I do.\u201d A role provides a method to define a collection of permissions. Roles are assigned to users, services and EC2 instances, the \u201cwho am I\u201d element.
The IAM instance profile is the \u201cwho\u201d for an EC2 instance and assumes the IAM role, which has permissions. The instance profile has the same name as the IAM role.
An IAM instance profile does not need the
--s3-secret-keynor the--s3-access-keyif they are runningxbcloudfrom an Amazon EC2 instance. To configure or attach an instance metadata to an EC2 instance, see How can I grant my Amazon EC2 instance access to an Amazon S3 bucket.An example of the command:
$ xtrabackup ... | xbcloud put --storage=s3 --s3-bucket=bucket-name backup-name\nThe xbcloud binary outputs a connect message when successful.
Expected output221121 13:16:26 Using instance metadata for access and secret key\n221121 13:16:26 xbcloud: Successfully connected.\nAn important consideration is that the instance metadata has a time to live (TTL) of 6 hours. A backup that takes more than that time contains Expired token errors. Use Exponential Backoff to retry the upload after fetching new keys from the instance metadata.
Output when keys have expired
"},{"location":"xbcloud-minio.html","title":"Use the xbcloud binary with MinIO","text":""},{"location":"xbcloud-minio.html#create-a-full-backup-with-minio","title":"Create a full backup with MinIO","text":"221121 13:04:52 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:52 xbcloud: Sleeping for 2384 ms before retrying test/mysql.ibd.00000000000000000002 [1]\n221121 13:04:55 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:55 xbcloud: Sleeping for 2887 ms before retrying test/mysql.ibd.00000000000000000003 [1]\n221121 13:04:58 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:58 xbcloud: Sleeping for 2778 ms before retrying test/undo_002.00000000000000000000 [1]\n221121 13:05:00 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:00 xbcloud: Sleeping for 2916 ms before retrying test/undo_002.00000000000000000001 [1]\n221121 13:05:03 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:03 xbcloud: Sleeping for 2794 ms before retrying test/undo_002.00000000000000000002 [1]\n221121 13:05:06 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:06 xbcloud: Sleeping for 2336 ms before retrying test/undo_001.00000000000000000000 [1]\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000002, size: 5242923\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000003, size: 23\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000000, size: 10485802\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000001, size: 6291498\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000002, size: 22\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000000, size: 10485802\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000001, size: 6291498\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000002, size: 22\n. . .\n221121 13:05:18 xbcloud: successfully uploaded chunk: test/xtrabackup_tablespaces.00000000000000000001, size: 36\n221121 13:05:19 xbcloud: Upload completed. \n
"},{"location":"xbcloud-s3.html","title":"Use the xbcloud binary with Amazon S3","text":""},{"location":"xbcloud-s3.html#create-a-full-backup-with-amazon-s3","title":"Create a full backup with Amazon S3","text":"$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='play.minio.io:9000' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='s3.amazonaws.com' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\nThe following options are available when using Amazon S3:
Option Details \u2013s3-access-key Use to supply the AWS access key ID \u2013s3-secret-key Use to supply the AWS secret access key \u2013s3-bucket Use supply the AWS bucket name \u2013s3-region Use to specify the AWS region. The default value is us-east-1 \u2013s3-api-version = Select the signing algorithm. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-bucket-lookup = Specify whether to use bucket.endpoint.com or endpoint.com/bucket*style requests. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-storage-class= Specify the S3 storage class. The default storage class depends on the provider. The name options are the following:- STANDARD
- STANDARD_IA
- GLACIER
Following the principle of \u201cleast-privilege\u201d, these are the minimum bucket permissions needed for xbcloud to write backups to S3: LIST/PUT/GET/DELETE.
The following example shows the policy definition for writing to the
xbcloud-testingbucket on the AWS S3 storage.
"},{"location":"xbcloud-s3.html#environment-variables","title":"Environment variables","text":"{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": \"arn:aws:s3:::xbcloud-testing\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:PutObject\",\n\"s3:PutObjectAcl\",\n\"s3:GetObject\",\n\"s3:GetObjectAcl\",\n\"s3:DeleteObject\"\n],\n\"Resource\": \"arn:aws:s3:::xbcloud-testing/*\"\n}\n]\n}\nThe following environment variables are recognized. xbcloud maps them automatically to corresponding parameters applicable to the selected storage.
-
AWS_ACCESS_KEY_ID (or ACCESS_KEY_ID)
-
AWS_SECRET_ACCESS_KEY (or SECRET_ACCESS_KEY)
-
AWS_DEFAULT_REGION (or DEFAULT_REGION)
-
AWS_ENDPOINT (or ENDPOINT)
-
AWS_CA_BUNDLE
"},{"location":"xbcloud-swift.html","title":"Use the xbcloud binary with Swift","text":""},{"location":"xbcloud-swift.html#create-a-full-backup-with-swift","title":"Create a full backup with Swift","text":"$ xbcloud get s3://operator-testing/bak22 \\\n--s3-endpoint=https://storage.googleapis.com/ \\\n--parallel=10 2>download.log | xbstream -x -C restore --parallel=8\nThe following example shows how to make a full backup and upload it to Swift.
$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\n--parallel=10 \\\nfull_backup\nThe following OpenStack environment variables are also recognized and mapped automatically to the corresponding swift parameters (
--storage=swift):-
OS_AUTH_URL
-
OS_TENANT_NAME
-
OS_TENANT_ID
-
OS_USERNAME
-
OS_PASSWORD
-
OS_USER_DOMAIN
-
OS_USER_DOMAIN_ID
-
OS_PROJECT_DOMAIN
-
OS_PROJECT_DOMAIN_ID
-
OS_REGION_NAME
-
OS_STORAGE_URL
-
OS_CACERT
$ xbcloud get [options] <name> [<list-of-files>] | xbstream -x\nThe following example shows how to fetch and restore the backup from Swift:
"},{"location":"xbcloud-swift.html#command-line-options","title":"Command-line options","text":"$ xbcloud get --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\nfull_backup | xbstream -xv -C /tmp/downloaded_full\n\n$ xbcloud delete --storage=swift --swift-user=xtrabackup \\\n--swift-password=xtrabackup123! --swift-auth-version=3 \\\n--swift-auth-url=http://openstack.ci.percona.com:5000/ \\\n--swift-container=mybackup1 --swift-domain=Default\nxbcloud has the following command line options:
"},{"location":"xbcloud-swift.html#-storageswifts3google","title":"--storage(=[swift|s3|google])","text":"Cloud storage option. xbcloud supports Swift, MinIO, and AWS S3. The default value is
"},{"location":"xbcloud-swift.html#-swift-auth-url","title":"--swift-auth-url()","text":"swift.The URL of the Swift cluster
"},{"location":"xbcloud-swift.html#-swift-storage-url","title":"--swift-storage-url()","text":"The xbcloud tries to get the object-store URL for a given region (if any are specified) from the keystone response. You can override that URL by passing \u2013swift-storage-url=URL argument.
"},{"location":"xbcloud-swift.html#-swift-user","title":"--swift-user()","text":"The Swift username (X-Auth-User, specific to Swift)
"},{"location":"xbcloud-swift.html#-swift-key","title":"--swift-key()","text":"The Swift key/password (X-Auth-Key, specific to Swift)
"},{"location":"xbcloud-swift.html#-swift-container","title":"--swift-container()","text":"The container to back up into (specific to Swift)
"},{"location":"xbcloud-swift.html#-paralleln","title":"--parallel(=N)","text":"The maximum number of concurrent upload/download requests. The default value is
"},{"location":"xbcloud-swift.html#-cacert","title":"--cacert()","text":"1.The path to the file with CA certificates
"},{"location":"xbcloud-swift.html#-insecure","title":"--insecure()","text":"Do not verify server\u2019s certificate
"},{"location":"xbcloud-swift.html#swift-authentication-options","title":"Swift authentication options","text":"The Swift specification describes several authentication options. The xbcloud tool can authenticate against keystone with API version 2 and 3.
"},{"location":"xbcloud-swift.html#-swift-auth-version","title":"--swift-auth-version()","text":"Specifies the swift authentication version. The possible values are:
1.0- TempAuth,2.0- Keystone v2.0, and3- Keystone v3. The default value is1.0.For v2 additional options are:
"},{"location":"xbcloud-swift.html#-swift-tenant","title":"--swift-tenant()","text":"Swift tenant name
"},{"location":"xbcloud-swift.html#-swift-tenant-id","title":"--swift-tenant-id()","text":"Swift tenant ID
"},{"location":"xbcloud-swift.html#-swift-region","title":"--swift-region()","text":"Swift endpoint region
"},{"location":"xbcloud-swift.html#-swift-password","title":"--swift-password()","text":"Swift password for the user
For v3 additional options are:
"},{"location":"xbcloud-swift.html#-swift-user-id","title":"--swift-user-id()","text":"Swift user ID
"},{"location":"xbcloud-swift.html#-swift-project","title":"--swift-project()","text":"Swift project name
"},{"location":"xbcloud-swift.html#-swift-project-id","title":"--swift-project-id()","text":"Swift project ID
"},{"location":"xbcloud-swift.html#-swift-domain","title":"--swift-domain()","text":"Swift domain name
"},{"location":"xbcloud-swift.html#-swift-domain-id","title":"--swift-domain-id()","text":"Swift domain ID
"},{"location":"xbcrypt-binary-overview.html","title":"The xbcrypt binary overview","text":"To support encryption and decryption of the backups, a new tool
xbcryptwas introduced to Percona XtraBackup.Percona XtraBackup 8.0.28-20 implements the XBCRYPT_ENCRYPTION_KEY environment variable. The variable is only used in place of the
--encrypt_key=nameoption. You can use the environment variable or command line option. If you use both, the command line option takes precedence over the value specified in the environment variable.This utility has been modeled after The xbstream binary to perform encryption and decryption outside of Percona XtraBackup.
"},{"location":"xbcrypt-binary-overview.html#-d-decrypt","title":"-d(, --decrypt()","text":"xbcrypthas following command line options:Decrypt data input to output.
"},{"location":"xbcrypt-binary-overview.html#-i-inputname","title":"-i(, --input(=name)","text":"Optional input file. If not specified, input will be read from standard input.
"},{"location":"xbcrypt-binary-overview.html#-o-outputname","title":"-o(, --output(=name)","text":"Optional output file. If not specified, output will be written to standard output.
"},{"location":"xbcrypt-binary-overview.html#-a-encrypt-algoname","title":"-a(, --encrypt-algo(=name)","text":"Encryption algorithm.
"},{"location":"xbcrypt-binary-overview.html#-k-encrypt-keyname","title":"-k(, --encrypt-key(=name)","text":"Encryption key.
"},{"location":"xbcrypt-binary-overview.html#-f-encrypt-key-filename","title":"-f(, --encrypt-key-file(=name)","text":"File which contains encryption key.
"},{"location":"xbcrypt-binary-overview.html#-s-encrypt-chunk-size","title":"-s(, --encrypt-chunk-size(=#)","text":"Size of working buffer for encryption in bytes. The default value is 64K.
"},{"location":"xbcrypt-binary-overview.html#-encrypt-threads","title":"--encrypt-threads(=#)","text":"This option specifies the number of worker threads that will be used for parallel encryption/decryption.
"},{"location":"xbcrypt-binary-overview.html#-v-verbose","title":"-v(, --verbose()","text":"Display verbose status output.
"},{"location":"xbstream-binary-overview.html","title":"The xbstream binary overview","text":"To support simultaneous compression and streaming, a new custom streaming format called xbstream was introduced to Percona XtraBackup in addition to the TAR format. That was required to overcome some limitations of traditional archive formats such as tar, cpio and others which did not allow streaming dynamically generated files, for example dynamically compressed files. Other advantages of xbstream over traditional streaming/archive format include ability to stream multiple files concurrently (so it is possible to use streaming in the xbstream format together with the \u2013parallel option) and more compact data storage.
This utility has a tar-like interface:
-
with the
-xoption it extracts files from the stream read from its standard input to the current directory unless specified otherwise with the-coption. Support for parallel extraction with the--paralleloption has been implemented in Percona XtraBackup 2.4.7. -
with the
-coption it streams files specified on the command line to its standard output. -
with the
--decrypt=ALGOoption specified xbstream will automatically decrypt encrypted files when extracting input stream. Supported values for this option are:AES128,AES192, andAES256. Either--encrypt-keyor--encrypt-key-fileoptions must be specified to provide encryption key, but not both. This option has been implemented in Percona XtraBackup 2.4.7. -
with the
--encrypt-threadsoption you can specify the number of threads for parallel data encryption. The default value is1. This option has been implemented in Percona XtraBackup 2.4.7. -
the
--encrypt-keyoption is used to specify the encryption key that will be used. It can\u2019t be used with--encrypt-key-fileoption because they are mutually exclusive. This option has been implemented in Percona XtraBackup 2.4.7. -
the
--encrypt-key-fileoption is used to specify the file that contains the encryption key. It can\u2019t be used with--encrypt-keyoption. because they are mutually exclusive. This option has been implemented in Percona XtraBackup 2.4.7.
The utility also tries to minimize its impact on the OS page cache by using the appropriate
posix_fadvise()calls when available.When compression is enabled with xtrabackup all data is being compressed, including the transaction log file and meta data files, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:
quicklzNote
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either
LZ4or Zstandard (ZSTD) compression algorithms.The resulting files have the qpress archive format. Every
\\*.qpfile produced by xtrabackup is essentially a one-file qpress archive and can be extracted and uncompressed by theqpress file archiver. This means that there is no need to decompress entire backup to restore a single table as withtar.gz.To decompress individual files, run xbstream with the
--decompressoption. You may control the number of threads used for decompressing by passing the--decompress-threadsoption.Also, files can be decompressed using the qpress tool. The tool supports multi-threaded decompression.
lz4To compress files using the
lz4compression algorithm, set--compressoption tolz4:$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the
Zstandard (ZSTD)compression algorithm.ZSTDis a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.To compress files using the
ZSTDcompression algorithm, use the--compress=zstdoption. The resulting files have the\\*.zstformat.You can specify
ZSTDcompression level with the--compress-zstd-level(=#)option. The defaul value is1.$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nTo decompress files, use the
"},{"location":"xtrabackup-binary-overview.html","title":"The xtrabackup binary overview","text":"--decompressoption.The xtrabackup binary is a compiled C program that is linked with the InnoDB libraries and the standard MySQL client libraries.
xtrabackup enables point-in-time backups of InnoDB / XtraDB tables together with the schema definitions, MyISAM tables, and other portions of the server.
The InnoDB libraries provide the functionality to apply a log to data files. The MySQL client libraries are used to parse command-line options and configuration file.
The tool runs in either
"},{"location":"xtrabackup-exit-codes.html","title":"xtrabackup exit codes","text":"--backupor--preparemode, corresponding to the two main functions it performs. There are several variations on these functions to accomplish different tasks, and there are two less commonly used modes,--statsand--print-param.The xtrabackup binary exits with the traditional success value of 0 after a backup when no error occurs. If an error occurs during the backup, the exit value is 1.
In certain cases, the exit value can be something other than 0 or 1, due to the command-line option code included from the MySQL libraries. An unknown command-line option, for example, will cause an exit code of 255.
"},{"location":"xtrabackup-files.html","title":"Index of files created by Percona XtraBackup","text":"- Information related to the backup and the server
backup-my.cnfThis file contains information to start the mini instance of InnoDB during the--prepare. This **not** a backup of the originalmy.cnf. The InnoDB configuration is read from the filebackup-my.cnfcreated by xtrabackup when the backup was made. The--prepareuses InnoDB configuration frombackup-my.cnfby default, or from--defaults-file, if specified. The InnoDB's configuration in this context means server variables that affect dataformat, i.e.innodb_page_sizeoption,innodb_log_block_size, etc. Location-related variables, likeinnodb_log_group_home_dirorinnodb_data_file_pathare always ignored by--prepare, so preparing a backup always works with data files from the back directory, rather than any external ones.xtrabackup_checkpointsThe type of the backup (for example, full or incremental), its state (for example, prepared) and the LSN range contained in it. This information is used for incremental backups. Example of the
xtrabackup_checkpointsafter taking a full backup:backup_type = full-backuped\nfrom lsn= 0\nto_lsn = 15188961605\nlast_lsn = 15188961605\n
Example of thextrabackup_checkpointsafter taking an incremental backup:backup_type = incremental\nfrom_lsn = 15188961605\nto_lsn = 15189350111\nlast_lsn = 15189350111\n
xtrabackup_binlog_infoThe binary log file used by the server and its position at the moment of the backup. A result of the following query:
SELECT server_uuid, local, replication, storage_engines FROM performance_schema.log_status;\n
xtrabackup_binlogThe xtrabackup binary used in the process.xtrabackup_logfileContains data needed for running the:--prepare. > The bigger this file is the--prepareprocess > will take longer to finish.<table_name>.delta.metaThis file is going to be created when performing the incremental backup. > It contains the per-table delta metadata: page size, size of compressed > page (if the value is 0 it means the tablespace isn\u2019t compressed) and > space id. Example of this file:
page_size = 16384\nzip_size = 0\nspace_id = 0\n
-
Information related to the replication environment (if using the
--slave-infooption):xtrabackup_slave_infoThe
CHANGE MASTERstatement needed for setting up a replica. -
Information related to the Galera and Percona XtraDB Cluster (if using the
--galera-infooption):xtrabackup_galera_infoContains the values of
wsrep_local_state_uuidandwsrep_last_committedstatus variables
This page contains notes on various internal aspects of the xtrabackup tool\u2019s operation.
"},{"location":"xtrabackup-implementation-details.html#file-permissions","title":"File permissions","text":"xtrabackup opens the source data files in read-write mode, although it does not modify the files. This means that you must run xtrabackup as a user who has permission to write the data files. The reason for opening the files in read-write mode is that xtrabackup uses the embedded InnoDB libraries to open and read the files, and InnoDB opens them in read-write mode because it normally assumes it is going to write to them.
"},{"location":"xtrabackup-implementation-details.html#tune-the-os-buffers","title":"Tune the OS buffers","text":"Because xtrabackup reads large amounts of data from the filesystem, it uses
posix_fadvise()where possible, to instruct the operating system not to try to cache the blocks it reads from disk. Without this hint, the operating system would prefer to cache the blocks, assuming thatxtrabackupis likely to need them again, which is not the case. Caching such large files can place pressure on the operating system\u2019s virtual memory and cause other processes, such as the database server, to be swapped out. Thextrabackuptool avoids this with the following hint on both the source and destination files:posix_fadvise(file, 0, 0, POSIX_FADV_DONTNEED)\nIn addition, xtrabackup asks the operating system to perform more aggressive read-ahead optimizations on the source files:
"},{"location":"xtrabackup-implementation-details.html#copy-data-files","title":"Copy data files","text":"posix_fadvise(file, 0, 0, POSIX_FADV_SEQUENTIAL)\nWhen copying the data files to the target directory, xtrabackup reads and writes 1 MB of data at a time. This is not configurable. When copying the log file, xtrabackup reads and writes 512 bytes at a time. This is also not possible to configure, and matches InnoDB\u2019s behavior (workaround exists in Percona Server for MySQL because it has an option to tune
innodb_log_block_sizefor XtraDB, and in that case Percona XtraBackup will match the tuning).After reading from the files,
"},{"location":"xtrabackup-option-reference.html","title":"The xtrabackup option reference","text":"xtrabackupiterates over the 1MB buffer a page at a time, and checks for page corruption on each page with InnoDB\u2019sbuf_page_is_corrupted()function. If the page is corrupt, it re-reads and retries up to 10 times for each page. It skips this check on the doublewrite buffer.Here you can find all of the command-line options for the xtrabackup binary.
"},{"location":"xtrabackup-option-reference.html#modes-of-operation","title":"Modes of operation","text":"You invoke xtrabackup in one of the following modes:
-
--backupmode to make a backup in a target directory -
--preparemode to restore data from a backup (created in--backupmode) -
--copy-backto copy data from a backup to the location that contained the original data; to move data instead of copying use the alternate--move-backmode. -
--statsmode to scan the specified data files and print out index statistics.
When you intend to run xtrabackup in any of these modes, use the following syntax:
$ xtrabackup [--defaults-file=#] --backup|--prepare|--copy-back|--stats [OPTIONS]\nFor example, the
--preparemode is applied as follows:$ xtrabackup --prepare --target-dir=/data/backup/mysql/\nFor all modes, the default options are read from the xtrabackup and mysqld configuration groups from the following files in the given order:
-
/etc/my.cnf -
/etc/mysql/my.cnf -
/usr/etc/my.cnf -
~/.my.cnf.
As the first parameter to xtrabackup (in place of the
--defaults-file, you may supply one of the following:-
--print-defaultsto have xtrabackup print the argument list and exit. -
--no-defaultsto forbid reading options from any file but the login file. -
--defaults-fileto read the default options from the given file. -
--defaults-extra-fileto read the specified additional file after the global files have been read. -
--defaults-group-suffixto read the configuration groups with the given suffix. The effective group name is constructed by concatenating the default configuration groups (xtrabackup and mysqld) with the given suffix. -
--login-pathto read the given path from the login file.
There is a large group of InnoDB options that are normally read from the
"},{"location":"xtrabackup-option-reference.html#options","title":"Options","text":""},{"location":"xtrabackup-option-reference.html#-apply-log-only","title":"--apply-log-only()","text":"my.cnfconfiguration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify them explicitly. These options have the same behavior in InnoDB and XtraDB. See--innodb-miscellaneousfor more information.This option causes only the redo stage to be performed when preparing a backup. It is very important for incremental backups.
"},{"location":"xtrabackup-option-reference.html#-backup","title":"--backup()","text":"Make a backup and place it in
"},{"location":"xtrabackup-option-reference.html#-backup-lock-timeout","title":"--backup-lock-timeout()","text":"--target-dir. See Creating a backup.The timeout in seconds for attempts to acquire metadata locks.
"},{"location":"xtrabackup-option-reference.html#-backup-lock-retry-count","title":"--backup-lock-retry-count()","text":"The number of attempts to acquire metadata locks.
"},{"location":"xtrabackup-option-reference.html#-backup-locks","title":"--backup-locks()","text":"This option controls if backup locks should be used instead of
"},{"location":"xtrabackup-option-reference.html#-check-privileges","title":"--check-privileges()","text":"FLUSH TABLES WITH READ LOCKon the backup stage. The option has no effect when backup locks are not supported by the server. This option is enabled by default, disable with--no-backup-locks.This option checks if Percona XtraBackup has all required privileges. If a missing privilege is required for the current operation, it will terminate and print out an error message. If a missing privilege is not required for the current operation, but may be necessary for some other XtraBackup operation, the process is not aborted and a warning is printed.
"},{"location":"xtrabackup-option-reference.html#-close-files","title":"--close-files()","text":"xtrabackup: Error: missing required privilege LOCK TABLES on *.*\nxtrabackup: Warning: missing required privilege REPLICATION CLIENT on *.*\nDo not keep files opened. When xtrabackup opens tablespace it normally doesn\u2019t close its file handle in order to handle the DDL operations correctly. However, if the number of tablespaces is really huge and can not fit into any limit, there is an option to close file handles once they are no longer accessed. Percona XtraBackup can produce inconsistent backups with this option enabled. Use at your own risk.
"},{"location":"xtrabackup-option-reference.html#-compress","title":"--compress()","text":"This option tells xtrabackup to compress all output data, including the transaction log file and meta data files, using either the
quicklz,lz4, orZSTDcompression algorithm.quicklzis chosen by default.Note
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either
LZ4or Zstandard (ZSTD) compression algorithms. See Create a compressed backup for more information.When using
--compress=quicklzor--compress, the resulting files have the qpress archive format. Every\\*.qpfile produced by xtrabackup is essentially a one-file qpress archive and can be extracted and uncompressed by theqpressfile archiver.--compress=lz4produces\\*.lz4files. You can extract the contents of these files by usinglz4program.
"},{"location":"xtrabackup-option-reference.html#-compress-chunk-size","title":"--compress-chunk-size(=#)","text":"--compress=zstdproduces\\*.zstfiles. You can extract the contents of these files by using the--decompressoption. You can specifyZSTDcompression level with the--compress-zstd-level(=#)option.Size of working buffer(s) for compression threads in bytes. The default value is 64K.
"},{"location":"xtrabackup-option-reference.html#-compress-threads","title":"--compress-threads(=#)","text":"This option specifies the number of worker threads used by xtrabackup for parallel data compression. This option defaults to
"},{"location":"xtrabackup-option-reference.html#-compress-zstd-level","title":"--compress-zstd-level(=#)","text":"1. Parallel compression (--compress-threads) can be used together with parallel file copying (--parallel). For example,--parallel=4 --compress --compress-threads=2will create 4 I/O threads that will read the data and pipe it to 2 compression threads.This option is tech preview quality. Before using
--compress-zstd-level(=#)in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.This option specifies
"},{"location":"xtrabackup-option-reference.html#-copy-back","title":"--copy-back()","text":"ZSTDcompression level. The default value is 1. Allowed range of values is from 1 to 19. The option has been implemented in Percona XtraBackup 8.0.30-22.Copy all the files in a previously made backup from the backup directory to their original locations. This option will not copy over existing files unless
"},{"location":"xtrabackup-option-reference.html#-core-file","title":"--core-file()","text":"--force-non-empty-directoriesoption is specified.Write core on fatal signals.
"},{"location":"xtrabackup-option-reference.html#-databases","title":"--databases(=#)","text":"This option specifies a list of databases and tables that should be backed up. The option accepts the list of the form
"},{"location":"xtrabackup-option-reference.html#-databases-excludename","title":"--databases-exclude(=name)","text":"\"databasename1[.table_name1] databasename2[.table_name2] . . .\".Excluding databases based on name, Operates the same way as
"},{"location":"xtrabackup-option-reference.html#-databases-file","title":"--databases-file(=#)","text":"--databases, but matched names are excluded from backup. Note that this option has a higher priority than--databases.This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form
"},{"location":"xtrabackup-option-reference.html#-datadirdirectory","title":"--datadir(=DIRECTORY)","text":"databasename1[.table_name1], one element per line.The source directory for the backup. This should be the same as the datadir for your MySQL server, so it should be read from
my.cnfif that exists; otherwise you must specify it on the command line.When combined with the
--copy-backor--move-backoption,--datadirrefers to the destination directory.Once connected to the server, in order to perform a backup you will need
"},{"location":"xtrabackup-option-reference.html#-debug-sleep-before-unlock","title":"--debug-sleep-before-unlock(=#)","text":"READandEXECUTEpermissions at a filesystem level in the server\u2019s datadir.This is a debug-only option used by the xtrabackup test suite.
"},{"location":"xtrabackup-option-reference.html#-debug-syncname","title":"--debug-sync(=name)","text":"The debug sync point. This option is only used by the xtrabackup test suite.
"},{"location":"xtrabackup-option-reference.html#-decompress","title":"--decompress()","text":"Decompresses all files in a backup previously made with the
--compressoption. The--paralleloption will allow multiple files to be decrypted simultaneously. In order to decompress, the qpress utility MUST be installed and accessible within the path. Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory users should use--remove-originaloption.The
"},{"location":"xtrabackup-option-reference.html#-decompress-threads","title":"--decompress-threads(=#)","text":"--decompressoption may be used with xbstream to decompress individual qpress files.Force xbstream to use the specified number of threads for decompressing.
"},{"location":"xtrabackup-option-reference.html#-decryptencryption-algorithm","title":"--decrypt(=ENCRYPTION-ALGORITHM)","text":"Decrypts all files with the
"},{"location":"xtrabackup-option-reference.html#-defaults-extra-filemycnf","title":"--defaults-extra-file(=[MY.CNF])","text":".xbcryptextension in a backup previously made with--encryptoption. The--paralleloption will allow multiple files to be decrypted simultaneously. Percona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should use--remove-originaloption.Read this file after the global files are read. Must be given as the first option on the command-line.
"},{"location":"xtrabackup-option-reference.html#-defaults-filemycnf","title":"--defaults-file(=[MY.CNF])","text":"Only read default options from the given file. Must be given as the first option on the command-line. Must be a real file; it cannot be a symbolic link.
"},{"location":"xtrabackup-option-reference.html#-defaults-groupgroup-name","title":"--defaults-group(=GROUP-NAME)","text":"This option is to set the group which should be read from the configuration file. This is used by xtrabackup if you use the
"},{"location":"xtrabackup-option-reference.html#-defaults-group-suffix","title":"--defaults-group-suffix(=#)","text":"--defaults-groupoption. It is needed formysqld_multideployments.Also reads groups with concat(group, suffix).
"},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool","title":"--dump-innodb-buffer-pool()","text":"This option controls whether or not a new dump of buffer pool content should be done.
With
--dump-innodb-buffer-pool, xtrabackup makes a request to the server to start the buffer pool dump (it takes some time to complete and is done in background) at the beginning of a backup provided the status variableinnodb_buffer_pool_dump_statusreports that the dump has been completed.$ xtrabackup --backup --dump-innodb-buffer-pool --target-dir=/home/user/backup\nBy default, this option is set to OFF.
If
innodb_buffer_pool_dump_statusreports that there is running dump of buffer pool, xtrabackup waits for the dump to complete using the value of--dump-innodb-buffer-pool-timeoutThe file
"},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool-timeout","title":"--dump-innodb-buffer-pool-timeout()","text":"ib_buffer_poolstores tablespace ID and page ID data used to warm up the buffer pool sooner.This option contains the number of seconds that xtrabackup should monitor the value of
innodb_buffer_pool_dump_statusto determine if buffer pool dump has completed.This option is used in combination with
"},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool-pct","title":"--dump-innodb-buffer-pool-pct()","text":"--dump-innodb-buffer-pool. By default, it is set to 10 seconds.This option contains the percentage of the most recently used buffer pool pages to dump.
This option is effective if
"},{"location":"xtrabackup-option-reference.html#-encryptencryption_algorithm","title":"--encrypt(=ENCRYPTION_ALGORITHM)","text":"--dump-innodb-buffer-pooloption is set to ON. If this option contains a value, xtrabackup sets the MySQL system variableinnodb_buffer_pool_dump_pct. As soon as the buffer pool dump completes or it is stopped (see--dump-innodb-buffer-pool-timeout), the value of the MySQL system variable is restored.This option instructs xtrabackup to encrypt backup copies of InnoDB data files using the algorithm specified in the ENCRYPTION_ALGORITHM. Currently supported algorithms are:
"},{"location":"xtrabackup-option-reference.html#-encrypt-keyencryption_key","title":"--encrypt-key(=ENCRYPTION_KEY)","text":"AES128,AES192andAES256A proper length encryption key to use. It is not recommended to use this option where there is uncontrolled access to the machine as the command line and thus the key can be viewed as part of the process info.
"},{"location":"xtrabackup-option-reference.html#-encrypt-key-fileencryption_key_file","title":"--encrypt-key-file(=ENCRYPTION_KEY_FILE)","text":"The name of a file where the raw key of the appropriate length can be read from. The file must be a simple binary (or text) file that contains exactly the key to be used.
It is passed directly to the xtrabackup child process. See the xtrabackup documentation for more details.
"},{"location":"xtrabackup-option-reference.html#-encrypt-threads","title":"--encrypt-threads(=#)","text":"This option specifies the number of worker threads that will be used for parallel encryption/decryption. See the xtrabackup documentation for more details.
"},{"location":"xtrabackup-option-reference.html#-encrypt-chunk-size","title":"--encrypt-chunk-size(=#)","text":"This option specifies the size of the internal working buffer for each encryption thread, measured in bytes. It is passed directly to the xtrabackup child process.
To adjust the chunk size for encrypted files, use
"},{"location":"xtrabackup-option-reference.html#-estimate-memory","title":"--estimate-memory(=#)","text":"--read-buffer-sizeand--encrypt-chunk-size.This option is in tech preview. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Implemented in Percona XtraBackup 8.0.32-26, the option lets you enable or disable the Smart memory estimation feature. The default value is OFF. Enable the feature by setting
--estimate-memory=ONin the backup phase and setting the--use-free-memory-pctoption in the--preparephase. If the--estimate-memorysetting is disabled, the--use-free-memory-pctsetting is ignored.An example of how to enable the Smart memory estimation feature:
$ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n
"},{"location":"xtrabackup-option-reference.html#-export","title":"--export()","text":"$ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\nCreate files necessary for exporting tables. See Restoring Individual Tables.
"},{"location":"xtrabackup-option-reference.html#-extra-lsndirdirectory","title":"--extra-lsndir(=DIRECTORY)","text":"(for \u2013backup): save an extra copy of the
"},{"location":"xtrabackup-option-reference.html#-force-non-empty-directories","title":"--force-non-empty-directories()","text":"xtrabackup_checkpointsandxtrabackup_infofiles in this directory.When specified, it makes
"},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-timeoutseconds","title":"--ftwrl-wait-timeout(=SECONDS)","text":"--copy-backand--move-backoption transfer files to non-empty directories. No existing files will be overwritten. If files that need to be copied/moved from the backup directory already exist in the destination directory, it will still fail with an error.This option specifies time in seconds that xtrabackup should wait for queries that would block
"},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-thresholdseconds","title":"--ftwrl-wait-threshold(=SECONDS)","text":"FLUSH TABLES WITH READ LOCKbefore running it. If there are still such queries when the timeout expires, xtrabackup terminates with an error. Default is0, in which case it does not wait for queries to complete and startsFLUSH TABLES WITH READ LOCKimmediately. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative toFLUSH TABLES WITH READ LOCKto copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.This option specifies the query run time threshold which is used by xtrabackup to detect long-running queries with a non-zero value of
"},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-query-typeallupdate","title":"--ftwrl-wait-query-type(=all|update)","text":"--ftwrl-wait-timeout.FLUSH TABLES WITH READ LOCKis not started until such long-running queries exist. This option has no effect if--ftwrl-wait-timeoutis0. Default value is60seconds. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative toFLUSH TABLES WITH READ LOCKto copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.This option specifies which types of queries are allowed to complete before xtrabackup will issue the global lock. Default is
"},{"location":"xtrabackup-option-reference.html#-galera-info","title":"--galera-info()","text":"all.This option creates the
"},{"location":"xtrabackup-option-reference.html#-generate-new-master-key","title":"--generate-new-master-key()","text":"xtrabackup_galera_infofile which contains the local node state at the time of the backup. Option should be used when performing the backup of Percona XtraDB Cluster. It has no effect when backup locks are used to create the backup.Generate a new master key when doing a copy-back.
"},{"location":"xtrabackup-option-reference.html#-generate-transition-key","title":"--generate-transition-key()","text":"xtrabackup needs to access the same keyring file or vault server during prepare and copy-back but it should not depend on whether the server keys have been purged.
"},{"location":"xtrabackup-option-reference.html#-get-server-public-key","title":"--get-server-public-key()","text":"--generate-transition-keycreates and adds to the keyring a transition key for xtrabackup to use if the master key used for encryption is not found because it has been rotated and purged.Get the server public key
"},{"location":"xtrabackup-option-reference.html#-help","title":"--help()","text":"When run with this option or without any options xtrabackup displays information about how to run the program on the command line along with all supported options and variables with default values where appropriate.
"},{"location":"xtrabackup-option-reference.html#-historyname","title":"--history(=NAME)","text":"This option enables the tracking of backup history in the
"},{"location":"xtrabackup-option-reference.html#-hosthost","title":"--host(=HOST)","text":"PERCONA_SCHEMA.xtrabackup_historytable. An optional history series name may be specified that will be placed with the history record for the current backup being taken.This option accepts a string argument that specifies the host to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-incremental-basedirdirectory","title":"--incremental-basedir(=DIRECTORY)","text":"When creating an incremental backup, this is the directory containing the full backup that is the base dataset for the incremental backups.
"},{"location":"xtrabackup-option-reference.html#-incremental-dirdirectory","title":"--incremental-dir(=DIRECTORY)","text":"When preparing an incremental backup, this is the directory where the incremental backup is combined with the full backup to make a new full backup.
"},{"location":"xtrabackup-option-reference.html#-incremental-force-scan","title":"--incremental-force-scan()","text":"When creating an incremental backup, force a full scan of the data pages in that instance.
"},{"location":"xtrabackup-option-reference.html#-incremental-history-namename","title":"--incremental-history-name(=name)","text":"This option specifies the name of the backup series stored in the
"},{"location":"xtrabackup-option-reference.html#-incremental-history-uuidname","title":"--incremental-history-uuid(=name)","text":"PERCONA_SCHEMA.xtrabackup_historyhistory record to base an incremental backup on. xtrabackup will search the history table looking for the most recent (highestinnodb_to_lsn), successful backup in the series and take the to_lsn value to use as the startinglsnfor the incremental backup. This will be mutually exclusive with--incremental-history-uuid,--incremental-basedirand--incremental-lsn. If no valid lsn can be found (no series by that name, no successful backups by that name) xtrabackup will return with an error. It is used with the--incrementaloption.This option specifies the UUID of the specific history record stored in the
"},{"location":"xtrabackup-option-reference.html#-incremental-lsnlsn","title":"--incremental-lsn(=LSN)","text":"PERCONA_SCHEMA.xtrabackup_historyto base an incremental backup on.--incremental-history-name,--incremental-basedirand--incremental-lsn. If no valid lsn can be found (no success record with that UUID) xtrabackup will return with an error. It is used with the \u2013incremental option.When creating an incremental backup, you can specify the log sequence number (LSN) instead of specifying
--incremental-basedir. For databases created in 5.1 and later, specify the LSN as a single 64-bit integer.Important
If a wrong LSN value is specified (a user error which Percona XtraBackup does not detect), the backup is unusable. Be careful!
"},{"location":"xtrabackup-option-reference.html#-innodbname","title":"--innodb([=name])","text":"This option is ignored for MySQL option compatibility.
"},{"location":"xtrabackup-option-reference.html#-innodb-miscellaneous","title":"--innodb-miscellaneous()","text":"There is a large group of InnoDB options that are normally read from the
"},{"location":"xtrabackup-option-reference.html#-keyring-file-datafilename","title":"--keyring-file-data(=FILENAME)","text":"my.cnfconfiguration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify these explicitly. These options have the same behavior in InnoDB and XtraDB:The path to the keyring file. Combine this option with
"},{"location":"xtrabackup-option-reference.html#-kill-long-queries-timeoutseconds","title":"--kill-long-queries-timeout(=SECONDS)","text":"--xtrabackup-plugin-dir.This option specifies the number of seconds xtrabackup waits between starting
"},{"location":"xtrabackup-option-reference.html#-kill-long-query-typeallselect","title":"--kill-long-query-type(=all|select)","text":"FLUSH TABLES WITH READ LOCKand killing those queries that block it. Default is 0 seconds, which means xtrabackup will not attempt to kill any queries. In order to use this option xtrabackup user should have thePROCESSandSUPERprivileges. Where supported, xtrabackup automatically uses Backup locks as a lightweight alternative toFLUSH TABLES WITH READ LOCKto copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.This option specifies which types of queries should be killed to unblock the global lock. Default is \u201cselect\u201d.
"},{"location":"xtrabackup-option-reference.html#-lock-ddl","title":"--lock-ddl()","text":"Issue
LOCK TABLES FOR BACKUPif it is supported by server (otherwise useLOCK INSTANCE FOR BACKUP) at the beginning of the backup to block all DDL operations.Note
Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.
As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.
"},{"location":"xtrabackup-option-reference.html#-lock-ddl-per-table","title":"--lock-ddl-per-table()","text":"Lock DDL for each table before xtrabackup starts to copy it and until the backup is completed.
Note
As of Percona XtraBackup 8.0.15, the
"},{"location":"xtrabackup-option-reference.html#-lock-ddl-timeout","title":"--lock-ddl-timeout()","text":"\u2013lock-ddl-per-tableoption is deprecated. Use the\u2013lock-ddloption instead.If
"},{"location":"xtrabackup-option-reference.html#-log","title":"--log()","text":"LOCK TABLES FOR BACKUPorLOCK INSTANCE FOR BACKUPdoes not return within given timeout, abort the backup.This option is ignored for MySQL
"},{"location":"xtrabackup-option-reference.html#-log-bin","title":"--log-bin()","text":"The base name for the log sequence.
"},{"location":"xtrabackup-option-reference.html#-log-bin-indexname","title":"--log-bin-index(=name)","text":"File that holds the names for binary log files.
"},{"location":"xtrabackup-option-reference.html#-log-copy-interval","title":"--log-copy-interval(=#)","text":"This option specifies the time interval between checks done by the log copying thread in milliseconds (default is 1 second).
"},{"location":"xtrabackup-option-reference.html#-login-path","title":"--login-path()","text":"Read the given path from the login file.
"},{"location":"xtrabackup-option-reference.html#-move-back","title":"--move-back()","text":"Move all the files in a previously made backup from the backup directory to their original locations. As this option removes backup files, it must be used with caution.
"},{"location":"xtrabackup-option-reference.html#-no-backup-locks","title":"--no-backup-locks()","text":"Explicity disables the
"},{"location":"xtrabackup-option-reference.html#-no-defaults","title":"--no-defaults()","text":"--backup-locksoption which is enabled by default.The default options are only read from the login file.
"},{"location":"xtrabackup-option-reference.html#-no-lock","title":"--no-lock()","text":"Disables table lock with
"},{"location":"xtrabackup-option-reference.html#-no-server-version-check","title":"--no-server-version-check()","text":"FLUSH TABLES WITH READ LOCK. Use it only if all your tables are InnoDB and you do not care about the binary log position of the backup. This option shouldn\u2019t be used if there are anyDDLstatements being executed or if any updates are happening on non-InnoDB tables (this includes the system MyISAM tables in the mysql database), otherwise it could lead to an inconsistent backup. Where supported xtrabackup will automatically use Backup locks as a lightweight alternative toFLUSH TABLES WITH READ LOCKto copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables. If you are considering to use this because your backups are failing to acquire the lock, this could be because of incoming replication events are preventing the lock from succeeding. Please try using--safe-slave-backupto momentarily stop the replication replica thread, this may help the backup to succeed and you do not need to use this option.Implemented in Percona XtraBackup 8.0.21.
The
--no-server-version-checkoption disables the server version check.The default behavior runs a check that compares the source system version to the Percona XtraBackup version. If the source system version is higher than the XtraBackup version, the backup is aborted with a message.
Adding the option overrides this check, and the backup proceeds, but there may be issues with the backup.
See Server Version and Backup Version Comparison for more information.
"},{"location":"xtrabackup-option-reference.html#-no-version-check","title":"--no-version-check()","text":"This option disables the version check. If you do not pass this option, the automatic version check is enabled implicitly when xtrabackup runs in the
--backupmode. To disable the version check, you should pass explicitly the--no-version-checkoption when invoking xtrabackup.When the automatic version check is enabled, xtrabackup performs a version check against the server on the backup stage after creating a server connection. xtrabackup sends the following information to the server:
-
MySQL flavour and version
-
Operating system name
-
Percona Toolkit version
-
Perl version
Each piece of information has a unique identifier. This is a MD5 hash value that Percona Toolkit uses to obtain statistics about how it is used. This is a random UUID; no client information is either collected or stored.
"},{"location":"xtrabackup-option-reference.html#-open-files-limit","title":"--open-files-limit(=#)","text":"The maximum number of file descriptors to reserve with setrlimit().
"},{"location":"xtrabackup-option-reference.html#-parallel","title":"--parallel(=#)","text":"This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer). In Percona XtraBackup 2.3.10 and newer, this option can be used with the
"},{"location":"xtrabackup-option-reference.html#-passwordpassword","title":"--password(=PASSWORD)","text":"--copy-backoption to copy the user data files in parallel (redo logs and system tablespaces are copied in the main thread).This option specifies the password to use when connecting to the database. It accepts a string argument. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-plugin-load","title":"--plugin-load()","text":"List of plugins to load.
"},{"location":"xtrabackup-option-reference.html#-portport","title":"--port(=PORT)","text":"This option accepts a string argument that specifies the port to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-prepare","title":"--prepare()","text":"Makes xtrabackup perform a recovery on a backup created with
"},{"location":"xtrabackup-option-reference.html#-print-defaults","title":"--print-defaults()","text":"--backup, so that it is ready to use. See preparing a backup.Print the program argument list and exit. Must be given as the first option on the command-line.
"},{"location":"xtrabackup-option-reference.html#-print-param","title":"--print-param()","text":"Makes xtrabackup print out parameters that can be used for copying the data files back to their original locations to restore them.
"},{"location":"xtrabackup-option-reference.html#-read-buffer-size","title":"--read-buffer-size()","text":"Set the read buffer size. The given value is scaled up to page size. The default size is 10MB. Use this option to increase the xbcloud/xbstream chunk size from the default size. To adjust the chunk size for encrypted files, use
"},{"location":"xtrabackup-option-reference.html#-rebuild-indexes","title":"--rebuild-indexes()","text":"--read-buffer-sizeand--encrypt-chunk-size.Rebuilds indexes in a compact backup. This option only has effect when the
"},{"location":"xtrabackup-option-reference.html#-rebuild-threads","title":"--rebuild-threads(=#)","text":"--prepareand--rebuild-threadsoptions are provided.Uses the given number of threads to rebuild indexes in a compact backup. This option only has effect with the
"},{"location":"xtrabackup-option-reference.html#-register-redo-log-consumer","title":"--register-redo-log-consumer()","text":"--prepareand--rebuild-indexesoptions.The
--register-redo-log-consumerparameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.Implemented in Percona XtraBackup 8.0.30-23.
"},{"location":"xtrabackup-option-reference.html#-remove-original","title":"--remove-original()","text":"Implemented in Percona XtraBackup 2.4.6, this option when specified will remove
"},{"location":"xtrabackup-option-reference.html#-rocksdb-datadir","title":"--rocksdb-datadir()","text":".qp,.xbcryptand.qp.xbcryptfiles after decryption and decompression.RocksDB data directory
"},{"location":"xtrabackup-option-reference.html#-rocksdb-wal-dir","title":"--rocksdb-wal-dir()","text":"RocksDB WAL directory.
"},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-age","title":"--rocksdb-checkpoint-max-age()","text":"The checkpoint cannot be older than this number of seconds when the backup completes.
"},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-count","title":"--rocksdb-checkpoint-max-count()","text":"Complete the backup even if the checkpoint age requirement has not been met after this number of checkpoints.
"},{"location":"xtrabackup-option-reference.html#-rollback-prepared-trx","title":"--rollback-prepared-trx()","text":"Force rollback prepared InnoDB transactions.
"},{"location":"xtrabackup-option-reference.html#-rsync","title":"--rsync()","text":"Uses the rsync utility to optimize local file transfers. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. This option cannot be used together with
"},{"location":"xtrabackup-option-reference.html#-safe-slave-backup","title":"--safe-slave-backup()","text":"--stream.When specified, xtrabackup will stop the replica SQL thread just before running
FLUSH TABLES WITH READ LOCKand wait to start backup untilSlave_open_temp_tablesinSHOW STATUSis zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail ifSlave_open_temp_tablesdoes not become zero after--safe-slave-backup-timeoutseconds. The replication SQL thread will be restarted when the backup finishes. This option is implemented in order to deal with replicating temporary tables and isn\u2019t necessary with Row-Based-Replication.Note
Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.
As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.
"},{"location":"xtrabackup-option-reference.html#-safe-slave-backup-timeoutseconds","title":"--safe-slave-backup-timeout(=SECONDS)","text":"How many seconds
"},{"location":"xtrabackup-option-reference.html#-secure-auth","title":"--secure-auth()","text":"--safe-slave-backupshould wait forSlave_open_temp_tablesto become zero. Defaults to 300 seconds.Refuse client connecting to server if it uses old (pre-4.1.1) protocol. (Enabled by default; use \u2013skip-secure-auth to disable.)
"},{"location":"xtrabackup-option-reference.html#-server-id","title":"--server-id(=#)","text":"The server instance being backed up.
"},{"location":"xtrabackup-option-reference.html#-server-public-key-path","title":"--server-public-key-path()","text":"The file path to the server public RSA key in the PEM format.
"},{"location":"xtrabackup-option-reference.html#-skip-tables-compatibility-check","title":"--skip-tables-compatibility-check()","text":"See
"},{"location":"xtrabackup-option-reference.html#-slave-info","title":"--slave-info()","text":"--tables-compatibility-check.This option is useful when backing up a replication replica server. It prints the binary log position of the source server. It also writes the binary log coordinates to the
"},{"location":"xtrabackup-option-reference.html#-socket","title":"--socket()","text":"xtrabackup_slave_infofile as aCHANGE MASTERcommand. A new replica for this source can be set up by starting a replica server on this backup and issuing aCHANGE MASTERcommand with the binary log position saved in thextrabackup_slave_infofile.This option accepts a string argument that specifies the socket to use when connecting to the local database server with a UNIX domain socket. It is passed to the mysql child process without alteration. See mysql --help for details.
"},{"location":"xtrabackup-option-reference.html#-ssl","title":"--ssl()","text":"Enable secure connection.
"},{"location":"xtrabackup-option-reference.html#-ssl-ca","title":"--ssl-ca()","text":"Path of the file which contains list of trusted SSL CAs.
"},{"location":"xtrabackup-option-reference.html#-ssl-capath","title":"--ssl-capath()","text":"Directory path that contains trusted SSL CA certificates in PEM format.
"},{"location":"xtrabackup-option-reference.html#-ssl-cert","title":"--ssl-cert()","text":"Path of the file which contains X509 certificate in PEM format.
"},{"location":"xtrabackup-option-reference.html#-ssl-cipher","title":"--ssl-cipher()","text":"List of permitted ciphers to use for connection encryption.
"},{"location":"xtrabackup-option-reference.html#-ssl-crl","title":"--ssl-crl()","text":"Path of the file that contains certificate revocation lists. MySQL server documentation.
"},{"location":"xtrabackup-option-reference.html#-ssl-crlpath","title":"--ssl-crlpath()","text":"Path of directory that contains certificate revocation list files.
"},{"location":"xtrabackup-option-reference.html#-ssl-fips-mode","title":"--ssl-fips-mode()","text":"SSL FIPS mode (applies only for OpenSSL); permitted values are: OFF, ON, STRICT.
"},{"location":"xtrabackup-option-reference.html#-ssl-key","title":"--ssl-key()","text":"Path of file that contains X509 key in PEM format.
"},{"location":"xtrabackup-option-reference.html#-ssl-mode","title":"--ssl-mode()","text":"Security state of connection to server.
"},{"location":"xtrabackup-option-reference.html#-ssl-verify-server-cert","title":"--ssl-verify-server-cert()","text":"Verify server certificate Common Name value against host name used when connecting to server.
"},{"location":"xtrabackup-option-reference.html#-stats","title":"--stats()","text":"Causes xtrabackup to scan the specified data files and print out index statistics.
"},{"location":"xtrabackup-option-reference.html#-streamformat","title":"--stream(=FORMAT)","text":"Stream all backup files to the standard output in the specified format. Currently, this option only supports the xbstream format.
"},{"location":"xtrabackup-option-reference.html#-strict","title":"--strict()","text":"If this option is specified, xtrabackup fails with an error when invalid parameters are passed.
"},{"location":"xtrabackup-option-reference.html#-tablesname","title":"--tables(=name)","text":"A regular expression against which the full tablename, in
"},{"location":"xtrabackup-option-reference.html#-tables-compatibility-check","title":"--tables-compatibility-check()","text":"databasename.tablenameformat, is matched. If the name matches, the table is backed up. See partial backups.Enables the engine compatibility warning. The default value is ON. To disable the engine compatibility warning use
"},{"location":"xtrabackup-option-reference.html#-tables-excludename","title":"--tables-exclude(=name)","text":"--skip-tables-compatibility-check.Filtering by regexp for table names. Operates the same way as
"},{"location":"xtrabackup-option-reference.html#-tables-filename","title":"--tables-file(=name)","text":"--tables, but matched names are excluded from backup. Note that this option has a higher priority than--tables.A file containing one table name per line, in databasename.tablename format. The backup will be limited to the specified tables.
"},{"location":"xtrabackup-option-reference.html#-target-dirdirectory","title":"--target-dir(=DIRECTORY)","text":"This option specifies the destination directory for the backup. If the directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup will succeed. xtrabackup will not overwrite existing files, however; it will fail with operating system error 17,
file exists.If this option is a relative path, it is interpreted as being relative to the current working directory from which xtrabackup is executed.
In order to perform a backup, you need
"},{"location":"xtrabackup-option-reference.html#-innodb-temp-tablespaces-dirdirectory","title":"--innodb-temp-tablespaces-dir(=DIRECTORY)","text":"READ,WRITE, andEXECUTEpermissions at a filesystem level for the directory that you supply as the value of--target-dir.Directory where temp tablespace files live, this path can be absolute.
"},{"location":"xtrabackup-option-reference.html#-throttle","title":"--throttle(=#)","text":"This option limits the number of chunks copied per second. The chunk size is 10 MB.
To limit the bandwidth to 10 MB/s, set the option to 1.
"},{"location":"xtrabackup-option-reference.html#-tls-ciphersuites","title":"--tls-ciphersuites()","text":"TLS v1.3 cipher to use.
"},{"location":"xtrabackup-option-reference.html#-tls-version","title":"--tls-version()","text":"TLS version to use, permitted values are: TLSv1, TLSv1.1, TLSv1.2, TLSv1.3.
"},{"location":"xtrabackup-option-reference.html#-tmpdirname","title":"--tmpdir(=name)","text":"Specify the directory that will be used to store temporary files during the backup
"},{"location":"xtrabackup-option-reference.html#-transition-keyname","title":"--transition-key(=name)","text":"This option is used to enable processing the backup without accessing the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and uses it to encrypt tablespace keys of tablespaces being backed up.
If
"},{"location":"xtrabackup-option-reference.html#-use-free-memory-pct","title":"--use-free-memory-pct()","text":"--transition-keydoes not have any value, xtrabackup will ask for it. The same passphrase should be specified for the--preparecommand.The
--use-free-memory-pctis a tech preview option. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.Implemented in Percona XtraBackup 8.0.30-23, this option lets you configure the Smart memory estimation feature. The option controls the amount of free memory that can be used to
--preparea backup. The default value is 0 (zero) which defines the option as disabled. For example, if you set--use-free-memory-pct=50, then 50% of the free memory is used topreparea backup. The maximum allowed value is 100.This option works, only if
--estimate-memoryoption is enabled. If the--estimate-memoryoption is disabled, the--use-free-memory-pctsetting is ignored.An example of how to enable the Smart memory estimation feature:
$ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n
"},{"location":"xtrabackup-option-reference.html#-use-memory","title":"--use-memory()","text":"$ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\nThis option affects how much memory is allocated and is similar to
"},{"location":"xtrabackup-option-reference.html#-userusername","title":"--user(=USERNAME)","text":"innodb_buffer_pool_size. This option is only relevant in the--preparephase or when analyzing statistics with--stats. The default value is 100MB. The recommended value is between 1GB to 2GB. Multiple values are supported if you provide the unit (for example, 1MB, 1M, 1GB, 1G).This option specifies the MySQL username used when connecting to the server, if that\u2019s not the current user. The option accepts a string argument. See mysql \u2013help for details.
"},{"location":"xtrabackup-option-reference.html#-v","title":"-v()","text":"See
"},{"location":"xtrabackup-option-reference.html#-version","title":"--version()","text":"--versionThis option prints xtrabackup version and exits.
"},{"location":"xtrabackup-option-reference.html#-xtrabackup-plugin-dirdirname","title":"--xtrabackup-plugin-dir(=DIRNAME)","text":"The absolute path to the directory that contains the
"},{"location":"xtrabackup-version-numbers.html","title":"Understand version numbers","text":"keyringplugin.A version number identifies the product release. The product contains the latest Generally Available (GA) features at the time of that release.
8.0.30 -23 Base version Minor build versionPercona uses semantic version numbering, which follows the pattern of base version and build version. Percona assigns unique, non-negative integers in increasing order for each version release. The version number combines the base MySQL 8.0 version number and the minor build version.
Note that Percona XtraBackup numbering changed after the 8.0.14 version to align Percona XtraBackup versions with Percona Server and MySQL. Find more information in Aligning Percona XtraBackup Versions with Percona Server for MySQL blog post.
The version numbers for Percona XtraBackup 8.0.30-23 define the following information:
-
Base version - the leftmost numbers indicate the MySQL 8.0 version used as a base. An increase in base version resets the minor build version to 0.
-
Minor build version - an internal number that denotes the version of the software. A build version increases by one each time the Percona XtraBackup is released.
Percona XtraBackup 8.0.28-20 and 8.0.28-21 are multiple releases based on MySQL 8.0.28.
"},{"location":"yum-download-rpm.html","title":"Install Percona XtraBackup 8.0 using downloaded RPM packages","text":"Download
RPMpackages of the desired series for your architecture from the download page.The following example downloads Percona XtraBackup 8.0.4 release package for CentOS 7:
$ wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-8.0.4/binary/redhat/7/x86_64/percona-xtrabackup-80-8.0.4-1.el7.x86_64.rpm\nInstall Percona XtraBackup by running:
$ yum localinstall percona-xtrabackup-80-8.0.4-1.el7.x86_64.rpm\nWhen installing packages manually like this, you\u2019ll need to make sure to resolve all the dependencies and install missing packages yourself.
"},{"location":"yum-files.html","title":"Files in the RPM package built for Percona XtraBackup 8.0","text":"The following tables show what data each
Package Contains percona-xtrabackup-80 The latest Percona XtraBackup GA binaries and associated files percona-xtrabackup-80-debuginfo The debug symbols for binaries in percona-xtrabackup-80 percona-xtrabackup-test-80 The test suite for Percona XtraBackup percona-xtrabackup The older version of the Percona XtraBackup"},{"location":"yum-repo.html","title":"Use a YUM repository to install Percona XtraBackup","text":"RPMpackage contains:Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page. The Percona yum repository supports popular
RPM-based operating systems, including theAmazon Linux AMI.The easiest way to install the Percona Yum repository is to install an
RPMthat configures yum and installs the Percona GPG key.Specific information on the supported platforms, products, and versions is described in Percona Software and Platform Lifecycle.
"},{"location":"yum-repo.html#install-percona-xtrabackup-from-percona-yum-repository","title":"Install Percona XtraBackup from Perconayumrepository","text":"-
Install the Percona yum repository by running the following command as the
rootuser or with sudo:$ sudo yum install \\\nhttps://repo.percona.com/yum/percona-release-latest.\\\nnoarch.rpm\n -
Enable the repository:
$ sudo percona-release enable-only tools release\nIf Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the `tools repository:
$ sudo percona-release enable-only tools\n -
Install Percona XtraBackup by running:
$ sudo yum install percona-xtrabackup-80\nWarning
Make sure that you have the
libevpackage installed before installing Percona XtraBackup on CentOS 6. For this operating system, thelibevpackage is available from the EPEL repositories. -
To decompress backups made using
Install theLZ4orZSTDcompression algorithm, install the corresponding package:lz4packageInstall thezstdpackage$ sudo yum install lz4\n$ sudo yum install zstd\n
See also
To install Percona XtraBackup using downloaded rpm packages, see Install with package manager.
To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup
"},{"location":"yum-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.0 on Red Hat Enterprise Linux and CentOS","text":"To completely uninstall Percona XtraBackup, remove all the installed packages:
"},{"location":"archive/enabling_tcp.html","title":"Enable the server to communicate via TCP/IP","text":"$ yum remove percona-xtrabackup\nMost of the Linux distributions do not enable by default to accept TCP/IP connections from outside in their MySQL or Percona Server packages.
You can check it with
netstaton a shell:$ netstat -lnp | grep mysql\ntcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 2480/mysqld\nunix 2 [ ACC ] STREAM LISTENING 8101 2480/mysqld /tmp/mysql.sock\nYou should check two things:
-
there is a line starting with
tcp(the server is indeed accepting TCP connections) -
the first address (
0.0.0.0:3306in this example) is different from127.0.0.1:3306(the bind address is not localhost\u2019s).
In the first case, the first place to look is the
my.cnffile. If you find the optionskip-networking, comment it out or just delete it. Also check that if the variablebind_addressis set, then it should not be set to localhost\u2019s but to the host\u2019s IP. Then restart the MySQL server and check it again withnetstat. If the changes had no effect, then you should look at your distribution\u2019s startup scripts (likerc.mysqld). You should comment out flags like--skip-networkingand/or change thebind-address.After you get the server listening to remote TCP connections properly, the last thing to do is checking that the port (3306 by default) is indeed open. Check your firewall configurations (
iptables -L) and that you are allowing remote hosts on that port (in/etc/hosts.allow).And we\u2019re done! We have a MySQL server running which is able to communicate with the world through TCP/IP.
"},{"location":"archive/incremental_backups.html","title":"Incremental backups","text":"xtrabackup supports incremental backups. It copies only the data that has changed since the last full backup. You can perform multiple incremental backups between each full backup. You can set up a backup process with a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.
Note
Incremental backups on the MyRocks storage engine do not determine if an earlier full backup or incremental backup contains the same files. Percona XtraBackup copies all of the MyRocks files each time it takes a backup.
The write-ahead log (WAL) is immutable and append-only. All writes to the WAL are sequential. Every record has a unique, uniformly increasing log sequence number (LSN) which is assigned to page changes. Each page contains a Page LSN that shows the most recent update to that page. An incremental backup copies each page whose LSN is newer than the previous incremental or the full backup\u2019s LSN. An algorithm finds the pages that match the criteria. The algorithm reads the data pages and checks the page LSN.
Incremental backups do not actually compare the data files to the previous backup\u2019s data files. In fact, you can use
"},{"location":"archive/incremental_backups.html#use-the-changed-page-tracking-algorithm","title":"Use the changed page tracking algorithm","text":"--incremental-lsnto perform an incremental backup without even having the previous backup, if you know its LSN. Incremental backups simply read the pages and compare their LSN to the last backup\u2019s LSN. You still need a full backup to recover the incremental changes, however; without a full backup to act as a base, the incremental backups are useless.Percona XtraBackup 8.0.30 removes the algorithm that used the changed page tracking feature in Percona Server for MySQL. Percona Server for MySQL 8.0.27 deprecated the changed page tracking feature.
With PXB 8.0.27 or earlier, another algorithm enabled the Percona Server for MySQL changed page tracking feature. The algorithm generates a bitmap file. The xtrabackup binary uses that bitmap file to read only those pages needed for the incremental backup. This method potentially saves resources. The backup enables the algorithm by default if the xtrabackup binary discovers the bitmap file. You can override the algorithm with
"},{"location":"archive/incremental_backups.html#create-an-incremental-backup","title":"Create an incremental backup","text":"--incremental-force-scanwhich forces a read of all pages even if the bitmap file is available.To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called
xtrabackup_checkpointsinto the backup\u2019s target directory. This file contains a line showing theto_lsn, which is the database\u2019s LSN at the end of the backup. Create the full backup with a command such as the following:$ xtrabackup --backup --target-dir=/data/backups/base --datadir=/var/lib/mysql/\nIf you look at the
Expected outputxtrabackup_checkpointsfile, you should see contents similar to the following:backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 1291135\nNow that you have a full backup, you can make an incremental backup based on it. Use a command such as the following:
$ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base --datadir=/var/lib/mysql/\nThe
Expected output/data/backups/inc1/directory should now contain delta files, such asibdata1.deltaandtest/table1.ibd.delta. These represent the changes since theLSN 1291135. If you examine thextrabackup_checkpointsfile in this directory, you should see something similar to the following:backup_type = incremental\nfrom_lsn = 1291135\nto_lsn = 1291340\nThe meaning should be self-evident. It\u2019s now possible to use this directory as the base for yet another incremental backup:
"},{"location":"archive/incremental_backups.html#prepare-the-incremental-backups","title":"Prepare the incremental backups","text":"$ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1 --datadir=/var/lib/mysql/\nThe
--preparestep for incremental backups is not the same as for normal backups. In normal backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing a backup, because transactions that were uncommitted at the time of your backup may be in progress, and it is likely that they will be committed in the next incremental backup. You should use the--apply-log-onlyoption to prevent the rollback phase.Note
If you do not use the
--apply-log-onlyoption to prevent the rollback phase, then your incremental backups will be useless. After transactions have been rolled back, further incremental backups cannot be applied.Beginning with the full backup you created, you can prepare it, and then apply the incremental differences to it. Recall that you have the following backups:
/data/backups/base\n/data/backups/inc1\n/data/backups/inc2\nTo prepare the base backup, you need to run
--prepareas usual, but prevent the rollback phase:
Expected output$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base\n101107 20:49:43 InnoDB: Shutdown completed; log sequence number 1291135\nThe log sequence number should match the
to_lsnof the base backup, which you saw previously.This backup is actually safe to restore as-is now, even though the rollback phase has been skipped. If you restore it and start MySQL, InnoDB will detect that the rollback phase was not performed, and it will do that in the background, as it usually does for a crash recovery upon start. It will notify you that the database was not shut down normally.
To apply the first incremental backup to the full backup, you should use the following command:
$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1\nThis applies the delta files to the files in
/data/backups/base, which rolls them forward in time to the time of the incremental backup. It then applies the redo log as usual to the result. The final data is in/data/backups/base, not in the incremental directory.incremental backup from 1291135 is enabled.\nxtrabackup: cd to /data/backups/base/\nxtrabackup: This target seems to be already prepared.\nxtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(1291340)\nApplying /data/backups/inc1/ibdata1.delta ...\nApplying /data/backups/inc1/test/table1.ibd.delta ...\n.... snip\n101107 20:56:30 InnoDB: Shutdown completed; log sequence number 1291340\nAgain, the LSN should match what you saw from your earlier inspection of the first incremental backup. If you restore the files from
/data/backups/base, you should see the state of the database as of the first incremental backup.Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:
$ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2\nNote
--apply-log-onlyshould be used when merging all incrementals except the last one. That\u2019s why the previous line doesn\u2019t contain the--apply-log-onlyoption. Even if the--apply-log-onlywas used on the last step, backup would still be consistent but in that case server would perform the rollback phase.If you wish to avoid the notice that InnoDB was not shut down normally, when you applied the desired deltas to the base backup, you can run
"},{"location":"archive/incremental_backups.html#restore-incremental-backups","title":"Restore incremental backups","text":"--prepareagain without disabling the rollback phase.After preparing the incremental backups, the base directory contains the same data as the full backup. To restoring this backup, you can use this command:
xtrabackup --copy-back --target-dir=BASE-DIRYou may have to change the ownership as detailed on Restoring a Backup.
"},{"location":"archive/incremental_backups.html#incremental-streaming-backups-using-xbstream","title":"Incremental streaming backups using xbstream","text":"Incremental streaming backups can be performed with the xbstream streaming option. Currently backups are packed in custom xbstream format. With this feature, you need to take a BASE backup as well.
"},{"location":"archive/incremental_backups.html#make-a-base-backup","title":"Make a base backup","text":"
"},{"location":"archive/incremental_backups.html#take-a-local-backup","title":"Take a local backup","text":"$ xtrabackup --backup --target-dir=/data/backups\n
"},{"location":"archive/incremental_backups.html#unpack-the-backup","title":"Unpack the backup","text":"$ xtrabackup --backup --incremental-lsn=LSN-number --stream=xbstream --target-dir=./ > incremental.xbstream\n
"},{"location":"archive/incremental_backups.html#take-a-local-backup-and-streaming-it-to-the-remote-server-and-unpack-it","title":"Take a local backup and streaming it to the remote server and unpack it","text":"$ xbstream -x < incremental.xbstream\n
"},{"location":"archive/recipes_xbk_compressed.html","title":"Make a compressed backup","text":"$ xtrabackup --backup --incremental-lsn=LSN-number --stream=xbstream --target-dir=./\n$ ssh user@hostname \" cat - | xbstream -x -C > /backup-dir/\"\nIn order to make a compressed backup, use the
--compressoption along with the--backupand--target-diroptions. Percona XtraBackup supports the following compression algorithms:quicklzNote
Starting with Percona XtraBackup 8.0.31-24 using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either
LZ4or Zstandard (ZSTD) compression algorithms.To compress files using the
quicklzcompression algorithm, use--compressoption:$ xtrabackup --backup --compress --target-dir=/data/backup\nlz4To compress files using the
lz4compression algorithm, set--compressoption tolz4:$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\nZstandard (ZSTD)The Zstandard (ZSTD) compression algorithm is a tech preview feature. Before using ZSTD in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.
Percona XtraBackup 8.0.30-23 adds support for the
Zstandard (ZSTD)compression algorithm.ZSTDis a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.To compress files using the
ZSTDcompression algorithm, set--compressoption tozstd:$ xtrabackup --backup --compress=zstd --target-dir=/data/backup\nYou can specify
ZSTDcompression level with the--compress-zstd-level(=#)option. The defaul value is1.$ xtrabackup --backup --compress-zstd-level=1 --target-dir=/data/backup\nIf you want to speed up the compression you can use the parallel compression, which can be enabled with
--compress-threadsoption. The following example uses four threads for compression.$ xtrabackup --backup --compress --compress-threads=4 --target-dir=/data/backup\nOutput should look like this:
Expected output
"},{"location":"archive/recipes_xbk_compressed.html#prepare-the-backup","title":"Prepare the backup","text":"...\n\n[01] Compressing ./imdb/comp_cast_type.ibd to /data/backup/imdb/comp_cast_type.ibd.qp\n[01] ...done\n...\n130801 11:50:24 xtrabackup: completed OK\nBefore you can prepare the backup you\u2019ll need to uncompress all the files.
If you used
quicklzcompression algorithm, useqpress(which is available from Percona Software repositories).You can use the following one-liner to uncompress all the files:
$ for bf in `find . -iname \"*\\.qp\"`; do qpress -d $bf $(dirname $bf) && rm $bf; done\nIf you used
lz4compression algorithm, change this script to search for\\*.lz4files:$ for bf in `find . -iname \"*\\.lz4\"`; do lz4 -d $bf $(dirname $bf) && rm $bf; done\nPercona XtraBackup has implemented
--decompressoption that can be used to decompress the backup made usingquicklz,lz4, orZSTDcompression algorithm.$ xtrabackup --decompress --target-dir=/data/compressed/\nWhen the files are uncompressed you can prepare the backup with the
--apply-log-onlyoption:$ xtrabackup --apply-log-only --target-dir=/data/backup/\nYou should check for a confirmation message:
Expected output130802 02:51:02 xtrabackup: completed OK!\nNow the files in
/data/backup/are ready to be used by the server.Note
Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory users should remove the
"},{"location":"archive/recipes_xbk_compressed.html#restore-the-backup","title":"Restore the backup","text":"\\*.qpfiles.Once the backup has been prepared you can use the
--copy-backto restore the backup.$ xtrabackup --copy-back --target-dir=/data/backup/\nThis will copy the prepared data back to its original location as defined by the
datadirvariable in themy.cnffile.After the confirmation message, you should check the file permissions after copying the data back.
Expected output130802 02:58:44 xtrabackup: completed OK!\nYou may need to adjust the file permissions. The following example demonstrates how to do it recursively by using chown:
$ chown -R mysql:mysql /var/lib/mysql\nNow, your data directory contains the restored data. You are ready to start the server.
"},{"location":"archive/recipes_xbk_full.html","title":"Make a full backup","text":"Backup the InnoDB data and log files located in
"},{"location":"archive/recipes_xbk_full.html#make-a-backup","title":"Make a backup","text":"/var/lib/mysql/to/data/backups/mysql/(destination). Then, prepare the backup files to be ready to restore or use (make the data files consistent).
"},{"location":"archive/recipes_xbk_full.html#prepare-the-backup","title":"Prepare the backup","text":"$ xtrabackup --backup --target-dir=/data/backup/mysql/\n$ xtrabackup --prepare --target-dir=/data/backup/mysql/\nNote
Starting with Percona Server for MySQL 8.0.30-22 only run prepare once.
"},{"location":"archive/recipes_xbk_full.html#success-criteria","title":"Success criteria","text":"- The exit status of xtrabackup is 0.
Note
You might want to set the
--use-memoryoption to a value close to the size of your buffer pool, if you are on a dedicated server that has enough free memory. The xtrabackup Option reference contains more details.Review Full Backups for a more detailed explanation.
"},{"location":"archive/recipes_xbk_inc.html","title":"Make an incremental backup","text":"Backup all InnoDB data and log files - located in
"},{"location":"archive/recipes_xbk_inc.html#create-one-full-backup","title":"Create one full backup","text":"/var/lib/mysql/- once, then make two daily incremental backups in/data/backups/mysql/(destination). Finally, prepare the backup files to be ready to restore or use.Making an incremental backup requires a full backup as a base:
$ xtrabackup --backup --target-dir=/data/backups/mysql/\nIt is important that you do not run the
"},{"location":"archive/recipes_xbk_inc.html#create-two-incremental-backups","title":"Create two incremental backups","text":"--preparecommand yet.Suppose the full backup is on Monday, and you will create an incremental one on Tuesday:
$ xtrabackup --backup --target-dir=/data/backups/inc/tue/ \\\n--incremental-basedir=/data/backups/mysql/\nand the same policy is applied on Wednesday:
"},{"location":"archive/recipes_xbk_inc.html#prepare-the-base-backup","title":"Prepare the base backup","text":"$ xtrabackup --backup --target-dir=/data/backups/inc/wed/ \\\n--incremental-basedir=/data/backups/inc/tue/\nPrepare the base backup (Monday\u2019s backup):
"},{"location":"archive/recipes_xbk_inc.html#roll-forward-the-base-data-to-the-first-increment","title":"Roll forward the base data to the first increment","text":"$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/\nRoll Monday\u2019s data forward to the state on Tuesday:
"},{"location":"archive/recipes_xbk_inc.html#roll-forward-again-to-the-second-increment","title":"Roll forward again to the second increment","text":"$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/ \\\n--incremental-dir=/data/backups/inc/tue/\nRoll forward again to the state on Wednesday (without --apply-log-only):
$ xtrabackup --prepare --target-dir=/data/backups/mysql/ \\\n--incremental-dir=/data/backups/inc/wed/\nNote
Starting with Percona Server for MySQL 8.0.30-22, preparing a backup in two steps, --prepare --apply-log-only and then a second --prepare is not supported. The second --prepare skips the dynamic metadata and causes corruption.
"},{"location":"archive/recipes_xbk_inc.html#notes","title":"Notes","text":"-
You might want to set the
--use-memoryto speed up the process if you are on a dedicated server that has enough free memory. See xtrabackup Option Reference -
See Incremental Backups
Many Linux distributions only install the ssh client by default. If you don\u2019t have the ssh server installed already, the easiest way of doing it is by using your distribution\u2019s packaging system:
Using apt, run the following:
$ sudo apt install openssh-server\nUsing Red Hat Linux or a derivative, use the following:
$ sudo yum install openssh-server\nReview your distribution\u2019s documentation on how to configure the server.
"},{"location":"release-notes/8.0/8.0-3-rc1.html","title":"Percona XtraBackup 8.0-3-rc1","text":"Percona is glad to announce the release of Percona XtraBackup 8.0-3-rc1 on October 31 2018. Downloads are available from our download site and from apt and yum repositories.
This is a Release Candidate quality release and it is not intended for production. If you want a high quality, Generally Available release, use the current Stable version (the most recent stable version at the time of writing is 2.4.12 in the 2.4 series).
"},{"location":"release-notes/8.0/8.0-3-rc1.html#things-to-note","title":"Things to Note","text":"-
innobackupexwas previously deprecated and has been removed -
Due to the new MySQL redo log and data dictionary formats the Percona XtraBackup 8.0.x versions will only be compatible with MySQL 8.0.x and the upcoming Percona Server for MySQL 8.0.x
-
For experimental migrations from earlier database server versions, you will need to backup and restore and using XtraBackup 2.4 and then use
mysql_upgradefrom MySQL 8.0.x
- PXB-1655: The
--lock-ddloption is supported when backing up MySQL 8
-
PXB-1678: Incremental backup prepare with the
--apply-log-onlyoption could roll back uncommitted transactions -
PXB-1672: The MTS slave without GTID could be backed up when the
--safe-slave-backupoption was applied.
-
Date
March 16, 2020
Installing Percona XtraBackup 8.0
Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.10.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-1982: The history table showed a wrong value for
lock_time. -
PXB-2099:
copy-backdid not move undo files to the data directory. -
PXB-2107: If the undo tablespace was created in a directory which is a symbolic link to another directory then the backup failed at backup stage.
-
PXB-2118: Undo log file was renamed incorrectly to undo tablespace name after restore
-
Date
April 13, 2020
Installing Percona XtraBackup 8.0
Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
This release fixes security vulnerability CVE-2020-10997.
"},{"location":"release-notes/8.0/8.0.12.html","title":"Percona XtraBackup 8.0.12","text":"-
Date
May 27, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.12 now supports backup and restore processing for all versions of MySQL; previous versions of Percona XtraBackup will not work with MySQL 8.0.20 and higher.
"},{"location":"release-notes/8.0/8.0.12.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-2133: Correct prtype stored for DECIMAL columns in .cfg file by \u2013export
-
PXB-2146: Amazon S3 session token support added to xbcloud
-
PXB-2179: Modify
xtrabackup prepareto shut down keyring plugin when run with \u2013generate-transition-key -
PXB-2157: Modify
xtrabackup \u2013prepareto generate cfg files for partition tables when \u2013export is used
-
Date
June 17, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.13 supports backup and restore processing for all versions of MySQL and has been tested with the latest MySQL 8.0.20.
"},{"location":"release-notes/8.0/8.0.13.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-2165: Modify xbcloud to store backups using s3 access key parameters if AWS access key env variables are set
-
PXB-2164: Modify xbcloud to return the error when the backup doesn\u2019t exist in s3 bucket
-
PXB-2127: Modify xbcloud to upload backups with empty database to min.io storage
-
PXB-2198: Modify xbcloud delete to return the error when the backup doesn\u2019t exist in s3 bucket
-
PXB-2155: Correct corruption when redo logs are encrypted using xtrabackup \u2013backup \u2013compress=lz4
-
PXB-2166: Modify xbcloud to check for bucket existence and return
okstatus when domain name can be resolved
-
Date
August 31, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.14 supports backup and restore processing for all versions of MySQL and has been tested with the latest MySQL 8.0.21.
"},{"location":"release-notes/8.0/8.0.14.html#improvements","title":"Improvements","text":"-
PXB-1605: Document how to use Percona Xtrabackup with Docker
-
PXB-2252: Introduce debug option to print redo log records scanned and applied
-
PXB-2215: Modify Import tablespace process to correctly calculate pack_length
-
PXB-2114: Document when table is ignored by full backup it is not automatically ignored for incremental backup
-
PXB-2255: Modify processing to Error out if undo truncation during backup
-
PXB-2249: Verify perl binary exists before completing version check
-
PXB-2243: Correct processing when undo truncation happens between full backup and incremental
-
PXB-2238: Provide binary tarball with shared libs and glibc suffix & minimal tarballs
-
PXB-2202: Modify Xbcloud to display an error when xtrabackup fails to create a backup
-
Date
December 14, 2020
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
NOTE: The naming structure for the releases has changed. See the aligning Percona XtraBackup versions with Percona Server for MySQL blog post for more information.
"},{"location":"release-notes/8.0/8.0.22-15.0.html#new-features","title":"New Features","text":"-
PXB-2112: xbcloud: support storage_class option with \u2013storage=s3 (Thanks to user rluisr for reporting this issue)
-
PXB-2242: Prevent back up if Source system (DB) version is higher then current PXB version
- PXB-2254: Redesign \u2013lock-ddl-per-table
-
PXB-793: Fix syntax error when executing \u2013lock-ddl-per-table queries
-
PXB-953: Improve stdout for the end of usage of \u2013lock-ddl-per-table
-
PXB-787: Modify scan to provide correct status when finding corrupted tablespace
-
PXB-2314: Handle Disk format changes introduced in MySQL 8.0.22 release
-
PXB-2279: Modify to look for prefixes in xtrabackup_tablespaces* without extensions (Thanks to user mrmainnet for reporting this issue)
-
PXB-2336: Modify to check to see if first block is an all-zero block and process accordingly
-
PXB-2275: Modify backup processing to add validations if an encrypted table is created
-
PXB-2272: Modify regexp to consider all #sql as temporary tables
-
PXB-2257: Modify \u2013lock-ddl-per-table to properly close database connection
-
Date
March 22, 2021
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
This release fixes the security vulnerability CVE-2020-29488
"},{"location":"release-notes/8.0/8.0.23-16.0.html#improvements","title":"Improvements","text":"-
PXB-2280: Provide SELinux and AppArmor default policies
-
PXB-1979: Enable \u2013lock-ddl by default to prevent corruption of the backup
-
PXB-2274: Correct restore processing when there are DML statements running during backup stage by writing the last_checkpoint and LSN from ps.log_status instead of the redo log (Thanks to user Li Biao for reporting this issue)
-
PXB-2430: Correct build failure by removing the dependency of no-server-version-check with version-check (Thanks to user agarner for reporting this issue)
-
PXB-2415: Add build dependencies to correct Debian/Ubuntu packages in docker (Thanks to user Matt Cole for reporting this issue)
-
PXB-2429: Correct Backup failure for encrypted tablespace by skipping the encryption reset operation during the redo scan phase
-
PXB-2418: Remove PROTOBUF_LITE_LIBRARY - it is not used and is not needed
-
PXB-2395: Update versions for xbstream and xbcrypt
-
PXB-2394: Correct spellings in xbcloud help
-
PXB-2357: Correct hang in backup with redo log archive by using block number instead of checksum (checksum of redo file and archive file can be different)
-
PXB-2180: Correct incremental prepare failure with logical redo by skipping the apply of logical redos (MLOG_TABLE_DYNAMIC_META) during the incremental prepare (except the last prepare).
-
Date
June 3, 2021
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.25-17.0.html#bugs-fixed","title":"Bugs Fixed","text":"-
PXB-2442 : Backup cannot be decompressed using the AppArmor profile
-
PXB-2444 : The xbcloud binary fails to upload backup with enforcing SELinux mode
-
PXB-2443 : Version check fails with AppArmor profile
-
PXB-2445 : Initializing the libgcrypt in xbcloud
-
PXB-2455 : XtraBackup prepare fails if the checkpoint LSN is greater than the last LSN
-
PXB-2473 : SELinux errors in audit logs
-
PXB-1462 : Long gtid_executed breaks \u2014history functionality
-
PXB-2369 : Issues with \u201cInstalling\u201d page in XtraBackup documentation
-
PXB-2457 : Incorrect binlog names if binlog name contains periods
-
PXB-2106 : Copy-back creates wrong binlog.index
-
Date
September 2, 2021
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.26-18.0.html#improvements","title":"Improvements","text":"-
PXB-2477: The xbcloud should retry on error and utilize incremental backoff (Thanks to Baptiste Mille-Mathias for reporting this issue)
-
PXB-2580: With the xbcloud binary, a chunk-upload on an SSL connect error to S3 was not retried (Thanks to Tim Vaillancourt for providing the patch)
-
PXB-2317: Remove the obsolete LOCKLESS binary log functionality since the
performance_schema.log_statustable is now used to get the log information on all the storages without locking them
-
PXB-2101: The Prepare step fails due to duplicate Serialized Dictionary Information (SDI) (Thanks to Fungo Wang for reporting this issue)
-
PXB-1504: The FIND_GCRYPT macro is broken (Thanks to Maxim Bublis for reporting this issue)
-
PXB-1961: The Prepare step of a full backup fails for encrypted tables with a generic error
-
PXB-2585: XtraBackup fails to backup archive RocksDB Write-Ahead Log (WAL) files
-
Date
February 2, 2022
-
Installation
Installing Percona XtraBackup
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.27-19.0.html#release-highlights","title":"Release Highlights","text":"The following list contains a number of the bug fixes for MySQL 8.0.27, provided by Oracle, and included in Percona Server for MySQL:
-
The
default_authentication_pluginis deprecated. Support for this plugin may be removed in future versions. Use theauthentication_policyvariable. -
The
binaryoperator is deprecated. Support for this operator may be removed in future versions. UseCAST(... AS BINARY). -
When a parent table updated or deleted a row, this operation initiated a cascading
SET NULLoperation on the child table. On the child table, a virtual column value could be set to NULL instead of the value derived from the base column value.
Find the full list of bug fixes and changes in the MySQL 8.0.27 Release Notes.
"},{"location":"release-notes/8.0/8.0.27-19.0.html#new-features","title":"New Features","text":"- PXB-1883: Added support for Microsoft Azure Cloud Storage in the xbcloud binary. (Thanks to Ivan for reporting this issue)
- PXB-2434: Use MySQL page tracking for incremental backup
-
PXB-1741: PS startup displays errors for databases excluded during partial backup
-
PXB-2614: The
xbstreambinary was enhanced to support sparse files on the XFS file system. -
PXB-2608: Upgrade the Vault API to V2 (Thanks to Benedito Marques Magalhaes for reporting this issue)
-
PXB-2543: Fix that adds
IB_EXPORT_CFG_VERSION_V6when exporting a.cfgfile for a table (Thanks to Peng Gao for reporting this issue) -
PXB-2465: Fix for querying the
ps.log_statuswhen group replication channels are items on that list. XtraBackup skips the name if it matches eithergroup_replication_applierorgroup_replication_recovery.(Thanks to Galamb Gerg\u0151 for reporting this issue) -
PXB-2625: The default version of CURL in Debian Buster failed to identify a TLS HTTP2 connection as closed and attempted to reuse the connection. (Thanks to Johan Andersson for reporting this issue)
-
Date
April 26, 2022
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.28-20.0.html#release-highlights","title":"Release Highlights","text":"The log statements structure in Percona XtraBackup has been improved. Now, the error logs have a standard structure. The uniformity of the headers makes it easier to follow an operation\u2019s progress or review the log to diagnose issues. The improved log structure is displayed in the backup, prepare, move-back/copy-back error logs.
Each log statement has the following attributes:
-
Timestamp - a timestamp for when the event occurred in a UTC format.
-
Severity - the severity level of a statement indicates the importance of an event.
-
ID - this identifier is currently not used but may be used in future versions.
-
Context - the name of the module that issued the log statement, such as XtraBackup, InnoDB, or Server.
-
Message - a description of the event generated by the module.
- PXB-2670: Improves the error logging framework in Percona XtraBackup.
-
PXB-1676: The error message after creating a backup was incorrect.
-
PXB-2716: Fix for a compilation error on systems without fallocate. (Thanks to user Bo98 for providing the patch to fix this issue.)
-
PXB-2506: During copy back and move back operations, added a check if the backup is fully prepared. Percona XtraBackup throws an error if the backup is not prepared or was prepared with \u2013apply-log-only.
-
PXB-2662: Percona XtraBackup exited during prepare when creating an incremental backup using page tracking.
-
PXB-2714: Fix for a memory leak on the keyring component.
-
PXB-2697: Undefined symbols in macOS resulted in an error when linked with Percona XtraBackup 8.0.27.
-
PXB-2722: Fix for when via command line, a password, passed using the -p option, was written into the backup tool_command in xtrabackup_info.
-
The Percona XtraBackup installation instructions
-
The Percona XtraBackup downloads
-
The Percona XtraBackup GitHub location
-
To contribute to the documentation, review the Documentation Contribution Guide
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.28-21.0.html#release-highlights","title":"Release Highlights","text":"Percona XtraBackup adds support for the Amazon Key Management Service (KMS) component.
"},{"location":"release-notes/8.0/8.0.28-21.0.html#new-features","title":"New Features","text":"- PXB-2721: Implements support for the Amazon Key Management Service component in Percona XtraBackup.
-
PXB-2761: Fix for a compilation error in GCC 11. (Thanks to user tcoyvwac for providing the patch to fix this issue.)
-
PXB-2422: The extraction of files failed when a file was located in another directory.
-
The Percona XtraBackup installation instructions
-
The Percona XtraBackup downloads
-
The Percona XtraBackup GitHub location
-
To contribute to the documentation, review the Documentation Contribution Guide
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.29-22.0.html#release-highlights","title":"Release Highlights","text":"Percona XtraBackup 8.0.29-22 adds new redo log types to support the changes in the
INSTANTalgorithm behavior. MySQL 8.0.29 extended the support forALGORITHM=INSTANTto allow columns to be added to any position in a table and column drops. Older versions of Percona XtraBackup are incompatible with MySQL 8.0.29 because of this new functionality.Note
MySQL 8.0.29 extended the support of the
ALTER TABLE \u2026 DROP COLUMNSstatement to useALGORITHM=INSTANT. These operations only modify the metadata in the data directory and do not affect the physical structure. TheALGORITHM=INSTANTis the default setting for supportedDDLoperations. Prior to MySQL 8.0.29, anALGORITHM=INSTANTcolumn could only be added as the last table column. As of MySQL 8.0.29, the column can be added to any table position. The ability to add or drop a column in any position increases the complexity of crash recovery since the redo log does not have the logical positions from the data dictionary. The redo log contains the physical order for the table.Percona Server for MySQL 8.0.29-21 contains fixes for these issues. Percona XtraBackup 8.0.29 can backup and restore tables that used the
"},{"location":"release-notes/8.0/8.0.29-22.0.html#useful-links","title":"Useful Links","text":"INSTANTalgorithm in ADD/DROP in Percona Server for MySQL. However, MySQL 8.0.29, if the server contains tables modified by INSTANT ADD/DROP, is unsafe for backups at this time. Percona XtraBackup detects the tables modified byINSTANTADD/DROP and generates an error. This error lists the affected tables and provides instructions to convert the modified tables to regular tables.-
The Percona XtraBackup installation instructions
-
The Percona XtraBackup downloads
-
The Percona XtraBackup GitHub location
-
To contribute to the documentation, review the Documentation Contribution Guide
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
For paid support, managed services or consulting services, contact Percona Sales.
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.30-23.0.html#release-highlights","title":"Release highlights","text":"The Take an incremental backup using page tracking feature is Generally Available (GA).
The following features and improvements introduced in Percona XtraBackup 8.0.30-23 are available only in tech preview:
-
Percona XtraBackup adds support for
Zstandard (ZSTD)compression algorithm.ZSTDis a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. -
Percona XtraBackup implements the
Smart memory estimationfeature to compute the memory required to--preparea backup. Percona XtraBackup has extended the crash recovery logic to extract the formula used to allocate memory. Now, in the backup phase, while copying redo log entries, Percona XtraBackup computes the required memory for--prepare. Percona XtraBackup also takes into consideration the number of InnoDB pages to be fetched from the disk. Percona XtraBackup checks the server\u2019s available free memory and uses that memory up to the limit specified in the--use-free-memory-pctoption to run--prepare.
PXB-2669: Implements support for
"},{"location":"release-notes/8.0/8.0.30-23.0.html#improvements","title":"Improvements","text":"Zstandard (ZSTD)compression algorithm.PXB-2710: Implements support for
Smart Memory Estimation.PXB-2844: Adjusts the logic to detect multi-threaded slave.
"},{"location":"release-notes/8.0/8.0.30-23.0.html#bug-fixes","title":"Bug fixes","text":"PXB-2681: Percona XtraBackup got stuck during incremental backup prepare of encrypted tables.
PXB-2517: Check if files-from is successfully closed before rsync (Thanks to Garen Chan for the patch.)
PXB-2702: Backup failed when undo truncation log is present (Thanks to Dmitry Smal for his contribution to the initial patch.)
PXB-2729: Percona XtraBackup compilation failed if GTest packages are installed.
PXB-2810: Percona XtraBackup crashed when AIO initialization failed (Thanks to Yan Huang for the patch.)
PXB-2840: Xbcloud attempted to create a non-existing bucket.
PXB-2874: When Percona XtraBackup generated a new master key on
--copy/move-back, the master key id was reset back to 1.PXB-2809:
"},{"location":"release-notes/8.0/8.0.30-23.0.html#platform-support","title":"Platform support","text":"wsrep_sync_wait<>0causedLock wait timeout exceeded; try restarting transactionerror (Thanks to Frank Well for providing the initial patch.)Percona XtraBackup 8.0.30 supports Oracle Linux/Red Hat Enterprise Linux 9.
"},{"location":"release-notes/8.0/8.0.30-23.0.html#useful-links","title":"Useful links","text":"-
The Percona XtraBackup GitHub location
-
Contribute to the documentation
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.31-24.0.html#release-highlights","title":"Release highlights","text":"This release adds the ability to use an IAM instance profile when running xbcloud from an EC2 instance. An IAM instance profile does not need the
--s3-secret-keynor the--s3-access-keybut the profile has a six-hour lifespan.With this version,
"},{"location":"release-notes/8.0/8.0.31-24.0.html#new-features","title":"New features","text":"qpress/QuickLZare deprecated.PXB-2856: Use IAM instance profile when running xbcloud from an EC2 instance.
PXB-2869: A warning is printed on backup that qpress/QuickLZ is deprecated.
"},{"location":"release-notes/8.0/8.0.31-24.0.html#bug-fixes","title":"Bug fixes","text":"PXB-2890: XtraBackup did not compress already compressed tables.
PXB-2928: XtraBackup exits with signal 11 when taking a backup using page tracking.
PXB-2854: Fixed a memory corruption issue in QuickLZ.
PXB-2925: The PXB start_lsn, during the prepare phase, did not match the redo file start LSN when a wait was added in the backup phase after calling
recv_find_max_checkpointand before copying the checkpoint header.PXB-2964: XtraBackup required a very high open files limit.
PXB-2959: XtraBackup now compiles on platforms with OpenSSL 3.0.
PXB-2971: XtraBackup copied local manifest files which caused node failures when the files overrode settings on another node.
"},{"location":"release-notes/8.0/8.0.31-24.0.html#deprecation-and-removal","title":"Deprecation and removal","text":"Starting with Percona XtraBackup 8.0.31-24, using qpress/QuickLZ to compress backups is deprecated and may be removed in future versions. We recommend using either the
"},{"location":"release-notes/8.0/8.0.31-24.0.html#useful-links","title":"Useful links","text":"LZ4or the Zstandard (ZSTD) compression algorithms.-
The Percona XtraBackup GitHub location
-
Contribute to the documentation
Percona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.32-25.0.html#release-highlights","title":"Release highlights","text":"This release merges the MySQL 8.0.32 code base. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
Starting with version 8.0.32-25, Percona XtraBackup allows instant columns in the backup for MySQL 8.0.32. This limitation still exists for backing up MySQL 8.0.31 and lower versions.
This limitation does not exist for backing up any version of Percona Server for MySQL.
"},{"location":"release-notes/8.0/8.0.32-25.0.html#useful-links","title":"Useful links","text":"The Percona XtraBackup GitHub location
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.32-26.0.html","title":"Percona XtraBackup 8.0.32-26 (2023-04-04)","text":"Release date April 4, 2023 Install instructions Install Percona XtraBackupPercona XtraBackup for MySQL Databases enables MySQL backups without blocking user queries. Percona XtraBackup is ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.
"},{"location":"release-notes/8.0/8.0.32-26.0.html#release-highlights","title":"Release highlights","text":"This release includes improvements and bug fixes. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
This release fixes the security vulnerability CVE-2022-25834 with PXB-2977.
This version adds the
"},{"location":"release-notes/8.0/8.0.32-26.0.html#improvements","title":"Improvements","text":"--estimate-memoryparameter to enable/disable the Smart memory estimation feature during the backup phase.PXB-2882: Removes bitmap code and variables.
PXB-2979: The Smart memory estimation feature parses metadata from the redo log to estimate the required memory to prepare a backup. On write-intensive workloads, this behavior caused the delay of the redo follow thread.
PXB-2980: Adds the
"},{"location":"release-notes/8.0/8.0.32-26.0.html#bug-fixes","title":"Bug fixes","text":"--estimate-memoryparameter to enable/disable the Smart memory estimation feature during the backup phase.PXB-2264: When a table with the next auto incremental value
2was exported using Percona XtraBackup, the table got the next auto incremental value1instead of2.PXB-2954: During prepare phase Percona XtraBackup uses Serialized Dictionary information (SDI) to create the tabels\u2019s metadata. If there was an orphan table (leftover from a upgrade) without SDI information, Percona XtraBackup failed to prepare a backup.
PXB-2970: Removed unnecessary check for redo log files at the end of backup in case the redo archive was enabled.
PXB-2972: An error after the redo apply phase caused an assertion failure instead of a graceful exit.
PXB-2977: Fixed the security vulnerability CVE-2022-25834.
PXB-2998: Due to a server bug in the upgrade, the server leaves the dictionary tables in a temporary schema instead of the
mysqlschema. Percona Xtrabackup can now successfully prepare these backup directories.PXB-2999: Removed the warning message that MLOG_INDEX_LOAD redo log record was found.
PXB-3022: Reduced the memory required for
"},{"location":"release-notes/8.0/8.0.32-26.0.html#useful-links","title":"Useful links","text":"--preparephase.The Percona XtraBackup GitHub location
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.33-27.0.html","title":"Percona XtraBackup 8.0.33-27 (2023-05-25)","text":"Release date May 25, 2023 Install instructions Install Percona XtraBackupPercona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.
"},{"location":"release-notes/8.0/8.0.33-27.0.html#release-highlights","title":"Release highlights","text":"This release merges the MySQL 8.0.33 code base. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
"},{"location":"release-notes/8.0/8.0.33-27.0.html#bug-fix","title":"Bug fix","text":"PXB-3026:
"},{"location":"release-notes/8.0/8.0.33-27.0.html#useful-links","title":"Useful links","text":"DWITH_NDB=OFFoption did not turn off theDWITH_NDB_CLUSTER_STORAGE_ENGINEand compiledNDBfiles.The Percona XtraBackup GitHub location
Download product binaries, packages, and tarballs at Percona Product Downloads
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.33-28.0.html","title":"Percona XtraBackup 8.0.33-28 (2023-07-19)","text":"Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.
"},{"location":"release-notes/8.0/8.0.33-28.0.html#release-highlights","title":"Release highlights","text":"This release includes improvements, features, and bug fixes. Percona has implemented a two-stage release process for each version. The first release primarily ensures compatibility with the latest MySQL version, to help those customers who need an updated version of Percona XtraBackup as soon as possible. The second release contains additional bug fixes and any improvements or new features.
Percona XtraBackup 8.0.33-28 implements a new data sink that uses the first in, first out (FIF0) method. With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to object storage. This feature is in tech preview.
Percona XtraBackup 8.0.33-28 implements the Dictionary cache feature. Dictionary cache improves the performance of xtrabackup
"},{"location":"release-notes/8.0/8.0.33-28.0.html#new-feature","title":"New feature","text":"--prepareoperation.PXB-2514: Implements a new data sink that uses
"},{"location":"release-notes/8.0/8.0.33-28.0.html#improvements","title":"Improvements","text":"FIFO(named pipes).-
PXB-2955: Implements dictionary cache.
-
PXB-3033: Executes
STOP SLAVEconditionally depending on--lock-ddl. -
PXB-3066: Fixes compilation issues on Debian 12.
-
PXB-2993: Makes Percona XtraBackup compatible with procps-4.
-
PXB-2811: Fix for several Percona XtraBackup compilation warnings.
-
PXB-3075: Percona XtraBackup 8.0.33 could not prepare backups from 8.0.29 and previous versions.
Percona XtraBackup 8.0.33-28 supports Debian 12.
"},{"location":"release-notes/8.0/8.0.33-28.0.html#useful-links","title":"Useful links","text":"Install Percona XtraBackup 8.0
The Percona XtraBackup GitHub location
Download product binaries, packages, and tarballs at Percona Product Downloads
Contribute to the documentation
For training, contact Percona Training - Start learning now.
"},{"location":"release-notes/8.0/8.0.4.html","title":"Percona XtraBackup 8.0.4","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.4 on December 10, 2018. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
This release of Percona Xtrabackup is a General Availability release ready for use in a production environment.
"},{"location":"release-notes/8.0/8.0.4.html#please-note-the-following-about-this-release","title":"Please note the following about this release:","text":"-
The deprecated innobackupex has been removed. Use the xtrabackup command to back up your instances:
$ xtrabackup --backup --target-dir=/data/backup -
When migrating from earlier database server versions, backup and restore and using Percona XtraBackup 2.4 and then use
mysql_upgradefrom MySQL 8.0.x -
If using
yumoraptrepositories to install Percona XtraBackup 8.0.4, ensure that you have enabled the new tools repository. You can do this with the percona-release enable tools release command and then install the percona-xtrabackup-80 package.
All Percona software is open-source and free. We are grateful to the community for the invaluable contributions to Percona XtraBackup. We would especially like to highlight the input of Alexey Kopytov who has been actively offering improvements and submitting bug reports for Percona XtraBackup.
"},{"location":"release-notes/8.0/8.0.4.html#new-features","title":"New Features","text":"- Percona XtraBackup 8.0.4 is based on MySQL 8.0.13 and fully supports Percona Server for MySQL 8.0 series and MySQL 8.0 series.
-
PXB-1699: xtrabackup \u2013prepare could fail on backups of MySQL\u00a08.0.13 databases
-
PXB-1704: xtrabackup \u2013prepare could hang while performing insert buffer merge
-
PXB-1668: When the \u2013throttle option was used, the applied value was different from the one specified by the user (off by one error)
-
PXB-1679: PXB could crash when
ALTER TABLE ... TRUNCATE PARTITIONcommand was run during a backup without locking DDL
Percona is glad to announce the release of Percona XtraBackup 8.0.5 on March 4, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
Percona XtraBackup 8.0.5 introduces the support of undo tablespaces created using the new syntax (
CREATE UNDO TABLESPACE) available since MySQL 8.0.14. Percona XtraBackup also supports the binary log encryption introduced in MySQL 8.0.14.Two new options were added to xbstream. Use the
--decompressoption with xbstream to decompress individual qpress files. With the--decompress-threadsoption, specify the number of threads to apply when decompressing. Thanks to Rauli Ikonen for this contribution.This release of Percona XtraBackup is a General Availability release ready for use in a production environment.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.5.html#please-note-the-following-about-this-release","title":"Please note the following about this release:","text":"-
The deprecated innobackupex has been removed. Use the xtrabackup command to back up your instances:
$ xtrabackup --backup --target-dir=/data/backup -
When migrating from earlier database server versions, backup and restore and using Percona XtraBackup 2.4 and then use
mysql_upgradefrom MySQL 8.0.x -
If using
yumoraptrepositories to install Percona Xtrabackup 8.0.5, ensure that you have enabled the new tools repository. You can do this with the percona-release enable tools release command and then install the percona-xtrabackup-80 package.
-
PXB-1548: Percona XtraBackup enables updating the
ib_buffer_poolfile with the latest pages present in the buffer pool using the--dump-innodb-buffer-pooloption. Thanks to Marcelo Altmann for contribution. -
PXB-1768: Added support for undo tablespaces created with the new MySQL 8.0.14 syntax.
-
PXB-1781: Added support for binary log encryption introduced in MySQL 8.0.14.
-
PXB-1797: For xbstream, two new options were added. The
--decompressoption enables xbstream to decompress individual qpress files. The--decompress-threadsoption controls the number of threads to apply when decompressing. Thanks to Rauli Ikonen for this contribution.
-
Using
--lock-ddl-per-tablecaused the server to scan all records of partitioned tables which could lead to the \u201cout of memory\u201d error. Bugs fixed PXB-1691 and PXB-1698. -
When Percona XtraBackup was started run with the
--slave-info, incorrect coordinates were written to the xtrabackup_slave_info file.. Bug fixed PXB-1737 -
Percona XtraBackup could crash at the prepare stage when making an incremental backup if the variable
innodb-rollback-segmentswas changed after starting the MySQL Server. Bug fixed PXB-1785. -
The full backup could fail when Percona Server for MySQL was started with the
--innodb-encrypt-tablesparameter. Bug fixed PXB-1793.
Other bugs fixed: PXB-1632, PXB-1715, PXB-1770, PXB-1771, PXB-1773.
"},{"location":"release-notes/8.0/8.0.6.html","title":"Percona XtraBackup 8.0.6","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.6 on May 9, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
In version 8.0.6, Percona XtraBackup introduces the support of the MyRocks storage engine with Percona Server for MySQL version 8.0.15-6 or higher.
Percona XtraBackup 8.0.6 enables saving backups to an Amazon S3, MinIO, and Google Cloud Storage (using interoperability mode) when using xbcloud. The following example demonstrates how to use an Amazon S3 storage to make a full backup:
"},{"location":"release-notes/8.0/8.0.6.html#new-features","title":"New Features","text":"$ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='s3.amazonaws.com' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n${date -I}-full_backup\n\nAll *Percona* software is open-source and free\n-
The MyRocks storage engine is now supported with Percona XtraBackup. More information in PXB-1754.
-
Amazon S3 is now supported in xbcloud. More information in PXB-1813.
-
Percona XtraBackup could fail to restore the undo tablespace created during or before incremental backup. Bug fixed PXB-1780.
-
A backup could fail if
log_bin_indexwas defined inmy.cnf. Bug fixed PXB-1801. -
When the row format was changed during the backup, xtrabackup could crash during the incremental prepare stage. Bug fixed PXB-1824.
-
During the
preparephase, Percona XtraBackup could freeze and never finish execution. Bug fixed PXB-1819. -
Percona XtraBackup could crash during the
preparestage when making a backup of a host running MySQL Server v8.0.16. Bug fixed PXB-1839.
Other bugs fixed: PXB-1809, PXB-1810, PXB-1832, PXB-1837.
"},{"location":"release-notes/8.0/8.0.7.html","title":"Percona XtraBackup 8.0.7","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.7 on August 7, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
In release 8.0.7, Percona XtraBackup enables making backups of databases that contain the encrypted system tablespace. Encrypted mysql tablespace is now also supported.
Percona XtraBackup 8.0.7 implements the support of the
lz4compression algorithm so that you could make compressed backups using lz4 (\u2013compress=lz4) in addition to the default quicklz method.All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.7.html#new-features-and-improvements","title":"New Features and Improvements","text":"-
Add support of the system tablespace encryption. More information in PXB-1649.
-
Implemented the support of the lz4 compression algorithm. More information in PXB-1857.
-
When the encrypted tablespaces feature was enabled, encrypted and compressed tables were not usable on the joiner node (Percona XtraDB Cluster) via SST (State Snapshot Transfer) with the xtrabackup-v2 method. Bug fixed PXB-1867.
-
xbclouddid not update date related fields of the HTTP header when retrying a request. Bug fixed PXB-1874. -
xbclouddid not retry to send the request after receiving the HTTP 408 error (request timeout). Bug fixed PXB-1875. -
xtrabackupdid not accept decimal fractions as values of theinnodb_max_dirty_pages_pctoption. Bug fixed PXB-1807. -
If the user tried to merge an already prepared incremental backup, a misleading error was produced without informing that incremental backups may not be used twice. Bug fixed PXB-1862.
Other bugs fixed: PXB-1493, PXB-1557, PXB-1887, PXB-1870, PXB-1879, PXB-1901.
"},{"location":"release-notes/8.0/8.0.8.html","title":"Percona XtraBackup 8.0.8","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.8 on November 21, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.8.html#new-features-and-improvements","title":"New Features and Improvements","text":"-
Support log archiving feature in PXB 8.0. More information in PXB-1912 and in the Redo Log section of MySQL documentation
-
For the MyRocks storage engine, support the creation of renewable checkpoints (controlled via
--rocksdb-checkpoint-max-ageand--rocksdb-checkpoint-max-count) to minimize the amount of binary logs to apply after the backup was completed. Using renewable checkpoints, Percona XtraBackup only copies the SST files that were created after the previous checkpoint. More information in PXB-1915. -
Two options (\u2013-backup-lock-timeout and -\u2013backup-lock-retry-count) were added to enable the configuring of the timeout for acquiring metadata locks in
FLUSH TABLES WITH READ LOCK,LOCK TABLE FOR BACKUP, andLOCK BINLOG FOR BACKUPstatements. More information in PXB-1914
-
An encrypted table could not be restored when
ADD INDEXorDROP INDEXcommands had been run on the table. Bug fixed PXB-1905 -
In some cases
xtrabackup --preparecould fail to decrypt a table but reported that the operation completed ok. Bug fixed PXB-1936 -
xtrabackup --move-backdid not complete successfully when the encrypted binlog file. Bug fixed PXB-1937. -
Percona XtraBackup could crash during the prepare stage when making an incremental
backup when a multi valued index was being added or dropped for JSON data. Bug fixed PXB-1913.
Other bugs fixed: PXB-1928, PXB-1938, PXB-1951, PXB-1953, PXB-1954.
"},{"location":"release-notes/8.0/8.0.9.html","title":"Percona XtraBackup 8.0.9","text":"Percona is glad to announce the release of Percona XtraBackup 8.0.9 on December 16, 2019. Downloads are available from our download site and from apt and yum repositories.
Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, it drives down backup costs while providing unique features for MySQL backups.
All Percona software is open-source and free.
"},{"location":"release-notes/8.0/8.0.9.html#bugs-fixed","title":"Bugs Fixed","text":"- Sometime between December 3rd and December 10th, a change was introduced in that caused an incompatibility with our Percona XtraBackup
xbcloudutility. Bug fixed PXB-1978.
Stream to an object storage
The streaming capacity for XtraBackup using STDOUT is 1.8G. This capacity is sufficient for streaming data using Wide Area Network (WAN) into, for example, Amazon Web Services or Google Cloud Platform.
-Starting with Percona XtraBackup 8.0.33-28, when you stream backups to Amazon S3 compatible storage using LAN with streaming capacity of 10Gbps, XtraBackup can use multiple FIFO streams to stream the backups faster.
-XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then multiple FIFO files are created to store the data from XtraBackup. Each XtraBackup copy thread writes the data chunks to a specific FIFO file. Xbcloud reads from the FIFO streams and uploads data to an object storage using async request. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).
+Starting with Percona XtraBackup 8.0.33-28, when you stream backups to Amazon S3 compatible storage using LAN with a streaming capacity of 10Gbps, XtraBackup can use multiple FIFO streams to stream the backups faster.
+XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then multiple FIFO files are created to store the data from XtraBackup. Each XtraBackup copy thread writes the data chunks to a specific FIFO file. Xbcloud reads from the FIFO streams and uploads data to an object storage using an async request. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).
Performance improvement¶
-Consider an example of using FIFO data sink compared to STDOUT method.
+Consider an example of using a FIFO data sink compared to the STDOUT method.
The database has 1TB of data in multiple tables. The link speed between the source server and destination server using MinIO is ~ 9.2 Gbps.
Both STDOUT and FIFO data sink scenarios push 1TB of data from two servers.
For the FIFO data sink we configure 8 parallel streams with
--fifo-streams=8both for XtraBackup and xbcloud.The results are the following:
- The
STDOUTmethod takes 01:25:24 to push 1TB of data using 239 MBps (1.8 Gbps).
- - The
FIFOmethod, using 8 streams, takes 00:16:01 to push 1TB of data using 1.15 Gbps (9.2 Gbps).
+ - The
FIFOmethod, using 8 streams, takes 00:16:01 to push 1TB of data using 1.15 GBps (9.2 Gbps).
-