From cf6e7d2e60c00d0b2b5be2012e6f8a2181f8df1a Mon Sep 17 00:00:00 2001 From: GitHub Action Date: Fri, 3 Nov 2023 11:57:04 +0000 Subject: [PATCH] Deployed e2a64ee9 to innovation-release with MkDocs 1.5.3 and mike 2.0.0 --- innovation-release/release-notes/8.1.0-1.html | 4 +- innovation-release/search/search_index.json | 2 +- innovation-release/sitemap.xml | 172 +++++++++--------- innovation-release/sitemap.xml.gz | Bin 1008 -> 1008 bytes versions.json | 18 +- 5 files changed, 106 insertions(+), 90 deletions(-) diff --git a/innovation-release/release-notes/8.1.0-1.html b/innovation-release/release-notes/8.1.0-1.html index 4f169343f..f7fba517e 100644 --- a/innovation-release/release-notes/8.1.0-1.html +++ b/innovation-release/release-notes/8.1.0-1.html @@ -3440,7 +3440,7 @@

Release highlightsImprovement

Deprecated or removed

The --stats mode of operation for the xtrabackup binary has been removed.

@@ -3466,7 +3466,7 @@

Get expert help Last update: - 2023-11-02 + 2023-11-03 diff --git a/innovation-release/search/search_index.json b/innovation-release/search/search_index.json index 5f6b1a488..7b0eccd1b 100644 --- a/innovation-release/search/search_index.json +++ b/innovation-release/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-,:!=\\[\\]()\"`/]+|\\.(?!\\d)|&[lg]t;|(?!\\b)(?=[A-Z][a-z])","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Percona XtraBackup 8.1 Documentation","text":"

This documentation is for the latest release: Percona XtraBackup 8.1 (Release Notes). This is an Innovation release. This type of release is only supported for a short time and is designed to be used in an environment with fast upgrade cycles. Developers and DBAs are exposed to the latest features and improvements.

Percona XtraBackup is an open source hot backup utility for MySQL-based servers that keep your database fully available during planned maintenance windows.

Whether it is a 24x7 highly loaded server or a low-transaction-volume Percona XtraBackup is designed to make backups seamless without disrupting the server\u2019s performance in a production environment. Percona XtraBackup (PXB) is a 100% open source backup solution with commercial support available for organizations who want to benefit from comprehensive, responsive, and cost-flexible database support for MySQL.

This is an Innovation release. This type of release is only supported for a short time and is designed to be used in an environment with fast upgrade cycles. Developers and DBAs are exposed to the latest features and improvements.

"},{"location":"index.html#supported-storage-engines","title":"Supported storage engines","text":"

Percona XtraBackup can back up data from InnoDB, XtraDB, MyISAM, MyRocks tables on MySQL 8.1 servers and Percona Server for MySQL with XtraDB, Percona Server for MySQL 8.1, and Percona XtraDB Cluster 8.1.

Percona XtraBackup 8.1 supports the MyRocks storage engine. An incremental backup on the MyRocks storage engine does not determine if an earlier full or incremental backup contains duplicate files. Percona XtraBackup copies all MyRocks files each time it takes a backup.

"},{"location":"index.html#limitations","title":"Limitations","text":"

Percona XtraBackup 8.1 does not support making backups of databases created in versions before 8.1 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster.

"},{"location":"index.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"404.html","title":"404 - Not Found","text":"

We can\u2019t find the page you are looking for. Try using the Search or return to the homepage.

"},{"location":"404.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"about-xtrabackup.html","title":"About Percona XtraBackup","text":"

Percona XtraBackup is the world\u2019s only open source, free MySQL hot backup software that performs non-blocking backups for InnoDB and XtraDB databases.

Percona XtraBackup has the following benefits:

  • Complete a backup quickly and reliably

  • Process transactions uninterrupted during a back up

  • Save on disk space and network bandwidth

  • Verify backup automatically

Percona XtraBackup makes hot backups for Percona Server for MySQL and MySQL, takes streaming, compressed, and incremental MySQL backups, and supports encryption.

Percona\u2019s enterprise-grade commercial MySQL Support contracts include support for Percona XtraBackup. We recommend support for critical production deployments.

"},{"location":"about-xtrabackup.html#supported-storage-engines","title":"Supported storage engines","text":"

Percona XtraBackup can back up data from InnoDB, XtraDB, MyISAM, and MyRocks tables on MySQL 8.1 servers as well as Percona Server for MySQL with XtraDB, and Percona XtraDB Cluster.

Percona XtraBackup supports the MyRocks storage engine. An incremental backup on the MyRocks storage engine does not determine if an earlier full or incremental backup contains the same files. Percona XtraBackup copies all MyRocks files each time it takes a backup. Percona XtraBackup does not support the TokuDB storage engine.

"},{"location":"about-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"accelerate-backup-process.html","title":"Accelerate backup process","text":""},{"location":"accelerate-backup-process.html#accelerate-the-backup-process","title":"Accelerate the backup process","text":""},{"location":"accelerate-backup-process.html#copy-with-the-parallel-and-compress-threads-options","title":"Copy with the --parallel and --compress-threads options","text":"

When making a local or streaming backup with xbstream binary, multiple files can be copied at the same time when using the --parallel option. This option specifies the number of threads created by xtrabackup to copy data files.

To take advantage of this option either the multiple tablespaces option must be enabled (innodb_file_per_table) or the shared tablespace must be stored in multiple ibdata files with the innodb_data_file_path option. Having multiple files for the database (or splitting one into many) doesn\u2019t have a measurable impact on performance.

As this feature is implemented at the file level, concurrent file transfer can sometimes increase I/O throughput when doing a backup on highly fragmented data files, due to the overlap of a greater number of random read requests. You should consider tuning the filesystem also to obtain the maximum performance (e.g. checking fragmentation).

If the data is stored on a single file, this option has no effect.

To use this feature, simply add the option to a local backup, for example:

$ xtrabackup --backup --parallel=4 --target-dir=/path/to/backup\n

By using the xbstream in streaming backups, you can additionally speed up the compression process with the --compress-threads option. This option specifies the number of threads created by xtrabackup for for parallel data compression. The default value for this option is 1.

To use this feature, simply add the option to a local backup, for example:

$ xtrabackup --backup --stream=xbstream --compress --compress-threads=4 --target-dir=./ > backup.xbstream\n

Before applying logs, compressed files will need to be uncompressed.

"},{"location":"accelerate-backup-process.html#the-rsync-option","title":"The --rsync option","text":"

In order to speed up the backup process and to minimize the time FLUSH TABLES WITH READ LOCK is blocking the writes, the option --rsync should be used. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. xtrabackup will call the rsync twice, once before the FLUSH TABLES WITH READ LOCK and once during to minimize the time the read lock is being held. During the second rsync call, it will only synchronize the changes to non-transactional data (if any) since the first call performed before the FLUSH TABLES WITH READ LOCK. Note that Percona XtraBackup will use Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK.

Percona XtraBackup uses these locks automatically to copy non-InnoDB data to avoid blocking Data manipulation language (DML) queries that modify InnoDB tables.

Note

This option cannot be used together with the --stream option.

"},{"location":"accelerate-backup-process.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"apt-download-deb.html","title":"Install using downloaded DEB packages","text":"

Download DEB packages of the desired series for your architecture from Percona Product Downloads. This method requires you to resolve all dependencies and install any missing packages.

The following example downloads Percona XtraBackup 8.1.0-1 release package for Ubuntu 20.04:

$ wget https://downloads.percona.com/downloads/Percona-XtraBackup-LATEST/Percona-XtraBackup-8.1.0-1/binary/debian/focal/x86_64/percona-xtrabackup-81_8.1.0-1.focal_amd64.deb\n

Install Percona XtraBackup by using dpkg. Run this command as root or use the sudo command:

$ sudo dpkg -i percona-xtrabackup-81_8.1.0-1.focal_amd64.deb\n
"},{"location":"apt-download-deb.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"apt-files.html","title":"Files in the DEB package","text":"

The following tables show what data each DEB package contains.

Package Contains percona-xtrabackup-81 The latest Percona XtraBackup GA binaries and associated files percona-xtrabackup-dbg-81 The debug symbols for binaries in percona-xtrabackup-81 percona-xtrabackup-test-81 The test suite for Percona XtraBackup percona-xtrabackup The older version of the Percona XtraBackup"},{"location":"apt-files.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"apt-pinning.html","title":"Apt pinning packages","text":"

In some cases you might need to pin the selected packages to avoid the upgrades from the distribution repositories.

The pinning takes place in the preference file. To pin a package, set the Pin-Priority to higher numbers.

Make a new file /etc/apt/preferences.d/00percona.pref. For example, add the following to the preference file:

Package:\nPin: release o=Percona Development Team\nPin-Priority: 1001\n

For more information about the pinning, check the official debian wiki.

"},{"location":"apt-pinning.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"apt-repo.html","title":"Install with an APT repository","text":"

Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page.

Specific information on the supported platforms, products, and versions is described in Percona Release Lifecycle Overview.

"},{"location":"apt-repo.html#install-percona-xtrabackup-through-percona-release","title":"Install Percona XtraBackup through percona-release","text":"

Percona XtraBackup, like many other Percona products, is installed with the percona-release package configuration tool.

  1. Download a DEB package for percona-release the repository packages from Percona web:

    $ wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb\n

  2. Install the downloaded package with dpkg. To do that, run the following commands as root or with sudo: dpkg -i percona-release_latest.$(lsb_release -sc)_all.deb

    Once you install this package the Percona repositories should be added. You can check the repository setup in the /etc/apt/sources.list.d/percona-release.list file.

  3. Enable the repository: percona-release enable-only tools release

    If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the tools repository: percona-release enable-only tools.

  4. Refresh the local cache to update the package information:

    $ sudo apt update\n

  5. Install the percona-xtrabackup-81 package:

    $ sudo apt install percona-xtrabackup-81\n

  6. To decompress backups made using LZ4 or ZSTD compression algorithm, install the corresponding package:

    Install the lz4 packageInstall the zstd package 
    $ sudo apt install lz4\n
    $ sudo apt install zstd\n

Note

For AppArmor profile information, see Working with AppArmor.

See also

To install Percona XtraBackup using downloaded deb packages, see Install Percona XtraBackup 8.1.

To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup 8.1

"},{"location":"apt-repo.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"apt-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.1 on Debian and Ubuntu","text":"

To completely uninstall Percona XtraBackup, remove all the installed packages:

$ sudo apt remove percona-xtrabackup-81\n
"},{"location":"apt-uninstall-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"binaries-overview.html","title":"Percona XtraBackup binaries overview","text":"

Percona XtraBackup contains a set of the following binaries:

  • xtrabackup - a compiled C binary that provides functionality to backup a whole MySQL database instance with MyISAM, InnoDB, and XtraDB tables.

  • xbcrypt - a utility used for encrypting and decrypting backup files.

  • xbstream - a utility that allows streaming and extracting files to or from the xbstream format.

  • xbcloud - a utility used for downloading and uploading full or part of xbstream archive from or to cloud.

The recommended way to take the backup is by using the xtrabackup script. For more information on script options, see xtrabackup.

"},{"location":"binaries-overview.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"binary-tarball-names.html","title":"Binary tarball file names available","text":"

Download the binary tarballs from Percona Product Downloads.

The following table lists the tarball types available in Linux - Generic. Select the Percona XtraBackup 8.1 version, the software or the operating system, and the type of tarball for your installation. Binary tarballs support all distributions.

After you have downloaded the binary tarballs, extract the tarball in the file location of your choice.

Type Name Operating systems Description Full percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.12.tar.gz Built for CentOS 6 Contains binary files, libraries, test files, and debug symbols Minimal percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.12-minimal.tar.gz Built for CentOS 6 Contains binary files, and libraries but does not include test files, or debug symbols Full percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.17.tar.gz Compatible with any operating system except for CentOS 6 Contains binary files, libraries, test files, and debug symbols Minimal percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.17-minimal.tar.gz Compatible with any operating system except for CentOS 6 Contains binary files, and libraries but does not include test files, or debug symbols

Select a different software, such as Ubuntu 20.04 (Focal Fossa), for a tarball for that operating system. You can download the packages together or separately.

"},{"location":"binary-tarball-names.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"binary-tarball.html","title":"Install from a binary tarball","text":"

Binary tarballs contain precompiled executable files, libraries, and other dependencies and are compressed tar archives. Extract the binary tarballs to any path.

Download the binary tarballs from the Linux - Generic section on Percona Product Downloads.

The following example downloads the a tarball:

$ wget https://downloads.percona.com/downloads/Percona-XtraBackup-8.1/Percona-XtraBackup-8.1.0-1/binary/tarball/percona-xtrabackup-8.1.0-1-Linux-x86_64.glibc2.17.tar.gz\n

The output displays the following information:

Expected output 
--2023-10-20 05:56:30--  https://downloads.percona.com/downloads/Percona-XtraBackup-8.1/Percona-XtraBackup-8.1.0-1/binary/tarball/percona-xtrabackup-8.1.0-1-Linux-x86_64.glibc2.17.tar.gz\nResolving downloads.percona.com (downloads.percona.com)... 2604:2dc0:200:69f::2, 147.135.54.159\nConnecting to downloads.percona.com (downloads.percona.com)|2604:2dc0:200:69f::2|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 470358258 (449M) [application/gzip]\nSaving to: \u2018percona-xtrabackup-8.1.0-1-Linux-x86_64.glibc2.17.tar.gz\u2019\n\npercona-xtrabackup   0%[                       ]   3.17M  1.54MB/s\n
"},{"location":"binary-tarball.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"compile-xtrabackup.html","title":"Compile xtrabackup","text":""},{"location":"compile-xtrabackup.html#compile-and-install-from-source","title":"Compile and install from source","text":"

The following instructions install Percona XtraBackup 8.1.

"},{"location":"compile-xtrabackup.html#1-install-percona-xtrabackup-from-the-git-source-tree","title":"1. Install Percona XtraBackup from the Git Source Tree","text":"

Percona uses the Github revision control system for development. To build the latest Percona Server for MySQL from the source tree, you will need git installed on your system.

You can now fetch the latest Percona XtraBackup 8.1 sources:

$ git clone https://github.com/percona/percona-xtrabackup.git\n$ cd percona-xtrabackup\n$ git checkout trunk\n$ git submodule update --init --recursive\n
"},{"location":"compile-xtrabackup.html#2-installation-prerequisites","title":"2. Installation prerequisites","text":"

The following packages and tools must be installed to compile Percona XtraBackup from source. These might vary from system to system.

Important

To build **Percona XtraBackup 8.1 from source, you must use cmake version 3. To check which version is currently installed, run cmake --version at a command prompt. If the version is not 3, install cmake3.

This cmake version may be available in your distribution as a separate package cmake3. For more information, see cmake.org.

Debian or Ubuntu using aptCentOS or Red Hat using yum 
sudo apt install bison pkg-config cmake devscripts debconf \\\ndebhelper automake bison ca-certificates libprocps-dev \\\nlibcurl4-openssl-dev cmake debhelper libaio-dev \\\nlibncurses-dev libssl-dev libtool libz-dev libgcrypt-dev libev-dev libprocps-dev \\\nlsb-release build-essential rsync libdbd-mysql-perl \\\nlibnuma1 socat librtmp-dev libtinfo5 vim-common \\\nliblz4-tool liblz4-1 liblz4-dev zstd python-docutils\n

To install the man pages, install the python3-sphinx package first:

$ sudo apt install python3-sphinx\n

Percona Xtrabackup requires GCC version 5.3 or higher. If the version of GCC installed on your system is lower then you may need to install and enable the Developer Toolset on RPM-based distributions to make sure that you use the latest GCC compiler and development tools. Then, install cmake and other dependencies:

$ sudo yum install cmake openssl-devel libaio libaio-devel automake autoconf \\\nbison libtool ncurses-devel libgcrypt-devel libev-devel libcurl-devel zlib-devel \\\nzstd vim-common procps-ng-devel\n

To install the man pages, install the python3-sphinx package first:

$ sudo yum install python3-sphinx\n
"},{"location":"compile-xtrabackup.html#3-generate-the-build-pipeline","title":"3. Generate the build pipeline","text":"

At this step, you have cmake run the commands in the CMakeList.txt file to generate the build pipeline, i.e., a native build environment that will be used to compile the source code).

  1. Change to the directory where you cloned the Percona XtraBackup repository

    $ cd percona-xtrabackup\n

  2. Create a directory to store the compiled files and then change to that directory:

    $ mkdir build\n$ cd build\n

  3. Run cmake or cmake3. In either case, the options you need to use are the same.

Note

You can build Percona XtraBackup with man pages but this requires python-sphinx package which isn\u2019t available from that main repositories for every distribution. If you installed the python-sphinx package you need to remove the -DWITH_MAN_PAGES=OFF from previous command.

$ cmake -DWITH_BOOST=PATH-TO-BOOST-LIBRARY -DDOWNLOAD_BOOST=ON \\\n-DBUILD_CONFIG=xtrabackup_release -DWITH_MAN_PAGES=OFF -B ..\n
"},{"location":"compile-xtrabackup.html#parameter-information","title":"Parameter Information","text":"Parameter Description -DWITH_BOOST For the -DWITH_BOOST parameter, specify the name of a directory to download the boost library to. This directory is created automatically in your current directory. -DWITH_MAN_PAGES To build Percona XtraBackup man pages, use ON or remove this parameter from the command line (it is ON by default). To install the man pages, install the python3-sphinx package first. -B (\u2013build) Percona XtraBackup is configured to forbid generating the build pipeline for make in the same directory where you store your sources. The -B parameter refers to the directory that contains the source code. In this example, we use the relative path to the parent directory (..).

Important

CMake Error at CMakeLists.txt:367 (MESSAGE): Please do not build in-source. Out-of source builds are highly recommended: you can have multiple builds for the same source, and there is an easy way to do cleanup, simply remove the build directory (note that \u2018make clean\u2019 or \u2018make distclean\u2019 does not work)

You can force in-source build by invoking cmake with -DFORCE_INSOURCE_BUILD=1.

"},{"location":"compile-xtrabackup.html#4-compile-the-source-code","title":"4. Compile the source code","text":"

To compile the source code in your build directory, use the make command.

  1. Change to the build directory (created at Step 2: Generating the build pipeline).

  2. Run the make command. This command may take a long time to complete.

    $ make\n

    To use all CPU threads and make compilation faster please use:

    $ make -j$(nproc --all)\n
"},{"location":"compile-xtrabackup.html#5-install-on-the-target-system","title":"5. Install on the target system","text":"

The following command installs all Percona XtraBackup binaries xtrabackup and tests to default location on the target system: /usr/local/xtrabackup.

Run make install to install Percona XtraBackup to the default location.

$ sudo make install\n
"},{"location":"compile-xtrabackup.html#install-to-a-non-default-location","title":"Install to a non-default location","text":"

You may use the DESTDIR parameter with make install to install Percona XtraBackup to another location. Make sure that the effective user is able to write to the destination you choose.

$ sudo make DESTDIR=<DIR_NAME> install\n

In fact, the destination directory is determined by the installation layout (-DINSTALL_LAYOUT) that cmake applies (see Step 2: Generating the build pipeline). In addition to the installation directory, this parameter controls a number of other destinations that you can adjust for your system.

By default, this parameter is set to STANDALONE, which implies the installation directory to be /usr/local/xtrabackup.

See also

MySQL Documentation: -DINSTALL_LAYOUT

"},{"location":"compile-xtrabackup.html#6-run-percona-xtrabackup","title":"6. Run Percona XtraBackup","text":"

After Percona XtraBackup is installed on your system, you may run it by using the full path to the xtrabackup command:

$ /usr/local/xtrabackup/bin/xtrabackup\n

Update your PATH environment variable if you would like to use the command on the command line directly.

$# Setting $PATH on the command line\n$ PATH=$PATH:/usr/local/xtrabackup/bin/xtrabackup\n\n$# Run xtrabackup directly\n$ xtrabackup\n

Alternatively, you may consider placing a soft link (using ln -s) to one of the locations listed in your PATH environment variable.

To view the documentation with man, update the MANPATH variable.

"},{"location":"compile-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"configure-xtrabackup.html","title":"Configure xtrabackup","text":"

All the xtrabackup configuration is done through options, which behave exactly like standard MySQL program options: they can be specified either at the command-line, or through a file such as /etc/my.cnf.

The xtrabackup binary reads the [mysqld] and [xtrabackup] sections from any configuration files, in that order. That is so that it can read its options from your existing MySQL installation, such as the datadir or some of the InnoDB options. If you want to override these, just specify them in the [xtrabackup] section, and because it is read later, it will take precedence.

You don\u2019t need to put any configuration in your my.cnf. You can specify the options on the command-line. Normally, the only thing you may find convenient to place in the [xtrabackup] section of your my.cnf file is the target_dir option. This options adds a default path to the directory for backups.

The following is an example:

[xtrabackup]\ntarget_dir = /data/backups/\n

This manual assumes you do not have any file-based configuration for xtrabackup and shows the command-line options.

Please see the option and variable reference for details on all the configuration options.

The xtrabackup binary does not accept exactly the same syntax in the my.cnf file as the mysqld server binary does. For historical reasons, the mysqld server binary accepts parameters with a --set-variable=<variable>=<value> syntax, which xtrabackup does not understand. If your my.cnf file has such configuration directives, you should rewrite them in the --variable=value syntax.

"},{"location":"configure-xtrabackup.html#system-configuration-and-nfs-volumes","title":"System configuration and NFS volumes","text":"

The xtrabackup tool requires no special configuration on most systems. However, the storage where the --target-dir is located must behave properly when fsync() is called. In particular, we have noticed that if you mount the NFS volume without the sync option the NFS volume does not sync the data. As a result, if you back up to an NFS volume mounted with the async option, and then try to prepare the backup from a different server that also mounts that volume, the data might appear to be corrupt. Use the sync mount option to avoid this issue.

"},{"location":"configure-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"copyright-and-licensing-information.html","title":"Copyright and licensing information","text":""},{"location":"copyright-and-licensing-information.html#documentation-licensing","title":"Documentation licensing","text":"

Percona XtraBackup documentation is (C)2009-2023 Percona LLC and/or its affiliates and is distributed under the Creative Commons Attribution 4.0 International License.

"},{"location":"copyright-and-licensing-information.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"create-compressed-backup.html","title":"Create a compressed backup","text":"

Percona XtraBackup supports compressed backups. To make a compressed backup, use the --compress option along with the --backup and --target-dir options. A local or streaming backup can be compressed or decompressed with xbstream.

By default, the --compress option uses the zstandard tool that you can install with the percona-release package configuration tool as follows:

$ sudo percona-release enable tools\n$ sudo apt update\n$ sudo apt install zstandard\n

Note

Enable the repository: percona-release enable-only tools release.

If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the tools repository: percona-release enable-only tools.

Percona XtraBackup supports the following compression algorithms:

"},{"location":"create-compressed-backup.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

The Zstandard (ZSTD) is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. ZSTD is the default compression algorithm for the --compress option.

To compress files using the ZSTD compression algorithm, use the --compress option:

$ xtrabackup --backup --compress --target-dir=/data/backup\n

The resulting files have the \\*.zst format.

You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.

$ xtrabackup \u2013backup \u2013compress \u2013compress-zstd-level=1 \u2013target-dir=/data/backup\n
"},{"location":"create-compressed-backup.html#lz4","title":"lz4","text":"

To compress files using the lz4 compression algorithm, set the --compress option to lz4:

$ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\n

The resulting files have the \\*.lz4 format.

If you want to speed up the compression you can use the parallel compression, which can be enabled with --compress-threads option. Following example will use four threads for compression:

$ xtrabackup --backup --compress --compress-threads=4 \\\n--target-dir=/data/compressed/\n
Expected output 
...\n170223 13:00:38 [01] Compressing ./test/sbtest1.frm to /tmp/compressed/test/sbtest1.frm.qp\n170223 13:00:38 [01]        ...done\n170223 13:00:38 [01] Compressing ./test/sbtest2.frm to /tmp/compressed/test/sbtest2.frm.qp\n170223 13:00:38 [01]        ...done\n...\n170223 13:00:39 [00] Compressing xtrabackup_info\n170223 13:00:39 [00]        ...done\nxtrabackup: Transaction log of lsn (9291934) to (9291934) was copied.\n170223 13:00:39 completed OK!\n
"},{"location":"create-compressed-backup.html#next-step","title":"Next step","text":"

Prepare the backup

"},{"location":"create-compressed-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"create-full-backup.html","title":"Create a full backup","text":"

To create a backup, run xtrabackup with the --backup option. You also must specify the --target-dir option, which is where the backup is stored. If the InnoDB data or log files are not stored in the same directory, you must specify their location. If the target directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup succeeds.

xtrabackup does not overwrite existing files. it will fail with operating system error 17, file exists.

The following command starts the process:

$ xtrabackup --backup --target-dir=/data/backups/\n

This operation stores the backup at /data/backups/. If you specify a relative path, the target directory is relative to the current directory.

During the backup process, the output shows the copied data files, and a log file thread that scans and copies from the log files.

The following is an example of the output:

Expected output 
160906 10:19:17 Finished backing up non-InnoDB tables and files\n160906 10:19:17 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...\nxtrabackup: The latest check point (for incremental): '62988944'\nxtrabackup: Stopping log copying thread.\n.160906 10:19:18 >> log scanned up to (137343534)\n160906 10:19:18 Executing UNLOCK TABLES\n160906 10:19:18 All tables unlocked\n160906 10:19:18 Backup created in directory '/data/backups/'\n160906 10:19:18 [00] Writing backup-my.cnf\n160906 10:19:18 [00]        ...done\n160906 10:19:18 [00] Writing xtrabackup_info\n160906 10:19:18 [00]        ...done\nxtrabackup: Transaction log of lsn (26970807) to (137343534) was copied.\n160906 10:19:18 completed OK!\n

The process ends with the following statement; the value of the <LSN> depends on your system:

$ xtrabackup: Transaction log of lsn (<LSN>) to (<LSN>) was copied.\n

Note

Log copying thread checks the transactional log every second to see if there were any new log records written that need to be copied, but there is a chance that the log copying thread might not be able to keep up with the amount of writes that go to the transactional logs, and will hit an error when the log records are overwritten before they could be read.

After the backup is finished, the target directory will contain files such as the following, assuming you have a single InnoDB table test.tbl1 and you are using MySQL\u2019s innodb_file_per_table option:

$ ls -lh /data/backups/\n

The result should look like this:

Expected output 
total 182M\ndrwx------  7 root root 4.0K Sep  6 10:19 .\ndrwxrwxrwt 11 root root 4.0K Sep  6 11:05 ..\n-rw-r-----  1 root root  387 Sep  6 10:19 backup-my.cnf\n-rw-r-----  1 root root  76M Sep  6 10:19 ibdata1\ndrwx------  2 root root 4.0K Sep  6 10:19 mysql\ndrwx------  2 root root 4.0K Sep  6 10:19 performance_schema\ndrwx------  2 root root 4.0K Sep  6 10:19 sbtest\ndrwx------  2 root root 4.0K Sep  6 10:19 test\ndrwx------  2 root root 4.0K Sep  6 10:19 world2\n-rw-r-----  1 root root  116 Sep  6 10:19 xtrabackup_checkpoints\n-rw-r-----  1 root root  433 Sep  6 10:19 xtrabackup_info\n-rw-r-----  1 root root 106M Sep  6 10:19 xtrabackup_logfile\n

The backup can take a long time, depending on how large the database is. It is safe to cancel at any time, because xtrabackup does not modify the database.

"},{"location":"create-full-backup.html#next-step","title":"Next step","text":"

Prepare the backup

"},{"location":"create-full-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"create-gtid-replica.html","title":"How to create a new (or repair a broken) GTID-based replica","text":"

Percona XtraBackup automatically stores the GTID value in the xtrabackup_binlog_info when doing the backup of MySQL and Percona Server for MySQL 8.1 with the GTID mode enabled. This information can be used to create a new (or repair a broken) GTID-based replica.

"},{"location":"create-gtid-replica.html#1-take-a-backup-from-any-server-on-the-replication-environment-source-or-replica","title":"1. Take a backup from any server on the replication environment, source or replica","text":"

The following command takes a backup and saves it in the /data/backups/$TIMESTAMP folder:

$ xtrabackup --backup --target-dir=/data/backups/\n

In the destination folder, there will be a file with the name xtrabackup_binlog_info. This file contains both binary log coordinates and the GTID information.

$ cat xtrabackup_binlog_info\n
The result could look like this:

Expected output 
mysql-bin.000002    1232        c777888a-b6df-11e2-a604-080027635ef5:1-4\n

That information is also printed by xtrabackup after taking the backup:

Expected output 
xtrabackup: MySQL binlog position: filename 'mysql-bin.000002', position 1232, GTID of the last change 'c777888a-b6df-11e2-a604-080027635ef5:1-4'\n
"},{"location":"create-gtid-replica.html#2-prepare-the-backup","title":"2. Prepare the backup","text":"

The backup will be prepared with the following command on the Source:

$ xtrabackup --prepare --target-dir=/data/backup\n

You need to select the path where your snapshot has been taken, for example /data/backups/2023-05-07_08-33-33. If everything is ok you should get the same OK message. Now, the transaction logs are applied to the data files, and new ones are created: your data files are ready to be used by the MySQL server.

"},{"location":"create-gtid-replica.html#3-move-the-backup-to-the-destination-server","title":"3. Move the backup to the destination server","text":"

Use rsync or scp to copy the data to the destination server. If you are synchronizing the data directly to the already running replica\u2019s data directory it is advised to stop the MySQL server there.

$ rsync -avprP -e ssh /path/to/backupdir/$TIMESTAMP NewSlave:/path/to/mysql/\n

After you copy the data over, make sure MySQL has proper permissions to access them.

$ chown mysql:mysql /path/to/mysql/datadir\n
"},{"location":"create-gtid-replica.html#4-configure-and-start-replication","title":"4. Configure and start replication","text":"

Set the gtid_purged variable to the GTID from xtrabackup_binlog_info. Then, update the information about the source node and, finally, start the replica.

Note

The example above is applicable to Percona XtraDB Cluster. The wsrep_on variable is set to 0 before resetting the source (RESET MASTER). The reason is that Percona XtraDB Cluster will not allow resetting the source if wsrep_on=1.

# Using the mysql shell\n > SET SESSION wsrep_on = 0;\n > RESET MASTER;\n > SET SESSION wsrep_on = 1;\n > SET GLOBAL gtid_purged='<gtid_string_found_in_xtrabackup_binlog_info>';\n > CHANGE REPLICATION SOURCE TO\n             SOURCE_HOST=\"$masterip\",\n             SOURCE_USER=\"repl\",\n             SOURCE_PASSWORD=\"$slavepass\",\n             SOURCE_AUTO_POSITION = 1;\n > START REPLICA;\n
"},{"location":"create-gtid-replica.html#5-check-the-replication-status","title":"5. Check the replication status","text":"

The following command returns the replica status:

mysql> SHOW REPLICA STATUS\\G\n
The results should be similar to the following:

Expected output 
[..]\nSlave_IO_Running: Yes\nSlave_SQL_Running: Yes\n[...]\nRetrieved_Gtid_Set: c777888a-b6df-11e2-a604-080027635ef5:5\nExecuted_Gtid_Set: c777888a-b6df-11e2-a604-080027635ef5:1-5\n

We can see that the replica has retrieved a new transaction with step 5, so transactions from 1 to 5 are already on the replica.

We have created a new replica in our GTID based replication environment.

"},{"location":"create-gtid-replica.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"create-incremental-backup.html","title":"Create an incremental backup","text":"

xtrabackup supports incremental backups, which means that they can copy only all the data that has changed since the last backup.

Note

Incremental backups on the MyRocks storage engine do not determine if an earlier full backup or incremental backup contains the same files. Percona XtraBackup copies all the MyRocks files each time it takes a backup.

You can perform many incremental backups between each full backup, so you can set up a backup process such as a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.

Incremental backups work because each InnoDB page contains a log sequence number, or LSN. The LSN is the system version number for the entire database. Each page\u2019s LSN shows how recently it was changed.

An incremental backup copies each page which LSN is newer than the previous incremental or full backup\u2019s LSN. An algorithm finds the pages that match the criteria. The algorithm reads the data pages and checks the page LSN.

"},{"location":"create-incremental-backup.html#create-an-incremental-backup_1","title":"Create an incremental backup","text":"

To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. Create the full backup with a following command:

$ xtrabackup --backup --target-dir=/data/backups/base\n

If you look at the xtrabackup_checkpoints file, you should see similar content depending on your LSN nuber:

Expected output 
backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 1626007\nlast_lsn = 1626007\ncompact = 0\nrecover_binlog_info = 1\n

Now that you have a full backup, you can make an incremental backup based on it. Use the following command:

$ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base\n

The /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 1626007. If you examine the xtrabackup_checkpoints file in this directory, you should see similar content to the following:

Expected output 
backup_type = incremental\nfrom_lsn = 1626007\nto_lsn = 4124244\nlast_lsn = 4124244\ncompact = 0\nrecover_binlog_info = 1\n

from_lsn is the starting LSN of the backup and for incremental it has to be the same as to_lsn (if it is the last checkpoint) of the previous/base backup.

It\u2019s now possible to use this directory as the base for yet another incremental backup:

$ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1\n

This folder also contains the xtrabackup_checkpoints:

Expected output 
backup_type = incremental\nfrom_lsn = 4124244\nto_lsn = 6938371\nlast_lsn = 7110572\ncompact = 0\nrecover_binlog_info = 1\n

Note

In this case you can see that there is a difference between the to_lsn (last checkpoint LSN) and last_lsn (last copied LSN), this means that there was some traffic on the server during the backup process.

"},{"location":"create-incremental-backup.html#next-step","title":"Next step","text":"

Prepare the backup

"},{"location":"create-incremental-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"create-individual-partition-backup.html","title":"Create an individual partitions backup","text":"

Percona XtraBackup lets you back up individual partitions because partitions are regular tables with specially formatted names. The only requirement for this feature is to enable the innodb_file_per_table option on the server.

There is one caveat about using this kind of backup: you can not copy back the prepared backup. Restoring partial backups should be done by importing the tables.

There are three ways of specifying which part of the whole data will be backed up: regular expressions ( \u2013tables), enumerating the tables in a file (\u2013tables) or providing a list of databases (\u2013databases).

The regular expression provided to this option will be matched against the fully qualified database name and table name, in the form of database-name.table-name.

If the partition 0 is not backed up, Percona XtraBackup cannot generate a .cfg file. MySQL 8.0 stores the table metadata in partition 0.

For example, this operation takes a back-up of the partition p4 from the table name located in the database imdb:

$ xtrabackup --tables=^imdb[.]name#p#p4 --backup\n

If partition 0 is not backed up, the following errors may occur:

The error message 
xtrabackup: export option not specified\nxtrabackup: error: cannot find dictionary record of table imdb/name#p#p4\n

Note that this option is passed to xtrabackup --tables and is matched against each table of each database, the directories of each database will be created even if they are empty.

"},{"location":"create-individual-partition-backup.html#next-step","title":"Next step","text":"

Prepare an individual partitions backup

"},{"location":"create-individual-partition-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"create-partial-backup.html","title":"Create a partial backup","text":"

xtrabackup supports taking partial backups when the innodb_file_per_table option is enabled.

Warning

Do not copy back the prepared backup.

Restoring partial backups should be done by importing the tables. We do not by using the \u2013copy-back option. This operation may lead to database inconsistencies.

We do not recommend running incremental backups after taking a partial backup.

The xtrabackup binary fails if you delete any of the matched or listed tables during the backup.

There are multiple ways of specifying which part of the whole data is backed up:

  • Use the --tables option to list the table names

  • Use the --tables-file option to list the tables in a file

  • Use the --databases option to list the databases

  • Use the --databases-file option to list the databases

The following examples assume a database named test that contains tables named t1 and t2.

\u2013-tables option-\u2013tables-file option--databases option--databases-file option

The first method involves the xtrabackup \u2013tables option. The option\u2019s value is a regular expression that is matched against the fully-qualified database name and table name using the databasename.tablename format.

To back up only tables in the test database, use the following command:

$ xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \\\n--tables=\"^test[.].*\"\n

To back up only the test.t1 table, use the following command:

$ xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \\\n--tables=\"^test[.]t1\"\n

The --tables-file option specifies a file that can contain multiple table names, one table name per line in the file. Only the tables named in the file will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching. The table names must be fully-qualified in databasename.tablename format.

$ echo \"mydatabase.mytable\" > /tmp/tables.txt\n$ xtrabackup --backup --tables-file=/tmp/tables.txt\n

The ` \u2013databases` option accepts a space-separated list of the databases and tables to backup in the databasename[.tablename] format. In addition to this list, make sure to specify the mysql, sys, and

performance_schema databases. These databases are required when restoring the databases using xtrabackup \u2013copy-back.

Note

Tables processed during the \u2013prepare step may also be added to the backup even if they are not explicitly listed by the parameter if they were created after the backup started.

$ xtrabackup --databases='mysql sys performance_schema test ...'\n

The \u2013databases-file option specifies a file that can contain multiple databases and tables in the databasename[.tablename] format, one element name per line in the file. Names are matched exactly, case-sensitive, with no pattern or regular expression matching.

Note

Tables processed during the \u2013prepare step may also be added to the backup even if they are not explicitly listed by the parameter if they were created after the backup started.

"},{"location":"create-partial-backup.html#next-step","title":"Next step","text":"

Prepare the backup

"},{"location":"create-partial-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"dictionary-cache.html","title":"Dictionary cache","text":"

Percona XtraBackup is based on how [crash recovery] works. Percona XtraBackup copies the InnoDB data files, which results in data that is internally inconsistent; then the prepare phase performs crash recovery on the files to make a consistent, usable database again.

The --prepare phase has the following operations:

  • Applies the redo log
  • Applies the undo log

As a physical operation, the changes from the redo log modifications are applied to a page. In the redo log operation, there is no concept of row or transaction. The redo apply operation does not make the database consistent with a transaction. An uncommitted transaction can be flushed or written to the redo log by the server. Percona XtraBackup applies changes recorded in the redo log.

Percona XtraBackup physically modifies a specific offset of a page within a tablespace (IBD file) when applying a redo log.

As a logical operation, Percona XtraBackup applies the undo log on a specific row. When the redo log completes, XtraBackup uses the undo log to roll back changes from uncommitted transactions during the backup.

"},{"location":"dictionary-cache.html#undo-log","title":"Undo log","text":"

There are two types of undo log records:

  • INSERT
  • UPDATE

An undo log record contains a table_id. Percona XtraBackup uses this table_id to find the table definition, and then uses that information to parse the records on an index page. The transaction rollback reads the undo log records and applies the changes.

After initializing the data dictionary engine and the data dictionary cache, the storage engine can request the table_id and uses this ID to fetch the table schema. An index search tuple (key) is created from the table schema and key fields from the undo log record. The server finds the record using the search tuple (key) and performs the undo operation.

For example, InnoDB uses the table_id (also known as the se_private_id) for a table definition. Percona XtraBackup does not behave like a server and does not have access to the data dictionary. XtraBackup initializes the InnoDB engine and uses the InnoDB table object (dict_table_t) when needed. XtraBackup relies on Serialized Dictionary Information (SDI) that is stored in the tablespace. SDI is a JSON representation of a table.

In Percona XtraBackup 8.1.0-1, tables are loaded as evictable. XtraBackup scans the B-tree index of the data dictionary tables mysql.indexes and mysql.index_partitions to establish a relationship between the table_id and the tablespace(space_id). XtraBackup uses this relationship during transaction rollback. XtraBackup does not load user tables unless there is a transaction rollback on them.

A background thread or a Percona XtraBackup main thread handles the cache eviction when the cache size limit is reached.

This design provides the following benefits during the --prepare phase:

  • Uses less memory
  • Uses less IO
  • Improves the time taken in the --prepare phase
  • Completes successfully, even if the --prepare phase has a huge number of tables
  • Completes the Percona XtraDB Cluster SST process faster
"},{"location":"dictionary-cache.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"docker.html","title":"Run a Docker container","text":"

Docker allows you to run applications in a lightweight unit called a container.

You can run Percona XtraBackup in a Docker container without installing the product. All required libraries are available in the container. Being a lightweight execution environment, Docker containers enable creating configurations where each program runs in a separate container. You may run Percona Server for MySQL in one container and Percona XtraBackup in another. Docker images offer a range of options.

Create a Docker container based on a Docker image. Docker images for Percona XtraBackup are hosted publicly on Docker Hub.

$ sudo docker create ... percona/percona-xtrabackup --name xtrabackup ...\n
"},{"location":"docker.html#scope-of-this-section","title":"Scope of this section","text":"

This section demonstrates how to back up data on a Percona Server for MySQL running in another Dockers container.

"},{"location":"docker.html#1-install-docker","title":"1. Install Docker","text":"

Your operating system may already provide a package for docker. However, the versions of Docker provided by your operating system are likely to be outdated.

Use the installation instructions for your operating system available from the Docker site to set up the latest version of docker.

"},{"location":"docker.html#2-connect-to-a-percona-server-for-mysql-container","title":"2. Connect to a Percona Server for MySQL container","text":"

Percona XtraBackup works in combination with a database server. When running a Docker container for Percona XtraBackup, you can make backups for a database server either installed on the host machine or running in a separate Docker container.

To set up a database server on a host machine or in Docker container, follow the documentation of the supported product that you intend to use with Percona XtraBackup.

See also

Docker volumes as container persistent data storage

More information about containers

$ sudo docker run -d --name percona-server-mysql \\\n-e MYSQL_ROOT_PASSWORD=root percona/percona-server:8.1\n

As soon as Percona Server for MySQL runs, add some data to it. Now, you are ready to make backups with Percona XtraBackup.

Important

When running Percona XtraBackup from a container and connecting to a MySQLserver container, we recommend using the \u2013user root option in the Docker command.

"},{"location":"docker.html#3-create-a-docker-container-from-percona-xtrabackup-image","title":"3. Create a Docker container from Percona XtraBackup image","text":"

You can create a Docker container based on Percona XtraBackup image with either docker create or the docker run command. docker create creates a Docker container and makes it available for starting later.

Docker downloads the Percona XtraBackup image from the Docker Hub. If it is not the first time you use the selected image, Docker uses the image available locally.

$ sudo docker create --name percona-xtrabackup --volumes-from percona-server-mysql \\\npercona/percona-xtrabackup  \\\nxtrabackup --backup --datadir=/var/lib/mysql/ --target-dir=/backup \\\n--user=root --password=mysql\n

With parameter name you give a meaningful name to your new Docker container so that you could easily locate it among your other containers.

The volumes-from flag refers to Percona Server for MySQL and indicates that you intend to use the same data as the Percona Server for MySQL container.

Run the container with exactly the same parameters that were used when the container was created:

$ sudo docker start -ai percona-xtrabackup\n

This command starts the percona-xtrabackup container, attaches to its input/output streams, and opens an interactive shell.

The docker run is a shortcut command that creates a Docker container and then immediately runs it.

$ sudo docker run --name percona-xtrabackup --volumes-from percona-server-mysql \\\npercona/percona-xtrabackup\nxtrabackup --backup --data-dir=/var/lib/mysql --target-dir=/backup --user=root --password=mysql\n
"},{"location":"docker.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"encrypt-backups.html","title":"Encrypt backups","text":"

Percona XtraBackup supports encrypting and decrypting local and streaming backups with xbstream option adding another layer of protection. The encryption is implemented using the libgcrypt library from GnuPG.

"},{"location":"encrypt-backups.html#create-encrypted-backups","title":"Create encrypted backups","text":"

To make an encrypted backup the following options need to be specified (options --encrypt-key and --encrypt-key-file are mutually exclusive, i.e. just one of them needs to be provided):

  • --encrypt

  • --encrypt-key

  • --encrypt-key-file

Both the --encrypt-key option and --encrypt-key-file option can be used to specify the encryption key. An encryption key can be generated with a command like openssl rand -base64 24

U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\n

This value then can be used as the encryption key

"},{"location":"encrypt-backups.html#the-encrypt-key-option","title":"The --encrypt-key option","text":"

Example of the xtrabackup command using the --encrypt-key should look like this:

$  xtrabackup --backup --encrypt=AES256 --encrypt-key=\"U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\" --target-dir=/data/backup\n
"},{"location":"encrypt-backups.html#the-encrypt-key-file-option","title":"The --encrypt-key-file option","text":"

Use the --encrypt-key-file option as follows:

$ xtrabackup --backup --encrypt=AES256 --encrypt-key-file=/data/backups/keyfile --target-dir=/data/backup\n

Note

Depending on the text editor that you use to make the KEYFILE, the editor can automatically insert the CRLF (end of line) character. This will cause the key size to grow and thus making it invalid. The suggested way to create the file is by using the command line: echo -n \u201cU2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\u201d > /data/backups/keyfile.

"},{"location":"encrypt-backups.html#optimize-the-encryption-process","title":"Optimize the encryption process","text":"

Two new options are available for encrypted backups that can be used to speed up the encryption process. These are --encrypt-threads and --encrypt-chunk-size. By using the --encrypt-threads option multiple threads can be specified to be used for encryption in parallel. Option --encrypt-chunk-size can be used to specify the size (in bytes) of the working encryption buffer for each encryption thread (default is 64K).

"},{"location":"encrypt-backups.html#decrypt-encrypted-backups","title":"Decrypt encrypted backups","text":"

Backups can be decrypted with The xbcrypt binary. The following one-liner can be used to encrypt the whole folder:

$ for i in `find . -iname \"*\\.xbcrypt\"`; do xbcrypt -d --encrypt-key-file=/root/secret_key --encrypt-algo=AES256 < $i > $(dirname $i)/$(basename $i .xbcrypt) && rm $i; done\n

Percona XtraBackup --decrypt option has been implemented that can be used to decrypt the backups:

$ xtrabackup --decrypt=AES256 --encrypt-key=\"U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\" --target-dir=/data/backup/\n

Percona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should remove the \\*.xbcrypt files.

Note

--parallel can be used with --decrypt option to decrypt multiple files simultaneously.

When the files are decrypted, the backup can be prepared.

"},{"location":"encrypt-backups.html#prepare-encrypted-backups","title":"Prepare encrypted backups","text":"

After the backups have been decrypted, they can be prepared in the same way as the standard full backups with the --prepare option:

$ xtrabackup --prepare --target-dir=/data/backup/\n
"},{"location":"encrypt-backups.html#restore-encrypted-backups","title":"Restore encrypted backups","text":"

xtrabackup offers the --copy-back option to restore a backup to the server\u2019s datadir:

$ xtrabackup --copy-back --target-dir=/data/backup/\n

It will copy all the data-related files back to the server\u2019s datadir, determined by the server\u2019s my.cnf configuration file. You should check the last line of the output for a success message:

Expected output 
150318 11:08:13  xtrabackup: completed OK!\n
"},{"location":"encrypt-backups.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"encrypted-innodb-tablespace-backups.html","title":"Encrypted InnoDB tablespace backups","text":"

InnoDB supports data encryption for InnoDB tables stored in file-per-table tablespaces. This feature provides an at-rest encryption for physical tablespace data files.

For an authenticated user or an application to access an encrypted F tablespace, InnoDB uses the master encryption key to decrypt the tablespace key. The master encryption key is stored in a keyring.

Percona XtraBackup supports the following keyring components and plugins:

  • keyring_vault component

  • keyring_file plugin

  • keyring_file component

  • Key Management Interoperability Protocol (KMIP)

  • Amazon Key Management Service (AWS KMS)

These components are stored in the plugin directory.

Note

Enable only one keyring plugin or one keyring component at a time for each server instance. Enabling multiple keyring plugins or keyring components or mixing keyring plugins or keyring components is not supported and may result in data loss.

"},{"location":"encrypted-innodb-tablespace-backups.html#use-the-keyring-vault-component","title":"Use the keyring vault component","text":"

The keyring_vault component extends the server capabilities. The server uses a manifest to load the component and the component has its own configuration file.

The following example is a global manifest file that does not use local manifests:

{\n \"read_local_manifest\": false,\n \"components\": \"file:///component_keyring_vault\"\n}\n

The following is an example of a global manifest file that points to a local manifest file:

{\n \"read_local_manifest\": true\n}\n

The following is an example of a local manifest file:

{\n \"components\": \"file:///component_keyring_vault\"\n}\n

The configuration settings are either in a global configuration file or a local configuration file.

Example of a configuration file in JSON format 
{\n \"timeout\": 15,\n \"vault_url\": \"https://vault.public.com:8202\",\n \"secret_mount_point\": \"secret\",\n \"secret_mount_point_version\": \"AUTO\",\n \"token\": \"58a20c08-8001-fd5f-5192-7498a48eaf20\",\n \"vault_ca\": \"/data/keyring_vault_confs/vault_ca.crt\"\n}\n

Find more information on configuring the keyring_vault component in Use the keyring vault component.

The component has no special requirements for backing up a database that contains encrypted InnoDB tablespaces.

The following command creates a backup in the /data/backup directory:

$ xtrabackup --backup --target-dir=/data/backup --user=root\n

After xtrabackup completes the action, the following message confirms the action:

Confirmation message 
xtrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\n
"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring-vault-component","title":"Prepare the backup with the keyring vault component","text":"

To prepare the backup, xtrabackup must access the keyring. xtrabackup does not communicate with the MySQL server or read the default my.cnf configuration file. Specify the keyring settings in the command line:

$ xtrabackup --prepare --target-dir=/data/backup --component-keyring-config==/etc/vault.cnf\n

After xtrabackup completes the action, the following message confirms the action:

Confirmation message 
InnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\n
"},{"location":"encrypted-innodb-tablespace-backups.html#restore-the-backup","title":"Restore the backup","text":"

As soon as the backup is prepared, you can restore it with the --copy-back option:

$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--component-keyring-config=/etc/vault.cnf\n
"},{"location":"encrypted-innodb-tablespace-backups.html#use-the-keyring-file-plugin","title":"Use the keyring file plugin","text":"

Warning

The keyring_file plugin should not be used for regulatory compliance.

In order to back up and prepare a database containing encrypted InnoDB tablespaces, specify the path to a keyring file as the value of the --keyring-file-data option.

$ xtrabackup --backup --target-dir=/data/backup/ --user=root \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

After xtrabackup takes the backup, the following message confirms the action:

Confirmation message 
xtrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\n

Warning

xtrabackup does not copy the keyring file into the backup directory. To prepare the backup, you must copy the keyring file manually.

"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring-file-plugin","title":"Prepare the backup with the keyring file plugin","text":"

To prepare the backup specify the keyring-file-data.

$ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

After xtrabackup takes the backup, the following message confirms the action:

Confirmation message 
InnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\n

The backup is now prepared and can be restored with the --copy-back option. In case the keyring has been rotated, you must restore the keyring used when the backup was taken and prepared.

"},{"location":"encrypted-innodb-tablespace-backups.html#use-the-keyring-file-component","title":"Use the keyring file component","text":"

The keyring_file component is part of the component-based MySQL infrastructure which extends the server capabilities.

The server uses a manifest to load the component and the component has its own configuration file. See the MySQL documentation on the component installation for more information.

An example of a manifest and a configuration file follows:

An example of ./bin/mysqld.my:

{\n   \"components\": \"file://component_keyring_file\"\n}\n

An example of /lib/plugin/component_keyring_file.cnf:

{\n   \"path\": \"/var/lib/mysql-keyring/keyring_file\", \"read_only\": false\n}\n

For more information, see Keyring Component Installation and Using the keyring_file File-Based Keyring Plugin.

With the appropriate privilege, you can SELECT on the performance_schema.keyring_component_status table to view the attributes and status of the installed keyring component when in use.

The component has no special requirements for backing up a database that contains encrypted InnoDB tablespaces.

$ xtrabackup --backup --target-dir=/data/backup --user=root\n

After xtrabackup completes the action, the following message confirms the action:

Confirmation message 
xtrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\n

Warning

xtrabackup does not copy the keyring file into the backup directory. To prepare the backup, you must copy the keyring file manually.

"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring_file-component","title":"Prepare the backup with the keyring_file component","text":"

xtrabackup reads the keyring_file component configuration from xtrabackup_component_keyring_file.cnf. You must specify the keyring_file data path if the --component-keyring-config is not located in the attribute PATH from the xtrabackup_component_keyring_file.cnf.

The following is an example of adding the location for the \u2013component-keyring-config:

$ xtrabackup --prepare --target-dir=/data/backup \\\n--component-keyring-config=/var/lib/mysql-keyring/keyring\n

xtrabackup attempts to read xtrabackup_component_keyring_file.cnf. You can assign another keyring file component configuration by passing the --component-keyring-config option.

After xtrabackup completes preparing the backup, the following message confirms the action:

Confirmation message 
InnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\n

The backup is prepared. To restore the backup use the --copy-back option. If the keyring has been rotated, you must restore the specific keyring used to take and prepare the backup.

"},{"location":"encrypted-innodb-tablespace-backups.html#incremental-encrypted-innodb-tablespace-backups-with-keyring-file-plugin","title":"Incremental encrypted InnoDB tablespace backups with keyring file plugin","text":"

The process of taking incremental backups with InnoDB tablespace encryption is similar to taking the Incremental Backups with unencrypted tablespace.

"},{"location":"encrypted-innodb-tablespace-backups.html#create-an-incremental-backup","title":"Create an incremental backup","text":"

To make an incremental backup, begin with a full backup. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. First you need to create a fullbackup with the following command:

$ xtrabackup --backup --target-dir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

In order to prepare the backup, you must make a copy of the keyring file yourself. xtrabackup does not copy the keyring file into the backup directory. Restoring the backup after the keyring has been changed causes errors like ERROR 3185 (HY000): Can't find master key from keyring, please check keyring plugin is loaded. when the restore process tries accessing an encrypted table.

If you look at the xtrabackup_checkpoints file, you should see the output similar to the following:

Expected output 
backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 7666625\nlast_lsn = 7666634\ncompact = 0\nrecover_binlog_info = 1\n

Now that you have a full backup, you can make an incremental backup based on it. Use a command such as the following:

$ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

To prepare the backup, you must copy the keyring file manually. xtrabackup does not copy the keyring file into the backup directory.

If the keyring has not been rotated you can use the same as the one you\u2019ve backed-up with the base backup. If the keyring has been rotated, or you have upgraded the plugin to a component, you must back up the keyring file. Otherwise, you cannot prepare the backup.

The /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 7666625. If you examine the xtrabackup_checkpoints file in this directory, you should see the output similar to the following:

Expected output 
backup_type = incremental\nfrom_lsn = 7666625\nto_lsn = 8873920\nlast_lsn = 8873929\ncompact = 0\nrecover_binlog_info = 1\n

Use this directory as the base for yet another incremental backup:

$ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-incremental-backups","title":"Prepare incremental backups","text":"

The --prepare step for incremental backups is not the same as for normal backups. In normal backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing a backup, because transactions that were uncommitted at the time of your backup may be in progress, and it\u2019s likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.

If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups are useless. After transactions have been rolled back, further incremental backups cannot be applied.

Beginning with the full backup you created, you can prepare it and then apply the incremental differences to it. Recall that you have the following backups:

/data/backups/base\n/data/backups/inc1\n/data/backups/inc2\n

To prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:

$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
Expected output 
InnoDB: Shutdown completed; log sequence number 7666643\nInnoDB: Number of pools: 1\n160401 12:31:11 completed OK!\n

To apply the first incremental backup to the full backup, you should use the following command:

$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

The backup should be prepared with the keyring file and type that was used when backup was being taken. This means that if the keyring has been rotated, or you have upgraded from a plugin to a component between the base and incremental backup that you must use the keyring that was in use when the first incremental backup has been taken.

Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:

$ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
Use --apply-log-only when merging all incremental backups except the last one. That\u2019s why the previous line does not contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase.

The backup is now prepared and can be restored with --copy-back option. In case the keyring has been rotated you\u2019ll need to restore the keyring which was used to take and prepare the backup.

"},{"location":"encrypted-innodb-tablespace-backups.html#restore-a-backup-when-the-keyring-is-not-available","title":"Restore a backup when the keyring is not available","text":"

While the described restore method works, this method requires access to the same keyring that the server is using. It may not be possible if the backup is prepared on a different server or at a much later time, when keys in the keyring are purged, or, in the case of a malfunction, when the keyring vault server is not available at all.

The --transition-key=<passphrase> option should be used to make it possible for xtrabackup to process the backup without access to the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and will use it to encrypt tablespace keys of tablespaces that are being backed up.

"},{"location":"encrypted-innodb-tablespace-backups.html#create-a-backup-with-a-passphrase","title":"Create a backup with a passphrase","text":"

The following example illustrates how the backup can be created in this case:

$ xtrabackup --backup --user=root -p --target-dir=/data/backup \\\n--transition-key=MySecretKey\n

If --transition-key is specified without a value, xtrabackup will ask for it.

xtrabackup scrapes --transition-key so that its value is not visible in the ps command output.

"},{"location":"encrypted-innodb-tablespace-backups.html#prepare-a-backup-with-a-passphrase","title":"Prepare a backup with a passphrase","text":"

The same passphrase should be specified for the prepare command:

$ xtrabackup --prepare --target-dir=/data/backup \\\n--transition-key=MySecretKey\n

There are no --keyring-vault...,``\u2013keyring-file\u2026``, or --component-keyring-config options here, because xtrabackup does not talk to the keyring in this case.

"},{"location":"encrypted-innodb-tablespace-backups.html#restore-a-backup-with-a-generated-key","title":"Restore a backup with a generated key","text":"

When restoring a backup you need to generate a new master key.

The example for keyring_file plugin:

$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

The example for keyring_file component:

$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--component-keyring-config=/var/lib/mysql-keyring/keyring\n

The example for keyring_vault component:

$ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--component-keyring-config=/etc/vault.cnf\n

xtrabackup generates a new master key, stores it in the target keyring vault server and re-encrypts the tablespace keys using this key.

"},{"location":"encrypted-innodb-tablespace-backups.html#make-a-backup-with-a-stored-transition-key","title":"Make a backup with a stored transition key","text":"

Finally, there is an option to store a transition key in the keyring. In this case, xtrabackup will need to access the same keyring file or vault server during prepare and copy-back but does not depend on whether the server keys have been purged.

In this scenario, the three stages of the backup process look as follows.

  • Back up
$ xtrabackup --backup --user=root -p --target-dir=/data/backup \\\n--generate-transition-key\n

  • Prepare

    • keyring_file plugin variant:
    $ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
    • keyring_file component variant:
    $ xtrabackup --prepare --target-dir=/data/backup \\\n--component-keyring-config=/var/lib/mysql-keyring/keyring\n
    • keyring_vault component variant:
    $ xtrabackup --prepare --target-dir=/data/backup \\\n--component-keyring-config=/etc/vault.cnf\n

  • Copy-back

    • keyring_file plugin variant:
    $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --keyring-file-data=/var/lib/mysql-keyring/keyring\n
    • keyring_file component variant:
    $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --component-keyring-config=/var/lib/mysql-keyring/keyring\n
    • keyring_vault component variant:
    $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --component-keyring-config=/etc/vault.cnf\n
"},{"location":"encrypted-innodb-tablespace-backups.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"faq.html","title":"Faq","text":""},{"location":"faq.html#frequently-asked-questions","title":"Frequently asked questions","text":""},{"location":"faq.html#does-percona-xtrabackup-81-support-making-backups-of-databases-in-versions-prior-to-81","title":"Does Percona XtraBackup 8.1 support making backups of databases in versions prior to 8.1?","text":"

Percona XtraBackup 8.1 does not support making backups of databases created in versions prior to 8.1 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster.

"},{"location":"faq.html#are-you-aware-of-any-web-based-backup-management-tools-commercial-or-not-built-around-percona-xtrabackup","title":"Are you aware of any web-based backup management tools (commercial or not) built around Percona XtraBackup*?","text":"

ZRM Community is a community tool that uses Percona XtraBackup for Non-Blocking Backups:

\u201cZRM provides support for non-blocking backups of MySQL using Percona XtraBackup. ZRM with \\Percona XtraBackup provides resource utilization management by providing throttling based on the number of IO operations per second. Percona XtraBackup based backups also allow for table level recovery even though the backup was done at the database level (needs the recovery database server to be Percona Server for MySQL with XtraDB).\u201d*

"},{"location":"faq.html#xtrabackup-binary-fails-with-a-floating-point-exception","title":"xtrabackup binary fails with a floating point exception","text":"

In most of the cases this is due to not having installed the required libraries (and version) by xtrabackup. Installing the GCC suite with the supporting libraries and recompiling xtrabackup solves the issue. See Compiling and Installing from Source Code for instructions on the procedure.

"},{"location":"faq.html#how-xtrabackup-handles-the-ibdataib_log-files-on-restore-if-they-arent-in-mysql-datadir","title":"How xtrabackup handles the ibdata/ib_log files on restore if they aren\u2019t in mysql datadir?","text":"

In case the ibdata and ib_log files are located in different directories outside the datadir, you will have to put them in their proper place after the logs have been applied.

"},{"location":"faq.html#backup-fails-with-error-24-too-many-open-files","title":"Backup fails with Error 24: \u2018Too many open files\u2019","text":"

This usually happens when database being backed up contains large amount of files and Percona XtraBackup can\u2019t open all of them to create a successful backup. In order to avoid this error the operating system should be configured appropriately so that Percona XtraBackup can open all its files. On Linux, this can be done with the ulimit command for specific backup session or by editing the /etc/security/limits.conf to change it globally (NOTE: the maximum possible value that can be set up is 1048576 which is a hard-coded constant in the Linux kernel).

"},{"location":"faq.html#how-to-deal-with-skipping-of-redo-logs-for-ddl-operations","title":"How to deal with skipping of redo logs for DDL operations?","text":"

To prevent creating corrupted backups when running DDL operations, Percona XtraBackup aborts if it detects that redo logging is disabled. In this case, the following error is printed:

[FATAL] InnoDB: An optimized (without redo logging) DDL operation has been performed. All modified pages may not have been flushed to the disk yet.\nPercona XtraBackup will not be able to take a consistent backup. Retry the backup operation.\n

Note

  • Redo logging is disabled during a sorted index build. To avoid this error, Percona XtraBackup can use metadata locks on tables while they are copied:

  • To block all DDL operations, use the --lock-ddl option that issues LOCK TABLES FOR BACKUP.

"},{"location":"faq.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"features.html","title":"Percona XtraBackup features","text":"

The following is a short list of the Percona XtraBackup features:

  • Creates hot InnoDB backups without pausing your database

  • Makes incremental backups of MySQL

  • Streams compressed MySQL backups to another server

  • Moves tables between MySQL servers on-line

  • Creates new MySQL replication replicas easily

  • Backs up MySQL without adding load to the server

  • Performs throttling based on the number of IO operations per second

  • Skips secondary index pages and recreates them when a compact backup is prepared

  • Exports individual tables from a full InnoDB backup

Percona XtraBackup automatically uses backup locks, a lightweight alternative to FLUSH TABLES WITH READ LOCK available in Percona Server, to copy non-InnoDB data. This operation avoids blocking DML queries that modify InnoDB tables.

See also

How Percona XtraBackup works

"},{"location":"features.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"flush-tables-with-read-lock.html","title":"FLUSH TABLES WITH READ LOCK option","text":"

The FLUSH TABLES WITH READ LOCK option does the following with a global read lock:

  • Closes all open tables

  • Locks all tables for all databases

Release the lock with UNLOCK TABLES.

Note

FLUSH TABLES WITH READ LOCK does not prevent inserting rows into the log tables.

To ensure consistent backups, use the FLUSH TABLES WITH READ LOCK option before taking a non-InnoDB file backup. The option does not affect long-running queries.

Enabling FLUSH TABLES WITH READ LOCK when the server has long-running queries can leave the server in a read-only mode until the queries finish. If the server is in either the Waiting for table flush or the Waiting for master to send event state, stopping the FLUSH TABLES WITH READ LOCK operation does not help. Stop any long-running queries to return to normal operation.

To prevent the server staying in a read-only mode until the queries finish, xtrabackup does the following:

  • checks if any queries run longer than specified in --ftwrl-wait-threshold. If xtrabackup finds such queries, xtrabackup waits for one second and checks again. If xtrabackup waits longer than specified in --ftwrl-wait-timeout, the backup is aborted. As soon as xtrabackup finds no queries running longer than specified in --ftwrl-wait-threshold, xtrabackup issues the global lock.

  • kills all queries or only the SELECT queries which prevent the global lock from being acquired.

Note

All operations described in this section have no effect when Backup locks are used.

Percona XtraBackup uses Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. This operation automatically copies non-InnoDB data and avoids blocking DML queries that modify InnoDB tables.

"},{"location":"flush-tables-with-read-lock.html#wait-for-queries-to-finish","title":"Wait for queries to finish","text":"

You should issue a global lock when no long queries are running. Waiting to issue the global lock for extended period of time is not a good method. The wait can extend the time needed for backup to take place. The \u2013ftwrl-wait-timeout option can limit the waiting time. If it cannot issue the lock during this time, xtrabackup stops the option, exits with an error message, and backup is not be taken.

The option\u2019s default value is zero (0), which turns off the option.

Another possibility is to specify the type of query to wait on. In this case --ftwrl-wait-query-type.

The possible values are all and update. When all is used xtrabackup will wait for all long running queries (execution time longer than allowed by --ftwrl-wait-threshold) to finish before running the FLUSH TABLES WITH READ LOCK. When update is used xtrabackup will wait on UPDATE/ALTER/REPLACE/INSERT queries to finish.

The time needed for a specific query to complete is hard to predict. We assume that the long-running queries will not finish in a timely manner. Other queries which run for a short time finish quickly. xtrabackup uses the value of \u2013ftwrl-wait-threshold option to specify the long-running queries and will block a global lock. To use this option xtrabackup user should havePROCESSandCONNECTION_ADMIN` privileges.

"},{"location":"flush-tables-with-read-lock.html#kill-the-blocking-queries","title":"Kill the blocking queries","text":"

The second option is to kill all the queries which prevent from acquiring the global lock. In this case, all queries which run longer than FLUSH TABLES WITH READ LOCK are potential blockers. Although all queries can be killed, additional time can be specified for the short running queries to finish using the --kill-long-queries-timeout option. This option specifies a query time limit. After the specified time is reached, the server kills the query. The default value is zero, which turns this feature off.

The --kill-long-query-type option can be used to specify all or only SELECT queries that are preventing global lock from being acquired. To use this option xtrabackup user should have PROCESS and CONNECTION_ADMIN privileges.

"},{"location":"flush-tables-with-read-lock.html#options-summary","title":"Options summary","text":"

  • --ftwrl-wait-timeout (seconds) - how long to wait for a good moment. Default is 0, not to wait.

  • --ftwrl-wait-query-type - which long queries should be finished before FLUSH TABLES WITH READ LOCK is run. Default is all.

  • --ftwrl-wait-threshold (seconds) - how long query should be running before we consider it long running and potential blocker of global lock.

  • --kill-long-queries-timeout (seconds) - how many time we give for queries to complete after FLUSH TABLES WITH READ LOCK is issued before start to kill. Default if 0, not to kill.

  • --kill-long-query-type - which queries should be killed once kill-long-queries-timeout has expired.

"},{"location":"flush-tables-with-read-lock.html#example","title":"Example","text":"

Running the xtrabackup with the following options will cause xtrabackup to spend no longer than 3 minutes waiting for all queries older than 40 seconds to complete.

$  xtrabackup --backup --ftwrl-wait-threshold=40 \\\n--ftwrl-wait-query-type=all --ftwrl-wait-timeout=180 \\\n--kill-long-queries-timeout=20 --kill-long-query-type=all \\\n--target-dir=/data/backups/\n

After FLUSH TABLES WITH READ LOCK is issued, xtrabackup will wait for 20 seconds for lock to be acquired. If lock is still not acquired after 20 seconds, it will kill all queries which are running longer that the FLUSH TABLES WITH READ LOCK.

"},{"location":"flush-tables-with-read-lock.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"glossary.html","title":"Glossary","text":""},{"location":"glossary.html#csm","title":".CSM","text":"

Each table with the CSV Storage Engine has .CSM file which contains the metadata of it.

"},{"location":"glossary.html#csv","title":".CSV","text":"

Each table with the CSV Storage engine has .CSV file which contains the data of it (which is a standard Comma Separated Value file).

"},{"location":"glossary.html#exp","title":".exp","text":"

Files with the .exp extension are created by Percona XtraBackup per each InnoDB tablespace when the --export option is used on prepare. See restore individual tables.

"},{"location":"glossary.html#frm","title":".frm","text":"

For each table, the server will create a file with the .frm extension containing the table definition (for all storage engines).

"},{"location":"glossary.html#general-availability-ga","title":"General availability (GA)","text":"

A finalized version of the product which is made available to the general public. It is the final stage in the software release cycle.

"},{"location":"glossary.html#ibd","title":".ibd","text":"

On a multiple tablespace setup ([innodb_file_per_table] enabled), MySQL will store each newly created table on a file with a .ibd extension.

"},{"location":"glossary.html#mrg","title":".MRG","text":"

Each table using the MERGE storage engine, besides of a .frm file, will have .MRG file containing the names of the MyISAM tables associated with it.

"},{"location":"glossary.html#myd","title":".MYD","text":"

Each MyISAM table has .MYD (MYData) file which contains the data on it.

"},{"location":"glossary.html#myi","title":".MYI","text":"

Each MyISAM table has .MYI (MYIndex) file which contains the table\u2019s indexes.

"},{"location":"glossary.html#opt","title":".opt","text":"

MySQL stores options of a database (like charset) in a file with a .opt extension in the database directory.

"},{"location":"glossary.html#par","title":".par","text":"

Each partitioned table has .par file which contains metadata about the partitions.

"},{"location":"glossary.html#trg","title":".TRG","text":"

The file contains the triggers associated with a table, for example, \\mytable.TRG. With the .TRN file, they represent all the trigger definitions.

"},{"location":"glossary.html#trn","title":".TRN","text":"

The file contains the names of triggers that are associated with a table, for example, \\mytable.TRN. With the .TRG file, they represent all the trigger definitions.

"},{"location":"glossary.html#backup","title":"backup","text":"

The process of copying data or tables to be stored in a different location.

"},{"location":"glossary.html#compression","title":"compression","text":"

The method that produces backups in a reduced size.

"},{"location":"glossary.html#configuration-file","title":"configuration file","text":"

The file that contains the server startup options.

"},{"location":"glossary.html#crash","title":"crash","text":"

An unexpected shutdown which does not allow the normal server shutdown cleanup activities.

"},{"location":"glossary.html#crash-recovery","title":"crash recovery","text":"

The actions that occur when MySQL is restarted after a crash.

"},{"location":"glossary.html#data-dictionary","title":"data dictionary","text":"

The metadata for the tables, indexes, and table columns stored in the InnoDB system tablespace.

"},{"location":"glossary.html#datadir","title":"datadir","text":"

The directory in which the database server stores its data files. Most Linux distribution use /var/lib/mysql by default.

"},{"location":"glossary.html#full-backup","title":"full backup","text":"

A backup that contains the complete source data from an instance.

"},{"location":"glossary.html#ibdata","title":"ibdata","text":"

The default prefix for tablespace files. For example, ibdata1 is a 10MB auto-extensible file that MySQL creates for a shared tablespace by default.

"},{"location":"glossary.html#incremental-backup","title":"incremental backup","text":"

A backup stores data from a specific point in time.

"},{"location":"glossary.html#innodb","title":"InnoDB","text":"

Storage engine which provides ACID-compliant transactions and foreign key support, among others improvements over MyISAM. It is the default engine for MySQL 8.1.

"},{"location":"glossary.html#innodb_buffer_pool_size","title":"innodb_buffer_pool_size","text":"

The size in bytes of the memory buffer to cache data and indexes of InnoDB\u2019s tables. This aims to reduce disk access to provide better performance.

[mysqld] innodb_buffer_pool_size=8MB

"},{"location":"glossary.html#innodb_data_home_dir","title":"innodb_data_home_dir","text":"

The directory (relative to datadir) where the database server stores the files in a shared tablespace setup. This option does not affect the location of innodb\\_file\\_per\\_table. For example:

[mysqld] innodb_data_home_dir = ./

"},{"location":"glossary.html#innodb_data_file_path","title":"innodb_data_file_path","text":"

Specifies the names, sizes and location of shared tablespace files:

[mysqld] innodb_data_file_path=ibdata1:50M;ibdata2:50M:autoextend

"},{"location":"glossary.html#innodb_file_per_table","title":"innodb_file_per_table","text":"

By default, InnoDB creates tables and indexes in a file-per-tablespace. If the innodb_file_per_table variable is disabled, you can enable the variable in your configuration file:

[mysqld] innodb_file_per_table or start the server with --innodb_file_per_table.

"},{"location":"glossary.html#innodb_log_group_home_dir","title":"innodb_log_group_home_dir","text":"

Specifies the location of the InnoDB log files:

[mysqld] innodb_log_group_home=/var/lib/mysql

"},{"location":"glossary.html#logical-backup","title":"logical backup","text":"

A backup which contains a set of SQL statements. The statements can be used to recreate the databases.

"},{"location":"glossary.html#lsn","title":"LSN","text":"

Each InnoDB page contains a log sequence number(LSN). The LSN is the system version number for the database. Each page\u2019s LSN shows how recently it was changed.

"},{"location":"glossary.html#mycnf","title":"my.cnf","text":"

The database server\u2019s main configuration file. Most Linux distributions place it as /etc/mysql/my.cnf or /etc/my.cnf, but the location and name depends on the particular installation. Note that this method is not the only way of configuring the server, some systems rely on the command options.

"},{"location":"glossary.html#myisam","title":"MyISAM","text":"

The MySQL default storage engine until version 5.5. It doesn\u2019t fully support transactions but in some scenarios may be faster than InnoDB. Each table is stored on disk in 3 files: .frm, .MYD, .MYI.

"},{"location":"glossary.html#physical-backup","title":"physical backup","text":"

A backup that copies the data files.

"},{"location":"glossary.html#point-in-time-recovery","title":"point in time recovery","text":"

This method restores the data into the state it was at any selected point of time.

"},{"location":"glossary.html#prepared-backup","title":"prepared backup","text":"

A consistent set of backup data that is ready to be restored.

"},{"location":"glossary.html#restore","title":"restore","text":"

Copies the database backups taken using the backup command to the original location or a different location. A restore returns data that has been either lost, corrupted, or stolen to the original condition at a specific point in time.

"},{"location":"glossary.html#tech-preview","title":"Tech preview","text":"

A tech preview item can be a feature, a variable, or a value within a variable. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment and also use an alternative backup method for redundancy. A tech preview item is included in a release for users to provide feedback. The item is either updated and released as general availability(GA) or removed if not useful. The functionality can change from tech preview to GA.

"},{"location":"glossary.html#xbcrypt","title":"xbcrypt","text":"

To support the encryption and the decryption of the backups. This utility has been modeled after the xbstream binary to perform encryption and decryption outside Percona XtraBackup.

"},{"location":"glossary.html#xbstream","title":"xbstream","text":"

To support simultaneous compression and streaming, Percona XtraBackup uses the xbstream format. For more information see xbstream

"},{"location":"glossary.html#xtradb","title":"XtraDB","text":"

Percona XtraDB is an enhanced version of the InnoDB storage engine, designed to better scale on modern hardware. Percona XtraDB includes features which are useful in a high performance environment. It is fully backward-compatible, and is a drop-in replacement for the standard InnoDB storage engine. For more information, see The Percona XtraDB Storage Engine.

"},{"location":"glossary.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.

"},{"location":"glossary.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"how-xtrabackup-works.html","title":"How Percona XtraBackup works","text":"

Percona XtraBackup is based on InnoDB\u2019s crash-recovery functionality. It copies your InnoDB data files, which results in data that is internally inconsistent; but then it performs crash recovery on the files to make them a consistent, usable database again.

This works because InnoDB maintains a redo log, also called the transaction log. This contains a record of every change to InnoDB data. When InnoDB starts, it inspects the data files and the transaction log, and performs two steps. It applies committed transaction log entries to the data files, and it performs an undo operation on any transactions that modified data but did not commit.

The --register-redo-log-consumer parameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.

Percona XtraBackup remembers the LSN when it starts, and then copies the data files. The operation takes time, and the files may change, then LSN reflects the state of the database at different points in time. Percona XtraBackup also runs a background process that watches the transaction log files, and copies any changes. Percona XtraBackup does this continually. The transaction logs are written in a round-robin fashion, and can be reused.

Percona XtraBackup uses Backup locks

where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. MySQL 8.1 allows acquiring an instance level backup lock via the LOCK INSTANCE FOR BACKUP statement.

Locking is only done for MyISAM and other non-InnoDB tables after Percona XtraBackup finishes backing up all InnoDB/XtraDB data and logs. Percona XtraBackup uses this automatically to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

Important

The BACKUP_ADMIN privilege is required to query the performance_schema_log_status for either LOCK INSTANCE FOR BACKUP or LOCK TABLES FOR BACKUP.

xtrabackup tries to avoid backup locks and FLUSH TABLES WITH READ LOCK when the instance contains only InnoDB tables. In this case, xtrabackup obtains binary log coordinates from performance_schema.log_status. FLUSH TABLES WITH READ LOCK is still required in MySQL 8.1 when xtrabackup is started with the --slave-info. The log_status table in Percona Server for MySQL 8.1 is extended to include the relay log coordinates, so no locks are needed even with the --slave-info option.

See also

MySQL Documentation: LOCK INSTANCE FOR BACKUP

When backup locks are supported by the server, xtrabackup first copies InnoDB data, runs the LOCK TABLES FOR BACKUP and then copies the MyISAM tables. Once this is done, the backup of the files will begin. It will backup .frm, .MRG, .MYD, .MYI, .CSM, .CSV, .sdi and .par files.

After that xtrabackup will use LOCK BINLOG FOR BACKUP to block all operations that might change either binary log position or Exec_Master_Log_Pos or Exec_Gtid_Set (i.e. source binary log coordinates corresponding to the current SQL thread state on a replication replica) as reported by SHOW MASTER/SLAVE STATUS. xtrabackup will then finish copying the REDO log files and fetch the binary log coordinates. After this is completed xtrabackup will unlock the binary log and tables.

Finally, the binary log position will be printed to STDERR and xtrabackup will exit returning 0 if all went OK.

Note that the STDERR of xtrabackup is not written in any file. You will have to redirect it to a file, for example, xtrabackup OPTIONS 2> backupout.log.

It will also create the following files in the directory of the backup.

During the prepare phase, Percona XtraBackup performs crash recovery against the copied data files, using the copied transaction log file. After this is done, the database is ready to restore and use.

The backed-up MyISAM and InnoDB tables will be eventually consistent with each other, because after the prepare (recovery) process, InnoDB\u2019s data is rolled forward to the point at which the backup completed, not rolled back to the point at which it started. This point in time matches where the FLUSH TABLES WITH READ LOCK was taken, so the MyISAM data and the prepared InnoDB data are in sync.

The xtrabackup offers many features not mentioned in the preceding explanation. The functionality of each tool is explained in more detail further in this manual. In brief, though, the tools enable you to do operations such as streaming and incremental backups with various combinations of copying the data files, copying the log files, and applying the logs to the data.

"},{"location":"how-xtrabackup-works.html#restoring-a-backup","title":"Restoring a backup","text":"

To restore a backup with xtrabackup you can use the --copy-back or --move-back options.

xtrabackup will read from the my.cnf the variables datadir, innodb_data_home_dir, innodb_data_file_path, innodb_log_group_home_dir and check that the directories exist.

It will copy the MyISAM tables, indexes, etc. (.MRG, .MYD, .MYI, .CSM, .CSV, .sdi, and par files) first, InnoDB tables and indexes next and the log files at last. It will preserve file\u2019s attributes when copying them, you may have to change the files\u2019 ownership to mysql before starting the database server, as they will be owned by the user who created the backup.

Alternatively, the --move-back option may be used to restore a backup. This option is similar to --copy-back with the only difference that instead of copying files it moves them to their target locations. As this option removes backup files, it must be used with caution. It is useful in cases when there is not enough free disk space to hold both data files and their backup copies.

"},{"location":"how-xtrabackup-works.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"installation.html","title":"Install overview","text":"

We recommend that you install Percona XtraBackup 8.1 from the official Percona software repositories using the appropriate package manager for your system:

  • Use an APT repo to install Percona XtraBackup

  • Use a YUM repo to install Percona XtraBackup

"},{"location":"installation.html#installation-alternatives","title":"Installation alternatives","text":"

Percona also provides the following methods:

  • Use DEB downloaded packages to install Percona XtraBackup

  • Use RPM downloaded packages to install Percona XtraBackup

  • Install Percona XtraBackup from a Binary Tarball with all the required files and binaries for a manual installation

  • Compile and install Percona XtraBackup from source code

  • Run Percona XtraBackup in a Docker container

Before installing Percona XtraBackup with any of these methods, we recommend that you review the release notes and Server version and backup version comparison.

"},{"location":"installation.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"limitations.html","title":"Limitations","text":"

Percona XtraBackup 8.1 does not support making backups of databases created in versions prior to 8.1 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster.

"},{"location":"limitations.html#additional-information","title":"Additional information","text":"

The InnoDB tables are locked while copying non-InnoDB data.

"},{"location":"limitations.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"lock-options.html","title":"lock-ddl-per-table option improvements","text":"

To block DDL statements on an instance, Percona Server implemented LOCK TABLES FOR BACKUP. Percona XtraBackup uses this lock for the duration of the backup. This lock does not affect DML statements.

Percona XtraBackup has also implemented --lock-ddl-per-table, which blocks DDL statements by using metadata locks (MDL).

The following procedures describe a simplified backup operation when using --lock-ddl-per-table:

  1. Parse and copy all redo logs after the checkpoint mark

  2. Fork a dedicated thread to continue following new redo log entries

  3. List the tablespaces required to copy

  4. Iterate through the list. The following steps occur with each listed tablespace:

  5. Query INFORMATION_SCHEMA.INNODB_TABLES to find which tables belong to the tablespace ID and take an MDL on the underlying table or tables in case there is a shared tablespace.

  6. Copy the tablespace .ibd files.

The backup process may encounter a redo log event, generated by the bulk load operations, which notifies backup tools that data file changes have been omitted from the redo log. This event is an MLOG_INDEX_LOAD. If this event is found by the redo follow thread, the backup continues and assumes the backup is safe because the MDL protects tablespaces already copied and the MLOG_INDEX_LOAD event is for a tablespace that is not copied.

These assumptions may not be correct and may lead to inconsistent backups.

"},{"location":"lock-options.html#-lock-ddl-per-table-redesign","title":"--lock-ddl-per-table redesign","text":"

The --lock-ddl-per-table option has been redesigned to minimize inconsistent backups.

The following procedure reorders the steps:

  • The MDL lock acquired at the beginning of the backup

  • Scan the redo logs. An MLOG_INDEX_LOAD event may be recorded if a CREATE INDEX statement has occurred before the backup starts. At this time, the backup process is safe and can parse and accept the event.

  • After the first scan has completed, the follow redo log thread is initiated. This thread stops the backup process if an MLOG_INDEX_LOAD event is found.

  • Gather the tablespace list to copy

  • Copy the .ibd files.

"},{"location":"lock-options.html#other-improvements","title":"Other improvements","text":"

The following improvements have been added:

  • If the .ibd file belongs to a temporary table, the SELECT query is skipped.

  • For a FullText Index, an MDL is acquired on the base table.

  • A SELECT query that acquires an MDL does not retrieve any data.

"},{"location":"lock-options.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"log-enhancements.html","title":"Improved log statements","text":"

Percona XtraBackup is an open-source command-line utility. Command-line tools have limited interaction with the background operations and the logs provide the progress of an operation or more information about errors.

The error logs did not have a standard structure and the log statements varied in the following ways:

  • The backup log statement header has the name of the module, xtrabackup, which generated the statement but no timestamp:

 $ xtrabackup: recognized client arguments: --parallel=4 --target-dir=/data/backups/ --backup=1\n
The output should be similar to the following:

Expected output 
./bin/xtrabackup version 8.1.0-1 based on MySQL server 8.1 Linux (x86_64) (revision id: b0f75188ca3)\n
  • The copy-back log statement has a timestamp but no module name. The timestamp is a mix of UTC and the local timezone.
220322 19:05:13 [01] Copying undo_001 to /data/backups/undo_001\n
  • The following prepare log statements do not have header information, which makes diagnosing an issue more difficult.
Completed space ID check of 1008 files.\nInitializing buffer pool, total size = 128.000000M, instances = 1, chunk size =128.000000M\nCompleted initialization of buffer pool\nIf the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().\n
"},{"location":"log-enhancements.html#log-statement-structure","title":"Log statement structure","text":"

The improved log structure is displayed in the backup, prepare, move-back/copy-back error logs.

Each log statement has the following attributes:

  • Timestamp - a timestamp for when the event occurred in a UTC format.

  • Severity - the severity level of a statement indicates the importance of an event.

  • ID - this identifier is currently not used but may be used in future versions.

  • Context - the name of the module that issued the log statement, such as XtraBackup, InnoDB, or Server.

  • Message - a description of the event generated by the module.

An example of a prepare log that is generated with the improved structure. The uniformity of the headers makes it easier to follow an operation\u2019s progress or review the log to diagnose issues.

Expected output 
2022-03-22T19:15:36.142247+05:30 0 [Note] [MY-011825] [Xtrabackup] This target seems to be not prepared yet.\n2022-03-22T19:15:36.142792+05:30 0 [Note] [MY-013251] [InnoDB] Number of pools: 1\n2022-03-22T19:15:36.149212+05:30 0 [Note] [MY-011825] [Xtrabackup] xtrabackup_logfile detected: size=8388608, start_lsn=(33311656)\n2022-03-22T19:15:36.149998+05:30 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration for recovery:\n2022-03-22T19:15:36.150023+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .\n2022-03-22T19:15:36.150036+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend\n2022-03-22T19:15:36.150078+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = .\n2022-03-22T19:15:36.150095+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 1\n2022-03-22T19:15:36.150111+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 8388608\n2022-03-22T19:15:36.151667+05:30 0 [Note] [MY-011825] [Xtrabackup] inititialize_service_handles suceeded\n2022-03-22T19:15:36.151903+05:30 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration for recovery:\n2022-03-22T19:15:36.151926+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .\n2022-03-22T19:15:36.151954+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend\n2022-03-22T19:15:36.151976+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = .\n2022-03-22T19:15:36.151991+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 1\n2022-03-22T19:15:36.152004+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 8388608\n2022-03-22T19:15:36.152021+05:30 0 [Note] [MY-011825] [Xtrabackup] Starting InnoDB instance for recovery.\n2022-03-22T19:15:36.152035+05:30 0 [Note] [MY-011825] [Xtrabackup] Using 104857600 bytes for buffer pool (set by --use-memory parameter)\n
"},{"location":"log-enhancements.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"lru-dump-backup.html","title":"LRU dump backup","text":"

Percona XtraBackup includes a saved buffer pool dump into a backup to enable reducing the warm up time. It restores the buffer pool state from ib_buffer_pool file after restart. Percona XtraBackup discovers ib_buffer_pool and backs it up automatically.

If the buffer restore option is enabled in my.cnf, buffer pool will be in the warm state after backup is restored.

Find the information on how to save and restore the buffer pool dump in Saving and Restoring the Buffer Pool State.

"},{"location":"lru-dump-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"make-backup-in-replication-env.html","title":"Make backups in replication environments","text":"

There are options specific to back up from a replication replica.

"},{"location":"make-backup-in-replication-env.html#the-slave-info-option","title":"The --slave-info option","text":"

This option is useful when backing up a replication replica server. It prints the binary log position and name of the source server. It also writes this information to the xtrabackup_slave_info file as a CHANGE MASTER statement.

This option is useful for setting up a new replica for this source. You can start a replica server with this backup and issue the statement saved in the xtrabackup_slave_info file. More details of this procedure can be found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.

"},{"location":"make-backup-in-replication-env.html#the-safe-slave-backup-option","title":"The --safe-slave-backup option","text":"

In order to assure a consistent replication state, this option stops the replication SQL thread and waits to start backing up until Slave_open_temp_tables in SHOW STATUS is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds (defaults to 300 seconds). The replication SQL thread will be restarted when the backup finishes.

Note

Using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.

Using this option is always recommended when taking backups from a replica server.

Warning

Make sure your replica is a true replica of the source before using it as a source for backup. A good tool to validate a replica is pt-table-checksum.

"},{"location":"make-backup-in-replication-env.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"page-tracking.html","title":"Take an incremental backup using page tracking","text":"

To create an incremental backup with page tracking, Percona XtraBackup uses the MySQL mysqlbackup component. This component provides a list of pages modified since the last backup, and Percona XtraBackup copies only those pages. This operation removes the need to scan the pages in the database. If the majority of pages have not been modified, the page tracking feature can improve the speed of incremental backups.

"},{"location":"page-tracking.html#install-the-component","title":"Install the component","text":"

To start using the page tracking functionality, do the following:

  1. Install the mysqlbackup component and enable it on the server:

    $ INSTALL COMPONENT \"file://component_mysqlbackup\";\n

  2. Check whether the mysqlbackup component is installed successfully:

    $ SELECT COUNT(1) FROM mysql.component WHERE component_urn='file://component_mysqlbackup';\n
"},{"location":"page-tracking.html#use-page-tracking","title":"Use page tracking","text":"

You can enable the page tracking functionality for the full and incremental backups with the --page-tracking option.

The option has the following benefits:

  • Resets page tracking to the start of the backup. This reset allows the next incremental backup to use page tracking.

  • Allows the use of page tracking for an incremental backup if the page tracking data is available from the backup\u2019s start checkpoint LSN.

Percona XtraBackup processes a list of all the tracked pages in memory. If Percona XtraBackup does not have enough available memory to process this list, the process throws an error and exits. For example, if an incremental backup uses 200GB, Percona XtraBackup can use an additional 100MB of memory to process and store the page tracking data.

The examples of creating full and incremental backups using the --page-tracking option:

Full backupIncremental backup 
$ xtrabackup --backup --target-dir=$FULL_BACK --page-tracking\n
$ xtrabackup --backup --target-dir=$INC_BACKUP  \n--incremental-basedir=$FULL_BACKUP --page-tracking\n

After enabling the functionality, the next incremental backup finds changed pages using page tracking.

The first full backup using page tracking, Percona XtraBackup may have a delay. The following is an example of the message:

Expected output 
xtrabackup: pagetracking: Sleeping for 1 second, waiting for checkpoint lsn 17852922 /\nto reach to page tracking start lsn 21353759\n

Enable page tracking before creating the first backup to avoid this delay. This method ensures that the page tracking log sequence number (LSN) is higher than the checkpoint LSN of the server.

"},{"location":"page-tracking.html#start-page-tracking-manually","title":"Start page tracking manually","text":"

After the mysqlbackup component is loaded and active on the server, you can start page tracking manually with the following option:

$ SELECT mysqlbackup_page_track_set(true);\n
"},{"location":"page-tracking.html#check-the-lsn-value","title":"Check the LSN value","text":"

Check the LSN value starting from which changed pages are tracked with the following option:

$ SELECT mysqlbackup_page_track_get_start_lsn();\n
"},{"location":"page-tracking.html#stop-page-tracking","title":"Stop page tracking","text":"

To stop page tracking, use the following command:

$ SELECT mysqlbackup_page_track_set(false);\n
"},{"location":"page-tracking.html#purge-page-tracking-data","title":"Purge page tracking data","text":"

When you start page tracking, it creates a file under the server\u2019s datadir to collect data about changed pages. This file grows until you stop the page tracking. If you stop the server and then restart it, page tracking creates a new file but also keeps the old one. The old file continues to grow until you stop the page tracking explicitly.

If you purge the page tracking data, you should create a full backup afterward. To purge the page tracking data, do the following steps:

$ SELECT mysqlbackup_page_track_set(false);\n$ SELECT mysqlbackup_page_track_purge_up_to(9223372036854775807);\n/* Specify the LSN up to which you want to purge page tracking data. /\n9223372036854775807 is the highest possible LSN which purges all page tracking files.*/\n$ SELECT mysqlbackup_page_track_set(true);\n
"},{"location":"page-tracking.html#known-issue","title":"Known issue","text":"

If the index is built in place using an exclusive algorithm and then is added to a table after the last LSN checkpoint, you may generate a bad incremental backup using page tracking. For more details see PS-8032.

"},{"location":"page-tracking.html#uninstall-the-mysqlbackup-component","title":"Uninstall the mysqlbackup component","text":"

To uninstall the mysqlbackup component, use the following statement:

$ UNINSTALL COMPONENT \"file://component_mysqlbackup\"\n
"},{"location":"page-tracking.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"permissions.html","title":"Permissions needed","text":"

We will be referring to permissions to the ability of a user to access and perform changes on the relevant parts of the host\u2019s filesystem, starting/stopping services and installing software.

By privileges, we refer to the abilities of a database user to perform different kinds of actions on the database server.

There are many ways for checking the permission on a file or directory. For example, ls -ls /path/to/file or stat /path/to/file | grep Access will do the job:

$ stat /etc/mysql | grep Access\n
The result could look like this:

Expected output 
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)\nAccess: 2011-05-12 21:19:07.129850437 -0300\n$ ls -ld /etc/mysql/my.cnf\n-rw-r--r-- 1 root root 4703 Apr  5 06:26 /etc/mysql/my.cnf\n

As in this example, my.cnf is owned by root and not writable for anyone else. Assuming that you do not have root\u2018s password, you can check what permissions you have on these types of files with sudo -l:

$ sudo -l\n
The results could look like this:

Expected output 
Password:\nYou may run the following commands on this host:\n(root) /usr/bin/\n(root) NOPASSWD: /etc/init.d/mysqld\n(root) NOPASSWD: /bin/vi /etc/mysql/my.cnf\n(root) NOPASSWD: /usr/local/bin/top\n(root) NOPASSWD: /usr/bin/ls\n(root) /bin/tail\n

Being able to execute with sudo scripts in /etc/init.d/, /etc/rc.d/ or /sbin/service is the ability to start and stop services.

Also, If you can execute the package manager of your distribution, you can install or remove software with it. If not, having rwx permission over a directory will let you do a local installation of software by compiling it there. This is a typical situation in many hosting companies\u2019 services.

There are other ways for managing permissions, such as using PolicyKit, Extended ACLs or SELinux, which may be preventing or allowing your access. You should check them in that case.

See also

Connection and privileges needed

"},{"location":"permissions.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"point-in-time-recovery.html","title":"Point-in-time recovery","text":"

Recovering up to particular moment in database\u2019s history can be done with xtrabackup and the binary logs of the server.

Note that the binary log contains the operations that modified the database from a point in the past. You need a full datadir as a base, and then you can apply a series of operations from the binary log to make the data match what it was at the point in time you want.

$ xtrabackup --backup --target-dir=/path/to/backup\n$ xtrabackup --prepare --target-dir=/path/to/backup\n

For more details on these procedures, see Creating a backup and Preparing a backup.

Now, suppose that some time has passed, and you want to restore the database to a certain point in the past, having in mind that there is the constraint of the point where the snapshot was taken.

To find out what is the situation of binary logging in the server, execute the following queries:

mysql> SHOW BINARY LOGS;\n
Expected output 
+------------------+-----------+\n| Log_name         | File_size |\n+------------------+-----------+\n| mysql-bin.000001 |       126 |\n| mysql-bin.000002 |      1306 |\n| mysql-bin.000003 |       126 |\n| mysql-bin.000004 |       497 |\n+------------------+-----------+\n

and

mysql> SHOW MASTER STATUS;\n
Expected output 
+------------------+----------+--------------+------------------+\n| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |\n+------------------+----------+--------------+------------------+\n| mysql-bin.000004 |      497 |              |                  |\n+------------------+----------+--------------+------------------+\n

The first query will tell you which files contain the binary log and the second one which file is currently being used to record changes, and the current position within it. Those files are stored usually in the datadir (unless other location is specified when the server is started with the --log-bin= option).

To find out the position of the snapshot taken, see the xtrabackup_binlog_info at the backup\u2019s directory:

$ cat /path/to/backup/xtrabackup_binlog_info\n
Expected output 
mysql-bin.000003      57\n

This will tell you which file was used at moment of the backup for the binary log and its position. That position will be the effective one when you restore the backup:

$ xtrabackup --copy-back --target-dir=/path/to/backup\n

As the restoration will not affect the binary log files (you may need to adjust file permissions, see Restoring a Backup), the next step is extracting the queries from the binary log with mysqlbinlog starting from the position of the snapshot and redirecting it to a file

$ mysqlbinlog /path/to/datadir/mysql-bin.000003 /path/to/datadir/mysql-bin.000004 \\\n    --start-position=57 > mybinlog.sql\n

Note that if you have multiple files for the binary log, as in the example, you have to extract the queries with one process, as shown above.

Inspect the file with the queries to determine which position or date corresponds to the point-in-time wanted. Once determined, pipe it to the server. Assuming the point is 11-12-25 01:00:00:

$ mysqlbinlog /path/to/datadir/mysql-bin.000003 /path/to/datadir/mysql-bin.000004 \\\n    --start-position=57 --stop-datetime=\"11-12-25 01:00:00\" | mysql -u root -p\n

and the database will be rolled forward up to that Point-In-Time.

"},{"location":"point-in-time-recovery.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"prepare-compressed-backup.html","title":"Prepare the backup","text":"

Before you can prepare the backup you\u2019ll need to uncompress all the files. Percona XtraBackup has implemented --decompress option that can be used to decompress the backup.

$ xtrabackup --decompress --target-dir=/data/compressed/\n

Note

--parallel can be used with --decompress option to decompress multiple files simultaneously.

Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory you should use --remove-original option. Even if they\u2019re not removed these files will not be copied/moved over to the datadir if --copy-back or --move-back are used.

When the files are uncompressed you can prepare the backup with the --prepare option:

$ xtrabackup --prepare --target-dir=/data/compressed/\n
Confirmation message 
InnoDB: Starting shutdown...\nInnoDB: Shutdown completed; log sequence number 9293846\n170223 13:39:31 completed OK!\n

Now the files in /data/compressed/ are ready to be used by the server.

"},{"location":"prepare-compressed-backup.html#next-step","title":"Next step","text":"

Restore the backup

"},{"location":"prepare-compressed-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"prepare-full-backup.html","title":"Prepare a full backup","text":"

After creating a backup with the --backup option, you need to prepare the backup and then restore it. Data files are not point-in-time consistent until they are prepared, because they were copied at different times as the program ran, and they might have been changed while this was happening.

If you try to start InnoDB with these data files, it will detect corruption and stop working to avoid running on damaged data. The --prepare step makes the files perfectly consistent at a single instant in time, so you can run InnoDB on them.

You can run the prepare operation on any machine; it does not need to be on the originating server or the server to which you intend to restore. You can copy the backup to a utility server and prepare it there.

Note that Percona XtraBackup 8.1 can only prepare backups of MySQL 8.1 and Percona Server for MySQL 8.1 databases. Releases prior to 8.1 are not supported.

During the prepare operation, xtrabackup boots up a kind of modified embedded InnoDB (the libraries xtrabackup was linked against). The modifications are necessary to disable InnoDB standard safety checks, such as complaining about the log file not being the right size. This warning is not appropriate for working with backups. These modifications are only for the xtrabackup binary; you do not need a modified InnoDB to use xtrabackup for your backups.

The prepare step uses this \u201cembedded InnoDB\u201d to perform crash recovery on the copied data files, using the copied log file. The prepare step is very simple to use: you simply run xtrabackup with the --prepare option and tell it which directory to prepare, for example, to prepare the previously taken backup run:

$ xtrabackup --prepare --target-dir=/data/backups/\n

When this finishes, you should see an InnoDB shutdown with a message such as the following, where again the value of LSN will depend on your system:

Expected output 
InnoDB: Shutdown completed; log sequence number 137345046\n160906 11:21:01 completed OK!\n

All following prepares will not change the already prepared data files, you\u2019ll see that output says:

Expected output 
xtrabackup: This target seems to be already prepared.\nxtrabackup: notice: xtrabackup_logfile was already used to '--prepare'.\n

It is not recommended to interrupt xtrabackup process while preparing backup because it may cause data files corruption and backup will become unusable. Backup validity is not guaranteed if prepare process was interrupted.

Note

If you intend the backup to be the basis for further incremental backups, you should use the --apply-log-only option when preparing the backup, or you will not be able to apply incremental backups to it. See the documentation on preparing incremental backups for more details.

"},{"location":"prepare-full-backup.html#next-step","title":"Next step","text":"

Restore the backup

"},{"location":"prepare-full-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"prepare-incremental-backup.html","title":"Prepare an incremental backup","text":"

The --prepare step for incremental backups is not the same as for full backups. In full backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing an incremental backup, because transactions that were uncommitted at the time of your backup may be in progress, and it\u2019s likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.

Warning

If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups will be useless. After transactions have been rolled back, further incremental backups cannot be applied.

Beginning with the full backup you created, you can prepare it, and then apply the incremental differences to it. Recall that you have the following backups:

/data/backups/base\n/data/backups/inc1\n/data/backups/inc2\n

To prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:

$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base\n

The output should end with text similar to the following:

Expected output 
InnoDB: Shutdown completed; log sequence number 1626007\n161011 12:41:04 completed OK!\n

The log sequence number should match the to_lsn of the base backup, which you saw previously.

Warning

This backup is actually safe to restore as-is now, even though the rollback phase has been skipped. If you restore it and start MySQL, InnoDB will detect that the rollback phase was not performed, and it will do that in the background, as it usually does for a crash recovery upon start. It will notify you that the database was not shut down normally.

To apply the first incremental backup to the full backup, run the following command:

$ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1\n

This applies the delta files to the files in /data/backups/base, which rolls them forward in time to the time of the incremental backup. It then applies the redo log as usual to the result. The final data is in /data/backups/base, not in the incremental directory. You should see an output similar to:

Expected output 
incremental backup from 1626007 is enabled.\nxtrabackup: cd to /data/backups/base\nxtrabackup: This target seems to be already prepared with --apply-log-only.\nxtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(4124244)\n...\nxtrabackup: page size for /tmp/backups/inc1/ibdata1.delta is 16384 bytes\nApplying /tmp/backups/inc1/ibdata1.delta to ./ibdata1...\n...\n161011 12:45:56 completed OK!\n

Again, the LSN should match what you saw from your earlier inspection of the first incremental backup. If you restore the files from /data/backups/base, you should see the state of the database as of the first incremental backup.

Warning

Percona XtraBackup does not support using the same incremental backup directory to prepare two copies of backup. Do not run --prepare with the same incremental backup directory (the value of \u2013incremental-dir) more than once.

Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:

$ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2\n

Note

--apply-log-only should be used when merging the incremental backups except the last one. That\u2019s why the previous line does not contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase.

"},{"location":"prepare-incremental-backup.html#next-step","title":"Next step","text":"

Restore the backup

"},{"location":"prepare-incremental-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"prepare-individual-partitions-backup.html","title":"Prepare an individual partitions backup","text":"

For preparing partial backups, the procedure is analogous to restoring individual tables. Apply the logs and use xtrabackup \u2013export:

$ xtrabackup --apply-log --export /mnt/backup/2012-08-28_10-29-09\n

You may see warnings in the output about tables that do not exist. This happens because InnoDB-based engines stores its data dictionary inside the tablespace files. xtrabackup removes the missing tables (those that haven\u2019t been selected in the partial backup) from the data dictionary in order to avoid future warnings or errors.

"},{"location":"prepare-individual-partitions-backup.html#next-step","title":"Next step","text":"

Restore the partition from the backup

"},{"location":"prepare-individual-partitions-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"prepare-partial-backup.html","title":"Prepare a partial backup","text":"

The procedure is analogous to restoring individual tables: apply the logs and use the --export option:

$ xtrabackup --prepare --export --target-dir=/path/to/partial/backup\n

When you use the --prepare option on a partial backup, you will see warnings about tables that don\u2019t exist. This is because these tables exist in the data dictionary inside InnoDB, but the corresponding .ibd files don\u2019t exist. They were not copied into the backup directory. These tables will be removed from the data dictionary, and when you restore the backup and start InnoDB, they will no longer exist and will not cause any errors or warnings to be printed to the log file.

Could not find any file associated with the tablespace ID: 5

Use --innodb-directories to find the tablespace files. If that fails then use -\u2013innodb-force-recovery=1 to ignore this and to permanently lose all changes to the missing tablespace(s).

"},{"location":"prepare-partial-backup.html#next-step","title":"Next step","text":"

Restore the partition from the backup

"},{"location":"prepare-partial-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"privileges.html","title":"Connection and privileges needed","text":"

Percona XtraBackup needs to be able to connect to the database server and perform operations on the server and the datadir when creating a backup, when preparing in some scenarios and when restoring it. In order to do so, there are privileges and permission requirements on its execution that must be fulfilled.

Privilege refers to the operations that a system user is permitted to do in the database server. They are set at the database server and only apply to users in the database server.

Permissions are those which permits a user to perform operations on the system, like reading, writing or executing on a certain directory or start/stop a system service. They are set at a system level and only apply to system users.

When xtrabackup is used, there are two actors involved: the user invoking the program - a system user - and the user performing action in the database server - a database user. Note that these are different users in different places, even though they may have the same username.

All the invocations of xtrabackup in this documentation assume that the system user has the appropriate permissions, and you are providing the relevant options for connecting the database server - besides the options for the action to be performed - and the database user has adequate privileges.

"},{"location":"privileges.html#connect-to-the-server","title":"Connect to the server","text":"

The database user used to connect to the server and its password are specified by the --user and --password option:

$ xtrabackup --user=DVADER --password=14MY0URF4TH3R --backup \\\n--target-dir=/data/bkps/\n

If you don\u2019t use the --user option, Percona XtraBackup will assume the database user whose name is the system user executing it.

"},{"location":"privileges.html#other-connection-options","title":"Other connection options","text":"

According to your system, you may need to specify one or more of the following options to connect to the server:

Option Description -port Use this port when connecting to the database with TCP/IP -socket Use this socket when connecting to the local database. -host Use this host when connecting to the database server with TCP/IP

These options are passed to the mysql child process without alteration, see mysql --help for details.

Note

In case of multiple server instances, the correct connection parameters (port, socket, host) must be specified in order for xtrabackup to talk to the correct server.

"},{"location":"privileges.html#privileges-needed","title":"Privileges needed","text":"

Once connected to the server, in order to perform a backup you need READ and EXECUTE permissions at a filesystem level in the server\u2019s datadir.

The database user needs the following privileges to back up tables or databases:

  • RELOAD and FLUSH_TABLES in order to run FLUSH TABLES WITH READ LOCK.

  • The BACKUP_ADMIN privilege is needed to query the performance_schema.log_status table, and run LOCK INSTANCE FOR BACKUP, LOCK BINLOG FOR BACKUP, or LOCK TABLES FOR BACKUP. The BACKUP_ADMIN privilege is required to use the Page tracking feature.

  • REPLICATION CLIENT in order to obtain the binary log position,

  • CREATE TABLESPACE in order to import tables (see Restoring Individual Tables),

  • PROCESS in order to run SHOW ENGINE INNODB STATUS (which is mandatory), and optionally to see all threads which are running on the server (see FLUSH TABLES WITH READ LOCK option),

  • REPLICATION_SLAVE_ADMIN in order to start/stop the replication threads in a replication environment,

  • CREATE privilege in order to create the PERCONA_SCHEMA.xtrabackup_history database and table,

  • ALTER privilege in order to upgrade the PERCONA_SCHEMA.xtrabackup_history database and table,

  • INSERT privilege in order to add history records to the PERCONA_SCHEMA.xtrabackup_history table,

  • SELECT privilege in order to use --incremental-history-name or --incremental-history-uuid in order for the feature to look up the innodb_to_lsn values in the PERCONA_SCHEMA.xtrabackup_history table.

  • SELECT privilege on the keyring_component_status table to view the attributes and status of the installed keyring component when in use.

  • SELECT privilege on the replication_group_members table to validate if the instance is part of group replication cluster.

A SQL example of creating a database user with the minimum privileges required to take full backups would be:

mysql> CREATE USER 'bkpuser'@'localhost' IDENTIFIED BY 's3cr%T';\nmysql> GRANT BACKUP_ADMIN, PROCESS, RELOAD, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'bkpuser'@'localhost';\nmysql> GRANT SELECT ON performance_schema.log_status TO 'bkpuser'@'localhost';\nmysql> GRANT SELECT ON performance_schema.keyring_component_status TO bkpuser@'localhost';\nmysql> GRANT SELECT ON performance_schema.replication_group_members TO bkpuser@'localhost';\nmysql> FLUSH PRIVILEGES;\n
"},{"location":"privileges.html#query-the-privileges","title":"Query the privileges","text":"

To query the privileges that your database user has been granted at the console of the server execute:

mysql> SHOW GRANTS;\n

or for a particular user with:

mysql> SHOW GRANTS FOR 'db-user'@'host';\n

It will display the privileges using the same format as for the GRANT statement.

Note that privileges may vary across versions of the server. To list the exact list of privileges that your server support (and a brief description of them) execute:

mysql> SHOW PRIVILEGES;\n

See also

Permissions needed

"},{"location":"privileges.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"quickstart-overview.html","title":"Quickstart Guide for Percona XtraBackup 8.1","text":"

Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.

"},{"location":"quickstart-overview.html#install-percona-xtrabackup","title":"Install Percona XtraBackup:","text":"

You can install Percona XtraBackup using different methods:

  • Use the Percona Repositories

  • Use APT

  • Use YUM

  • Use binary tarballs

  • Use Docker

"},{"location":"quickstart-overview.html#for-superior-and-optimized-performance","title":"For superior and optimized performance","text":"

Percona Server for MySQL (PS) is a freely available, fully compatible, enhanced, and open source drop-in replacement for any MySQL database. It provides superior and optimized performance, greater scalability and availability, enhanced backups, increased visibility, and instrumentation. Percona Server for MySQL is trusted by thousands of enterprises to provide better performance and concurrency for their most demanding workloads.

Install Percona Server for MySQL.

"},{"location":"quickstart-overview.html#for-high-availability","title":"For high availability","text":"

Percona XtraDB Cluster (PXC) is a 100% open source, enterprise-grade, highly available clustering solution for MySQL multi-master setups based on Galera. PXC helps enterprises minimize unexpected downtime and data loss, reduce costs, and improve performance and scalability of your database environments supporting your critical business applications in the most demanding public, private, and hybrid cloud environments.

Install Percona XtraDB Cluster.

"},{"location":"quickstart-overview.html#for-monitoring-and-management","title":"For Monitoring and Management","text":"

Percona Monitoring and Management (PMM) monitors and provides actionable performance data for MySQL variants, including Percona Server for MySQL, Percona XtraDB Cluster, Oracle MySQL Community Edition, Oracle MySQL Enterprise Edition, and MariaDB. PMM captures metrics and data for the InnoDB, XtraDB, and MyRocks storage engines, and has specialized dashboards for specific engine details.

Install PMM and connect your MySQL instances to it.

"},{"location":"quickstart-overview.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"restore-a-backup.html","title":"Restore a backup","text":"

Warning

Backup needs to be prepared before it can be restored.

For convenience, xtrabackup binary has the --copy-back option to copy the backup to the datadir of the server:

$ xtrabackup --copy-back --target-dir=/data/backups/\n

If you don\u2019t want to save your backup, you can use the --move-back option which will move the backed up data to the datadir.

If you don\u2019t want to use any of the above options, you can additionally use rsync or cp to restore the files.

Note

The datadir must be empty before restoring the backup. Also, it\u2019s important to note that MySQL server needs to be shut down before restore is performed. You cannot restore to a datadir of a running mysqld instance (except when importing a partial backup).

Example of the rsync command that can be used to restore the backup can look like this:

$ rsync -avrP /data/backup/ /var/lib/mysql/\n

You should check that the restored files have the correct ownership and permissions.

As files\u2019 attributes are preserved, in most cases you must change the files\u2019 ownership to mysql before starting the database server, as the files are owned by the user who created the backup:

$ chown -R mysql:mysql /var/lib/mysql\n

Data is now restored, and you can start the server.

"},{"location":"restore-a-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"restore-individual-partitions.html","title":"Restore the partition from the backup","text":"

Restoring should be done by importing the tables in the partial backup to the server.

First step is to create new table in which data will be restored.

mysql> CREATE TABLE `name_p4` (\n`id` int(11) NOT NULL AUTO_INCREMENT,\n`name` text NOT NULL,\n`imdb_index` varchar(12) DEFAULT NULL,\n`imdb_id` int(11) DEFAULT NULL,\n`name_pcode_cf` varchar(5) DEFAULT NULL,\n`name_pcode_nf` varchar(5) DEFAULT NULL,\n`surname_pcode` varchar(5) DEFAULT NULL,\nPRIMARY KEY (`id`)\n) ENGINE=InnoDB AUTO_INCREMENT=2812744 DEFAULT CHARSET=utf8\n

Note

Generate a .cfg metadata file by running FLUSH TABLES ... FOR EXPORT. The command can only be run on a table, not on the individual table partitions. The file is located in the table schema directory and is used for schema verification when importing the tablespace.

To restore the partition from the backup, the tablespace must be discarded for that table:

mysql> ALTER TABLE name_p4 DISCARD TABLESPACE;\n

The next step is to copy the .ibd file from the backup to the MySQL data directory:

cp /mnt/backup/2012-08-28_10-29-09/imdb/name#P#p4.ibd /var/lib/mysql/imdb/name_p4.ibd\n

Note

Make sure that the copied files can be accessed by the user running MySQL.

The last step is to import the tablespace:

mysql> ALTER TABLE name_p4 IMPORT TABLESPACE;\n
"},{"location":"restore-individual-partitions.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"restore-individual-tables.html","title":"Restore individual tables","text":"

Percona XtraBackup can export a table that is contained in its own .ibd file. With Percona XtraBackup, you can export individual tables from any InnoDB database, and import them into Percona Server for MySQL with XtraDB or MySQL 8.1. The source doesn\u2019t have to be XtraDB or MySQL 8.1, but the destination does. This method only works on individual .ibd files.

The following example exports and imports the following table:

CREATE TABLE export_test (\na int(11) DEFAULT NULL\n) ENGINE=InnoDB DEFAULT CHARSET=latin1;\n
"},{"location":"restore-individual-tables.html#export-the-table","title":"Export the table","text":"

To generate an .ibd file in the target directory, create the table using the innodb_file_per_table mode:

$ find /data/backups/mysql/ -name export_test.*\n/data/backups/mysql/test/export_test.ibd\n

During the --prepare step, add the --export option to the command. For example:

$ xtrabackup --prepare --export --target-dir=/data/backups/mysql/\n

When restoring an encrypted InnoDB tablespace table, add the keyring file:

$ xtrabackup --prepare --export --target-dir=/tmp/table \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

The following files are the only files required to import the table into a server running Percona Server for MySQL with XtraDB or MySQL 8.1. If the server uses InnoDB Tablespace Encryption, add the .cfp file, which contains the transfer key and an encrypted tablespace key.

The files are located in the target directory:

/data/backups/mysql/test/export_test.ibd\n/data/backups/mysql/test/export_test.cfg\n
"},{"location":"restore-individual-tables.html#import-the-table","title":"Import the table","text":"

On the destination server running Percona Server for MySQL with XtraDB or MySQL 8.1, create a table with the same structure, and then perform the following steps:

  1. Run the ALTER TABLE test.export_test DISCARD TABLESPACE; command. If you see the following error message:

    Error message 
    ERROR 1809 (HY000): Table 'test/export_test' in system tablespace\n

    enable innodb_file_per_table option on the server and create the table again.

    $ set global innodb_file_per_table=ON;\n

  2. Copy the exported files to the test/ subdirectory of the destination server\u2019s data directory.

  3. Run ALTER TABLE test.export_test IMPORT TABLESPACE;

    The table is imported, and you can run a SELECT to see the imported data.

"},{"location":"restore-individual-tables.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"restore-partial-backup.html","title":"Restore a partial backup","text":"

Restoring should be done by restoring individual tables in the partial backup to the server.

It can also be done by copying back the prepared backup to a \u201cclean\u201d datadir (in that case, make sure to include the mysql database) to the datadir you are moving the backup to. A system database can be created with the following:

$ sudo mysql --initialize --user=mysql\n

Once you start the server, you may see mysql complaining about missing tablespaces:

Expected output 
2021-07-19T12:42:11.077200Z 1 [Warning] [MY-012351] [InnoDB] Tablespace 4, name 'test1/t1', file './d2/test1.ibd' is missing!\n2021-07-19T12:42:11.077300Z 1 [Warning] [MY-012351] [InnoDB] Tablespace 4, name 'test1/t1', file './d2/test1.ibd' is missing!\n

In order to clean the orphan database from the data dictionary, you must manually create the missing database directory and then DROP this database from the server.

Example of creating the missing database:

$ mkdir /var/lib/mysql/test1/d2\n

Example of dropping the database from the server:

mysql> DROP DATABASE d2;\n
Expected output 
Query OK, 2 rows affected (0.5 sec)\n
"},{"location":"restore-partial-backup.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"server-backup-version-comparison.html","title":"Server version and backup version comparison","text":"

A MySQL change to a feature, such as the structure of a redo log record, can cause older versions of Percona XtraBackup to fail. To ensure that you can back up and restore your data, use a Percona XtraBackup version that is equal to your source server major version. This means if you use Percona XtraBackup 8.1.x, you can safely back up the source server from 8.1.x to 8.1.xx.

See also

How XtraBackup works

The --no-server-version-check option performs the following test. Before the backup/prepare starts, XtraBackup compares the source server version to the Percona XtraBackup version. If the source server version is greater than the XtraBackup major version, XtraBackup stops the backup and returns an error message. This comparison prevents a failed backup or a corrupted backup due to source server changes.

The parameter checks for the following scenarios:

  • The source server and the PXB major version are the same, the backup proceeds

  • The source server is greater than the PXB major version, and the parameter is not overridden, the backup is stopped and returns an error message

  • The source server is less than the PXB major version, and the parameter is not overridden, the backup is stopped and returns an error message

  • The source server is greater than the PXB major version up to the last release in the LTS series, and the parameter is overridden, the backup proceeds

Explicitly adding the --no-server-version-check parameter, like the example, overrides the parameter and the backup proceeds.

$ xtrabackup --backup --no-server-version-check --target-dir=$mysql/backup1\n

Overriding the --no-server-version-check parameter allows taking backups using a Percona XtraBackup version that is equal to a version of your source server up to the last release in the LTS series. This means if you use Percona XtraBackup 8.1.x, you can back up the source server from 8.1.x to 8.4.xx.

When you override the parameter, the following events can happen:

  • Backup fails

  • Creates a corrupted backup

  • Backup successful

"},{"location":"server-backup-version-comparison.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"set-up-replication.html","title":"How to set up a replica for replication in 6 simple steps with Percona XtraBackup","text":"

Data is, by far, the most valuable part of a system. Having a backup done systematically and available for a rapid recovery in case of failure is admittedly essential to a system. However, it is not common practice because of its costs, infrastructure needed or even the boredom associated to the task. Percona XtraBackup is designed to solve this problem.

You can have almost real-time backups in 6 simple steps by setting up a replication environment with Percona XtraBackup.

"},{"location":"set-up-replication.html#things-you-need","title":"Things you need","text":"

Setting up a replica for replication with Percona XtraBackup is a straightforward procedure. In order to keep it simple, here is a list of the things you need to follow the steps without hassles:

Source

A system with a MySQL-based server installed, configured and running. This system is called Source. The Source server stores your data and can be replicated. We assume the following about this system:

  • the MySQL server is able to communicate with others by the standard TCP/IP port;

  • the SSH server is installed and configured;

  • you have a user account in the system with the appropriate permissions;

  • you have a MySQL\u2019s user account with appropriate privileges.

  • server has binlogs enabled and server-id set up to 1.

Replica

Another system, with a MySQL-based server installed on it. We refer to this machine as Replica and assume the same things we did about Source, except that the server-id on Replica is 2.

Percona XtraBackup

We use this backup tool. Install Percona XtraBackup on both computers for convenience.

Note

It is not recommended to mix MySQL variants (Percona Server, MySQL) in your replication setup. This may produce incorrect xtrabackup_slave_info file when adding a new replica.

"},{"location":"set-up-replication.html#1-make-a-backup-on-the-source-and-prepare-it","title":"1. Make a backup on the Source and prepare it","text":"

At the Source, issue the following to a shell:

$ xtrabackup --backup --user=yourDBuser --password=MaGiCdB1 --target-dir=/path/to/backupdir\n

After this is finished you should get:

Expected output 
xtrabackup: completed OK!\n

This operation makes a copy of your MySQL data dir to the /path/to/backupdir directory. You have told Percona XtraBackup to connect to the database server using your database user and password, and do a hot backup of all your data in it (all MyISAM, InnoDB tables and indexes in them).

In order for snapshot to be consistent, prepare the data on the source:

$ xtrabackup --prepare --target-dir=/path/to/backupdir\n

Select the path where your snapshot has been taken. Apply the transaction logs to the data files and your data files are ready to be used by the MySQL server.

Percona XtraBackup reads the my.cnf file to locate your data. If you have your configuration file in a non-standard place, you should use the flag --defaults-file =/location/of/my.cnf.

If you want to skip writing the username and password every time you want to access MySQL, you can set it up in .mylogin.cnf as follows:

mysql_config_editor set --login-path=client --host=localhost --user=root --password\n

For more information, see MySQL Configuration Utility.

This statement provides root access to MySQL.

"},{"location":"set-up-replication.html#2-copy-backed-up-data-to-the-replica","title":"2. Copy backed up data to the Replica","text":"

On the Source, use rsync or scp to copy the data from the Source to the Replica. If you are syncing the data directly to replica\u2019s data directory, we recommend that you stop the mysqld there.

$ rsync -avpP -e ssh /path/to/backupdir Replica:/path/to/mysql/\n

After data is copied, you can back up the original or previously installed MySQL datadir.

Note

Make sure mysqld is shut down before you move the contents of its datadir, or move the snapshot into its datadir.

Run the following commands on the Replica:

$ mv /path/to/mysql/datadir /path/to/mysql/datadir_bak\n

and move the snapshot from the Source in its place:

$ xtrabackup --move-back --target-dir=/path/to/mysql/backupdir\n

After you copy data over, make sure the Replica MySQL has the proper permissions to access them.

$ chown mysql:mysql /path/to/mysql/datadir\n

If the ibdata and iblog files are located in directories outside the datadir, you must put these files in their proper place after the logs have been applied.

"},{"location":"set-up-replication.html#3-configure-the-sources-mysql-server","title":"3. Configure the Source\u2019s MySQL server","text":"

On the source, run the following command to add the appropriate grant. This grant allows the replica to be able to connect to source:

mysql> GRANT REPLICATION SLAVE ON *.*  TO 'repl'@'$replicaip'\nIDENTIFIED BY '$replicapass';\n

Also make sure that firewall rules are correct and that the Replica can connect to the Source. Run the following command on the Replica to test that you can run the mysql client on Replica, connect to the Source, and authenticate.

mysql> mysql --host=Source --user=repl --password=$replicapass\n

Verify the privileges.

mysql> SHOW GRANTS;\n
"},{"location":"set-up-replication.html#4-configure-the-replicas-mysql-server","title":"4. Configure the Replica\u2019s MySQL server","text":"

Copy the my.cnf file from the Source to the Replica:

$ scp user@Source:/etc/mysql/my.cnf /etc/mysql/my.cnf\n

and change the following options in /etc/mysql/my.cnf:

server-id=2\n

and start/restart mysqld on the Replica.

In case you\u2019re using init script on Debian-based system to start mysqld, be sure that the password for debian-sys-maint user has been updated, and it\u2019s the same as that user\u2019s password on the Source. Password can be seen and updated in /etc/mysql/debian.cnf.

"},{"location":"set-up-replication.html#5-start-the-replication","title":"5. Start the replication","text":"

On the Replica, review the content of the xtrabackup_binlog_info file:

$ cat /var/lib/mysql/xtrabackup_binlog_info\n

The results should resemble the following:

Expected output 
Source-bin.000001     481\n

Do the following on a MySQL console and use the username and password you\u2019ve set up in STEP 3:

Use the CHANGE_REPLICATION_SOURCE_TO statement

CHANGE REPLICATION SOURCE TO\n    SOURCE_HOST='$sourceip',\n    SOURCE_USER='repl',\n    SOURCE_PASSWORD='$replicapass',\n    SOURCE_LOG_FILE='Source-bin.000001',\n    SOURCE_LOG_POS=481;\n

Start the replica:

START REPLICA;\n
"},{"location":"set-up-replication.html#6-check","title":"6. Check","text":"

On the Replica, check that everything went OK with:

SHOW REPLICA STATUS\\G\n

The result shows the status:

Expected output 
Slave_IO_Running: Yes\nSlave_SQL_Running: Yes\nSeconds_Behind_Master: 13\n

Both IO and SQL threads need to be running. The Seconds_Behind_Master means the SQL currently being executed has a current_timestamp of 13 seconds ago. It is an estimation of the lag between the Source and the Replica. Note that at the beginning, a high value could be shown because the Replica has to \u201ccatch up\u201d with the Source.

"},{"location":"set-up-replication.html#adding-more-replicas-to-the-source","title":"Adding more replicas to the Source","text":"

You can use this procedure with slight variation to add new replicas to a source. We will use Percona XtraBackup to clone an already configured replica. We will continue using the previous scenario for convenience, but we will add a NewReplica to the plot.

At the Replica, do a full backup:

$ xtrabackup --user=yourDBuser --password=MaGiCiGaM \\\n   --backup --slave-info --target-dir=/path/to/backupdir\n

By using the --slave-info Percona XtraBackup creates additional file called xtrabackup_slave_info.

Apply the logs:

$ xtrabackup --prepare --use-memory=2G --target-dir=/path/to/backupdir/\n

Note

In the prepare phase, the --use-memory parameter speeds up the process if the amount of RAM assigned to the option is available. Use the parameter only in the prepare phase. In the other phases the parameter makes the application lazy allocate this memory (reserve) but does not affect database pages.

Copy the directory from the Replica to the NewReplica:

Note

Make sure mysqld is shut down on the NewReplica before you copy the contents the snapshot into its datadir.

rsync -avprP -e ssh /path/to/backupdir NewReplica:/path/to/mysql/datadir\n

For example, to set up a new user, user2, you add another grant on the Source:

> GRANT REPLICATION SLAVE ON *.*  TO 'user2'@'$newreplicaip'\n IDENTIFIED BY '$replicapass';\n

On the NewReplica, copy the configuration file from the Replica:

$ scp user@Replica:/etc/mysql/my.cnf /etc/mysql/my.cnf\n

Make sure you change the server-id variable in /etc/mysql/my.cnf to 3 and disable the replication on start:

skip-slave-start\nserver-id=3\n

After setting server_id, start mysqld.

Fetch the source_log_file and source_log_pos from the file xtrabackup_slave_info, execute the statement for setting up the source and the log file for the NewReplica:

CHANGE REPLICATION SOURCE TO\n    SOURCE_HOST='$sourceip',\n    SOURCE_USER='repl',\n    SOURCE_PASSWORD='$replicapass',\n    SOURCE_LOG_FILE='Source-bin.000001',\n    SOURCE_LOG_POS=481;\n

Start the replica:

> START REPLICA;\n

If both IO and SQL threads are running when you check the NewReplica, server is replicating the Source.

See also

How to create a new (or repair a broken) GTID based slave

"},{"location":"set-up-replication.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"smart-memory-estimation.html","title":"Smart memory estimation","text":"

The Smart memory estimation is tech preview feature. Before using Smart memory estimation in production, we recommend that you test restoring production from physical backups in your environment and also use the alternative backup method for redundancy.

Percona XtraBackup supports the Smart memory estimation feature. With this feature, Percona XtraBackup computes the memory required for prepare phase, while copying redo log entries during the backup phase. Percona XtraBackup also considers the number of InnoDB pages to be fetched from the disk.

Percona XtraBackup performs the backup procedure in two steps:

  • Creates a backup

    To create a backup, Percona XtraBackup copies your InnoDB data files. While copying the files, Percona XtraBackup runs a background process that watches the InnoDB redo log, also called the transaction log, and copies changes from it.

  • Prepares a backup

    During the prepare phase, Percona XtraBackup performs crash recovery against the copied data files using the copied transaction log file. Percona XtraBackup reads all the redo log entries into memory, categorizes them by space id and page id, reads the relevant pages into memory, and checks the log sequence number (LSN) on the page and on the redo log record. If the redo log LSN is more recent than the page LSN, Percona XtraBackup applies the redo log changes to the page.

    To prepare a backup, Percona Xtrabackup uses InnoDB Buffer Pool memory. Percona Xtrabackup reserves memory to load 256 pages into the buffer pool. The remaining memory is used for hashing/categorizing the redo log entries.

    The available memory is controlled by the --use-memory option. If the available memory on the buffer pool is insufficient, the work is performed in multiple batches. After the batch is processed, the memory is freed to release space for the next batch. This process greatly impacts performance as an InnoDB page holds data from multiple rows. If a change on a page happens in different batches, that page is fetched and evicted numerous times.

"},{"location":"smart-memory-estimation.html#how-does-smart-memory-estimation-work","title":"How does Smart memory estimation work","text":"

To run prepare, Percona XtraBackup checks the server\u2019s available free memory and uses that memory up to the limit specified in the --use-free-memory-pct option. Due to backward compatibility, the default value for the --use-free-memory-pct option is 0 (zero), which defines the option as disabled. For example, if you set --use-free-memory-pct=50, then 50% of the free memory is used to prepare a backup.

You can enable or disable the memory estimation during the backup phase with the --estimate-memory option. The default value is OFF. Enable the memory estimation with --estimate-memory=ON:

$ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n

In the prepare phase, enable the --use-free-memory-pct option by specifying the percentage of free memory to be used to prepare a backup. The --use-free-memory-pct value must be larger than 0.

For example:

$ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\n
"},{"location":"smart-memory-estimation.html#example-of-smart-memory-estimation-usage","title":"Example of Smart memory estimation usage","text":"

The examples of how Smart memory estimation can improve the time spent on prepare in Percona XtraBackup:

We back up 16, 32, and 64 tables using sysbench. Each set contains 1M rows. In the backup phase, we enable Smart memory estimation with --estimate-memory=ON. In the prepare phase, we set --use-free-memory-pct=50, and Percona XtraBackup uses 50% of the free memory to prepare a backup. The backup is run on an ec2 c4.8xlarge instance (36 vCPUs / 60GB memory / General Purpose SSD (gp2)).

During each --backup, the following sysbench is run:

sysbench --db-driver=mysql --db-ps-mode=disable --mysql-user=sysbench --mysql-password=sysbench --table_size=1000000 --tables=${NUM_OF_TABLES} --threads=24 --time=0 --report-interval=1 /usr/share/sysbench/oltp_write_only.lua run\n

The following table shows the backup details (all measurements are in Gigabytes):

Used memory Size of XtraBackup log Size of backup 16 tables 3.375 0.7 4.7 32 tables 8.625 2.6 11 64 tables 18.5 5.6 22

  • Used memory - the amount of memory required by Percona XtraBackup with --use-free-memory-pct=50

  • Size of XtraBackup log - the size of Percona XtraBackup log file (redo log entries copied during the backup)

  • Size of backup - the size of the resulting backup folder

Prepare executed without Smart memory estimation uses the default of 128MB for the buffer pool.

The results are the following:

Note

The following results are based on tests in a specific environment. Your results may vary.

  • 16 tables result - prepare time dropped to ~5.7% of the original time. An improvement in recovery time of about 17x.

  • 32 tables result - prepare time dropped to ~8,2% of the original time. An improvement in recovery time of about 12x.

  • 64 tables result - prepare time dropped to ~9.9% of the original time. An improvement in recovery time of about 10x.

"},{"location":"smart-memory-estimation.html#get-expert-help","title":"Get expert help","text":"

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

"},{"location":"store-backup-history.html","title":"Store backup history on the server","text":"

Percona XtraBackup supports storing the backups history on the server. Storing backup history on the server was implemented to provide users with additional information about backups that are being taken. Backup history information will be stored in the PERCONA_SCHEMA.XTRABACKUP_HISTORY table.

To use this feature the following options are available:

  • --history = : This option enables the history feature and allows the user to specify a backup series name that will be placed within the history record.

  • --incremental-history-name = : This option allows an incremental backup to be made based on a specific history series by name. xtrabackup will search the history table looking for the most recent (highest to_lsn) backup in the series and take the to_lsn value to use as it\u2019s starting lsn. This is mutually exclusive with --incremental-history-uuid, --incremental-basedir and --incremental-lsn options. If no valid LSN can be found (no series by that name) xtrabackup will return with an error.

  • --incremental-history-uuid = : Allows an incremental backup to be made based on a specific history record identified by UUID. xtrabackup will search the history table looking for the record matching UUID and take the to_lsn value to use as it\u2019s starting LSN. This options is mutually exclusive with --incremental-basedir, --incremental-lsn and --incremental-history-name options. If no valid LSN can be found (no record by that UUID or missing to_lsn), xtrabackup will return with an error.

    Note

    Backup that\u2019s currently being performed will NOT exist in the xtrabackup_history table within the resulting backup set as the record will not be added to that table until after the backup has been taken.

    If you want access to backup history outside of your backup set in the case of some catastrophic event, you will need to either perform a mysqldump, partial backup or SELECT * on the history table after xtrabackup completes and store the results with you backup set.

    For the necessary privileges, see Permissions and Privileges Needed.

    "},{"location":"store-backup-history.html#percona_schemaxtrabackup_history-table","title":"PERCONA_SCHEMA.XTRABACKUP_HISTORY table","text":"

    This table contains the information about the previous server backups. Information about the backups will only be written if the backup was taken with --history option.

    Column Name Description uuid Unique backup id name User provided name of backup series. There may be multiple entries with the same name used to identify related backups in a series. tool_name Name of tool used to take backup tool_command Exact command line given to the tool with \u2013password and \u2013encryption_key obfuscated tool_version Version of tool used to take backup ibbackup_version Version of the xtrabackup binary used to take backup server_version Server version on which backup was taken start_time Time at the start of the backup end_time Time at the end of the backup lock_time Amount of time, in seconds, spent calling and holding locks for FLUSH TABLES WITH READ LOCK binlog_pos Binlog file and position at end of FLUSH TABLES WITH READ LOCK innodb_from_lsn LSN at beginning of backup which can be used to determine prior backups innodb_to_lsn LSN at end of backup which can be used as the starting lsn for the next incremental partial Is this a partial backup, if N that means that it\u2019s the full backup incremental Is this an incremental backup format Description of result format (xbstream) compact Is this a compact backup compressed Is this a compressed backup encrypted Is this an encrypted backup"},{"location":"store-backup-history.html#limitations","title":"Limitations","text":"

    • --history option must be specified only on the command line and not within a configuration file in order to be effective.

    • --incremental-history-name and --incremental-history-uuid options must be specified only on the command line and not within a configuration file in order to be effective.

    "},{"location":"store-backup-history.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"take-streaming-backup.html","title":"Take a streaming backup","text":"

    Percona XtraBackup supports streaming mode. Streaming mode sends a backup to STDOUT in the xbstream format instead of copying the files to the backup directory.

    This method allows you to use other programs to filter the output of the backup, providing greater flexibility for storage of the backup. For example, compression is achieved by piping the output to a compression utility. One of the benefits of streaming backups and using Unix pipes is that the backups can be automatically encrypted.

    To use the streaming feature, you must use the --stream, providing the format of the stream (xbstream ) and where to store the temporary files:

    $ xtrabackup --stream=xbstream --target-dir=/tmp\n

    xtrabackup uses xbstream to stream all of the data files to STDOUT, in a special xbstream format. After it finishes streaming all of the data files to STDOUT, it stops xtrabackup and streams the saved log file too.

    When compression is enabled, xtrabackup compresses the output data, except for the meta and non-InnoDB files which are not compressed, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:

    "},{"location":"take-streaming-backup.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

    The Zstandard (ZSTD) is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. ZSTD is the default compression algorithm for the --compress option.

    To compress files using the ZSTD compression algorithm, use the --compress option:

    $ xtrabackup --backup --compress --target-dir=/data/backup\n

    The resulting files have the \\*.zst format.

    You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.

    $ xtrabackup \u2013backup \u2013compress \u2013compress-zstd-level=1 \u2013target-dir=/data/backup\n
    "},{"location":"take-streaming-backup.html#lz4","title":"lz4","text":"

    To compress files using the lz4 compression algorithm, set the --compress option to lz4:

    $ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\n

    The resulting files have the \\*.lz4 format.

    To decompress files, use the --decompress option.

    Using xbstream as a stream option, backups can be copied and compressed in parallel. This option can significantly improve the speed of the backup process. In case backups were both compressed and encrypted, they must be decrypted before they are uncompressed.

    Task Command Stream the backup into an archived named backup.xbstream $ xtrabackup --backup --stream=xbstream --target-dir=./ > backup.xbstream Stream the backup into a compressed archive named backup.xbstream $ xtrabackup --backup --stream=xbstream --compress --target-dir=./ > backup.xbstream Encrypt the backup $ xtrabackup --backup --stream=xbstream ./ > backup.xbstream gzip -`` | openssl des3 -salt -k \u201cpassword\u201d backup.xbstream.gz.des3 Unpack the backup to the current directory $ xbstream -x < backup.xbstream Send the backup compressed directly to another host and unpack it $ xtrabackup --backup --compress --stream=xbstream --target-dir=./ | ssh user@otherhost \"xbstream -x\" Send the backup to another server using netcat On the destination host:$ nc -l 9999 | cat - > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | nc desthost 9999 Send the backup to another server using a one-liner $ ssh user@desthost \u201c( nc -l 9999 > /data/backups/backup.xbstream & )\u201d && xtrabackup --backup --stream=xbstream ./ | nc desthost 9999 Throttle the throughput to 10MB/sec using the pipe viewer tool $ xtrabackup --backup --stream=xbstream ./ | pv -q -L10m ssh user@desthost \u201ccat - > /data/backups/backup.xbstream\u201d Checksum the backup during the streaming On the destination host:$ nc -l 9999 | tee >(sha1sum > destination_checksum) > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | tee >(sha1sum > source_checksum) | nc desthost 9999Compare the checksums on the source host:$ cat source_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbadCompare the checksums on the destination host:$ cat destination_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbad Parallel compression with parallel copying backup $ xtrabackup --backup --compress --compress-threads=8 --stream=xbstream --parallel=4 --target-dir=./ > backup.xbstream

    Important

    The streamed backup must be prepared before restoration. Streaming mode does not prepare the backup.

    "},{"location":"take-streaming-backup.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"throttling-backups.html","title":"Throttling backups","text":"

    Although xtrabackup does not block your database\u2019s operation, any backup can add load to the system being backed up. On systems that do not have much spare I/O capacity, it might be helpful to throttle the rate at which xtrabackup reads and writes data. You can do this with the --throttle option. This option limits the number of chunks copied per second. The chunk +size is 10 MB.

    The image below shows how throttling works when --throttle is set to 1.

    When specified with the --backup option, this option limits the number of pairs of read-and-write operations per second that xtrabackup will perform. If you are creating an incremental backup, then the limit is the number of read I/O operations per second.

    By default, there is no throttling, and xtrabackup reads and writes data as quickly as it can. If you set too strict of a limit on the IOPS, the backup might be so slow that it will never catch up with the transaction logs that InnoDB is writing, so the backup might never complete.

    "},{"location":"throttling-backups.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"toolkit-version-check.html","title":"Percona Toolkit version checking","text":"

    Some Percona software contains \u201cversion checking\u201d functionality which is a feature that enables Percona software users to be notified of available software updates to improve your environment security and performance. Alongside this, the version check functionality also provides Percona with information relating to which software versions you are running, coupled with the environment confirmation which the software is running within. This helps enable Percona to focus our development effort accordingly based on trends within our customer community.

    The purpose of this document is to articulate the information that is collected, as well as to provide guidance on how to disable this functionality if desired.

    "},{"location":"toolkit-version-check.html#usage","title":"Usage","text":"

    Version Check was implemented in Percona Toolkit 2.1.4, and was enabled by default in version 2.2.1. Currently, it is supported as a --[no]version-check option by a number of tools in Percona Toolkit, Percona XtraBackup, and Percona Monitoring and Management (PMM).

    When launched with Version Check enabled, the tool that supports this feature connects to a Percona\u2019s version check service via a secure HTTPS channel. It compares the locally installed version for possible updates, and also checks versions of the following software:

    • Operating System

    • Percona Monitoring and Management (PMM)

    • MySQL

    • Perl

    • MySQL driver for Perl (DBD::mysql)

    • Percona Toolkit

    Then it checks for and warns about versions with known problems if they are identified as running in the environment.

    Each version check request is logged by the server. Stored information consists of the checked system unique ID followed by the software name and version. The ID is generated either at installation or when the version checking query is submitted for the first time.

    Note

    Prior to version 3.0.7 of Percona Toolkit, the system ID was calculated as an MD5 hash of a hostname, and starting from Percona Toolkit 3.0.7 it is generated as an MD5 hash of a random number. Percona XtraBackup continues to use hostname-based MD5 hash.

    As a result, the content of the sent query is as follows:

    Expected output 
    85624f3fb5d2af8816178ea1493ed41a;DBD::mysql;4.044\nc2b6d625ef3409164cbf8af4985c48d3;MySQL;MySQL Community Server (GPL) 5.7.22-log\n85624f3fb5d2af8816178ea1493ed41a;OS;Manjaro Linux\n85624f3fb5d2af8816178ea1493ed41a;Percona::Toolkit;3.0.11-dev\n85624f3fb5d2af8816178ea1493ed41a;Perl;5.26.2\n
    "},{"location":"toolkit-version-check.html#disabling-version-check","title":"Disabling version check","text":"

    Although the version checking feature does not collect any personal information, you might prefer to disable this feature, either one time or permanently. To disable it one time, use --no-version-check option when invoking the tool from a Percona product which supports it. Here is a simple example which shows running pt-diskstats tool from the Percona Toolkit with version checking turned off:

    pt-diskstats --no-version-check\n

    Disabling version checking permanently can be done by placing no-version-check option into the configuration file of a Percona product (see correspondent documentation for exact file name and syntax). For example, in case of Percona Toolkit this can be done in a global configuration file /etc/percona-toolkit/percona-toolkit.conf:

    # Disable Version Check for all tools:\nno-version-check\n

    In case of Percona XtraBackup this can be done in its configuration file in a similar way:

    [xtrabackup]\nno-version-check\n
    "},{"location":"toolkit-version-check.html#frequently-asked-questions","title":"Frequently asked questions","text":""},{"location":"toolkit-version-check.html#why-is-this-functionality-enabled-by-default","title":"Why is this functionality enabled by default?","text":"

    We believe having this functionality enabled improves security and performance of environments running Percona Software and it is good choice for majority of the users.

    "},{"location":"toolkit-version-check.html#why-not-rely-on-operating-systems-built-in-functionality-for-software-updates","title":"Why not rely on Operating System\u2019s built in functionality for software updates?","text":"

    In many environments the Operating Systems repositories may not carry the latest version of software and newer versions of software often installed manually, so not being covered by operating system wide check for updates.

    "},{"location":"toolkit-version-check.html#why-do-you-send-more-information-than-just-the-version-of-software-being-run-as-a-part-of-version-check-service","title":"Why do you send more information than just the version of software being run as a part of version check service?","text":"

    Compatibility problems can be caused by versions of various components in the environment, for example problematic versions of Perl, DBD or MySQL could cause operational problems with Percona Toolkit.

    "},{"location":"toolkit-version-check.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"trademark-policy.html","title":"Trademark policy","text":"

    This Trademark Policy is to ensure that users of Percona-branded products or services know that what they receive has really been developed, approved, tested and maintained by Percona. Trademarks help to prevent confusion in the marketplace, by distinguishing one company\u2019s or person\u2019s products and services from another\u2019s.

    Percona owns a number of marks, including but not limited to Percona, XtraDB, Percona XtraDB, XtraBackup, Percona XtraBackup, Percona Server, and Percona Live, plus the distinctive visual icons and logos associated with these marks. Both the unregistered and registered marks of Percona are protected.

    Use of any Percona trademark in the name, URL, or other identifying characteristic of any product, service, website, or other use is not permitted without Percona\u2019s written permission with the following three limited exceptions.

    First, you may use the appropriate Percona mark when making a nominative fair use reference to a bona fide Percona product.

    Second, when Percona has released a product under a version of the GNU General Public License (\u201cGPL\u201d), you may use the appropriate Percona mark when distributing a verbatim copy of that product in accordance with the terms and conditions of the GPL.

    Third, you may use the appropriate Percona mark to refer to a distribution of GPL-released Percona software that has been modified with minor changes for the sole purpose of allowing the software to operate on an operating system or hardware platform for which Percona has not yet released the software, provided that those third party changes do not affect the behavior, functionality, features, design or performance of the software. Users who acquire this Percona-branded software receive substantially exact implementations of the Percona software.

    Percona reserves the right to revoke this authorization at any time in its sole discretion. For example, if Percona believes that your modification is beyond the scope of the limited license granted in this Policy or that your use of the Percona mark is detrimental to Percona, Percona will revoke this authorization. Upon revocation, you must immediately cease using the applicable Percona mark. If you do not immediately cease using the Percona mark upon revocation, Percona may take action to protect its rights and interests in the Percona mark. Percona does not grant any license to use any Percona mark for any other modified versions of Percona software; such use will require our prior written permission.

    Neither trademark law nor any of the exceptions set forth in this Trademark Policy permit you to truncate, modify or otherwise use any Percona mark as part of your own brand. For example, if XYZ creates a modified version of the Percona Server, XYZ may not brand that modification as \u201cXYZ Percona Server\u201d or \u201cPercona XYZ Server\u201d, even if that modification otherwise complies with the third exception noted above.

    In all cases, you must comply with applicable law, the underlying license, and this Trademark Policy, as amended from time to time. For instance, any mention of Percona trademarks should include the full trademarked name, with proper spelling and capitalization, along with attribution of ownership to Percona LLC and/or its affiliates. For example, the full proper name for XtraBackup is Percona XtraBackup. However, it is acceptable to omit the word \u201cPercona\u201d for brevity on the second and subsequent uses, where such omission does not cause confusion.

    In the event of doubt as to any of the conditions or exceptions outlined in this Trademark Policy, please contact trademarks@percona.com for assistance and we will do our very best to be helpful.

    "},{"location":"trademark-policy.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"update-curl-utility.html","title":"Update curl utility","text":""},{"location":"update-curl-utility.html#update-the-curl-utility-in-debian-10","title":"Update the curl utility in Debian 10","text":"

    The default curl version, 7.64.0, in Debian 10 has known issues when attempting to reuse an already closed connection. This issue directly affects xbcloud and users may see intermittent backup failures.

    For more details, see curl #3750 or curl #3763.

    Follow these steps to upgrade curl to version 7.74.0:

    1. Edit the /etc/apt/sources.list to add the following:

      deb http://ftp.de.debian.org/debian buster-backports main\n

    2. Refresh the apt sources:

      $ sudo apt update\n

    3. Install the version from buster-backports:

      $ sudo apt install curl/buster-backports\n

    4. Verify the version number:

      $ curl --version\n
      Expected output 
      curl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0\n
    "},{"location":"update-curl-utility.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"varify-backup.html","title":"Verify backups with replication and pt-checksum","text":"

    One way to verify if the backup is consistent is by setting up the replication and running pt-table-checksum. This can be used to verify any type of backups, but before setting up replication, backup should be prepared and be able to run (this means that incremental backups should be merged to full backups, encrypted backups decrypted etc.).

    "},{"location":"varify-backup.html#set-up-the-replication","title":"Set up the replication","text":"

    How to set up a replica for replication in 6 simple steps with Percona XtraBackup guide provides a detailed instructions on how to take the backup and set up the replication.

    For checking the backup consistency you can use either the original server where the backup was taken, or another test server created by using a different backup method (such as cold backup, mysqldump or LVM snapshots) as the source server in the replication setup.

    "},{"location":"varify-backup.html#use-pt-table-checksum","title":"Use pt-table-checksum","text":"

    This tool is part of the Percona Toolkit. It performs an online replication consistency check by executing checksum queries on the source, which produces different results on replicas that are inconsistent with the source.

    After you confirmed that replication has been set up successfully, you can install or download pt-table-checksum. This example shows downloading the latest version of pt-table-checksum:

    $ wget percona.com/get/pt-table-checksum\n

    Note

    In order for pt-table-checksum to work correctly libdbd-mysql-perl will need to be installed on Debian/Ubuntu systems or perl-DBD-MySQL on RHEL/CentOS. If you installed the percona-toolkit package from the Percona repositories package manager should install those libraries automatically.

    After this command has been run, pt-table-checksum will be downloaded to your current working directory.

    Running the pt-table-checksum on the source will create percona database with the checksums table which will be replicated to the replicas as well. Example of the pt-table-checksum will look like this:

    $ ./pt-table-checksum\n

    The results are similar to the following:

    Expected output 
    TS ERRORS  DIFFS     ROWS  CHUNKS SKIPPED    TIME TABLE\n04-30T11:31:50      0      0   633135       8       0   5.400 exampledb.aka_name\n04-30T11:31:52      0      0   290859       1       0   2.692 exampledb.aka_title\nChecksumming exampledb.user_info:  16% 02:27 remain\nChecksumming exampledb.user_info:  34% 01:58 remain\nChecksumming exampledb.user_info:  50% 01:29 remain\nChecksumming exampledb.user_info:  68% 00:56 remain\nChecksumming exampledb.user_info:  86% 00:24 remain\n04-30T11:34:38      0      0 22187768     126       0 165.216 exampledb.user_info\n04-30T11:38:09      0      0        0       1       0   0.033 mysql.time_zone_name\n04-30T11:38:09      0      0        0       1       0   0.052 mysql.time_zone_transition\n04-30T11:38:09      0      0        0       1       0   0.054 mysql.time_zone_transition_type\n04-30T11:38:09      0      0        8       1       0   0.064 mysql.user\n

    If all the values in the DIFFS column are 0 that means that backup is consistent with the current setup.

    In case backup was not consistent pt-table-checksum should spot the difference and point to the table that does not match. Following example shows adding new user on the backed up replica in order to simulate the inconsistent backup:

    mysql> grant usage on exampledb.* to exampledb@localhost identified by 'thisisnewpassword';\n

    If we run the pt-table-checksum now difference should be spotted

    $ ./pt-table-checksum\n

    The results are similar to the following:

    Expected output 
    TS ERRORS  DIFFS     ROWS  CHUNKS SKIPPED    TIME TABLE\n04-30T11:31:50      0      0   633135       8       0   5.400 exampledb.aka_name\n04-30T11:31:52      0      0   290859       1       0   2.692 exampledb.aka_title\nChecksumming exampledb.user_info:  16% 02:27 remain\nChecksumming exampledb.user_info:  34% 01:58 remain\nChecksumming exampledb.user_info:  50% 01:29 remain\nChecksumming exampledb.user_info:  68% 00:56 remain\nChecksumming exampledb.user_info:  86% 00:24 remain\n04-30T11:34:38      0      0 22187768     126       0 165.216 exampledb.user_info\n04-30T11:38:09      0      0        0       1       0   0.033 mysql.time_zone_name\n04-30T11:38:09      0      0        0       1       0   0.052 mysql.time_zone_transition\n04-30T11:38:09      0      0        0       1       0   0.054 mysql.time_zone_transition_type\n04-30T11:38:09      1      0        8       1       0   0.064 mysql.user\n

    This output shows that source and the replica aren\u2019t in consistent state and that the difference is in the mysql.user table.

    More information on different options that pt-table-checksum provides can be found in the pt-table-checksum documentation.

    "},{"location":"varify-backup.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"work-with-apparmor.html","title":"Work with AppArmor","text":"

    The Linux Security Module implements mandatory access controls (MAC) with AppArmor. Debian and Ubuntu systems install AppArmor by default. AppArmor uses profiles which define which files and permissions are needed for application.

    Percona XtraBackup does not have a profile and is not confined by AppArmor.

    For a list of common AppArmor commands, see Percona Server for MySQL - AppArmor.

    "},{"location":"work-with-apparmor.html#develop-a-profile","title":"Develop a profile","text":"

    Download the profile from:

    https://github.com/percona/percona-xtrabackup/tree/trunk/packaging/percona/apparmor/apparmor.d

    The following profile sections should be updated with your system information, such as location of the backup destination directory.

    Expected output 
    # enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\n

    Move the updated file:

    $ sudo mv usr.sbin.xtrabackup /etc/apparmor.d/\n

    Install the profile with the following command:

    $ sudo apparmor_parser -r -T -W /etc/apparmor.d/usr.sbin.xtrabackup\n

    Run the backup as usual.

    No additional AppArmor-related actions are required to restore a backup.

    "},{"location":"work-with-apparmor.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"work-with-selinux.html","title":"Work with SELinux","text":"

    Percona XtraBackup is installed as an unconfined process running in an undefined domain. SELinux allows unconfined processes almost all access and the processes only use Discretionary Access Control (DAC) rules.

    You find the current state of the Percona XtraBackup file with the following command:

    $ ls -Z /usr/bin | grep xtrabackup\n
    Expected output 
    -rwxr-xr-x. root root   system_u:object_r:bin_t:s0       xtrabackup\n

    The SELinux context is the following:

    • user (root)

    • role (object_r)

    • type (bin_t)

    • level (s0)

    The unconfined domain supports the network-facing services, which are protected by SELinux. These domains are not exposed. In this configuration, SELinux protects against remote intrusions but local intrusions, which require local access, are not confined.

    Percona XtraBackup works locally. The service is not network-facing and cannot be exploited externally. The service interacts only with the local user, who provides the parameters. Percona XtraBackup requires access to the target-dir location.

    "},{"location":"work-with-selinux.html#confine-xtrabackup","title":"Confine XtraBackup","text":"

    You can modify your security configuration to confine Percona XtraBackup. The first question is where to store the backup files. The service requires read and write access to the selected location.

    You can use either of the following methods:

    • Allow Percona XtraBackup to write to any location. The user provides any path to the target-dir parameter.

    • Allow Percona XtraBackup to write to a specific location, such as /backups or the user\u2019s home directory.

    The first option opens the entire system to read and write. Select the second option to harden your security.

    "},{"location":"work-with-selinux.html#install-selinux-tools","title":"Install SELinux tools","text":"

    To work with policies, you must install the SELinux tools. To find which package provides the semanage command and install the package. The following is an example on CentOS 7.

    $ yum provides *bin/semanage\n
    The result should list the packages.

    Expected output 
    ...\npolicycoreutils-python-2.5-34.el7.x86_64 : SELinux policy core python utilities\n...\n

    To install missing packages, run the following:

    $ sudo yum install -y policycoreutils-python\n

    The following is an example on CentOS 8:

    $ yum provides *bin/semanage\n
    The result should list the missing packages.

    Expected output 
    ...\npolicycoreutils-python-utils-2.8-16.1.el8.noarch : SELinux policy core python utilities\n

    Run the following to install the missing packages: 

    $ sudo yum install -y policycoreutils-python-utils\n
    "},{"location":"work-with-selinux.html#create-a-policy","title":"Create a policy","text":"

    Use a modular approach to create an SELinux policy. Create a policy module to manage XtraBackup. You must create a .te file for type enforcement, and an optional .fc file for the file contexts.

    Use $ ps -efZ | grep xtrabackup to verify the service is not confined by SELinux.

    Create the xtrabackup.fc file and add content. This file defines the security contexts.

    /usr/bin/xtrabackup    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcrypt    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbstream    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcloud    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/backups(/.*)?       system_u:object_r:xtrabackup_data_t:s0\n

    Note

    If you are using the /backups directory you must have the last line. If you are storing the backups in the user\u2019s home directory, you can omit this line.

    Download the xtrabackup.te file from the following location:

    https://github.com/percona/percona-xtrabackup/tree/trunk/packaging/percona/selinx

    Note

    In the file, the sections in bold should be modified for your system. The fc file can also be downloaded from the same location.

    Compile the policy module:

    $ make -f /usr/share/selinux/devel/Makefile xtrabackup.pp\n

    Install the module:

    $ semodule -i xtrabackup.pp\n

    Tag the PXB binaries with the proper SELinux tags, such as xtrabackup_exec_t.

    $ restorecon -v /usr/bin/*\n

    If you store your backups at /backups, restore the tag in that location:

    $ restorecon -v /backups\n

    Note

    Remember to add the standard Linux DAC permissions for this directory.

    Perform the backup in the standard way.

    "},{"location":"work-with-selinux.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"working-with-binary-logs.html","title":"Work with binary logs","text":"

    The xtrabackup binary integrates with the log_status table. This integration enables xtrabackup to print out the backup\u2019s corresponding binary log position, so that you can use this binary log position to provision a new replica or perform point-in-time recovery.

    "},{"location":"working-with-binary-logs.html#find-the-binary-log-position","title":"Find the binary log position","text":"

    You can find the binary log position corresponding to a backup after the backup has been taken. If your backup is from a server with binary logging enabled, xtrabackup creates a file named xtrabackup_binlog_info in the target directory. This file contains the binary log file name and position of the exact point when the backup was taken.

    Expected output during the backup stage 
    210715 14:14:59 Backup created in directory '/backup/'\nMySQL binlog position: filename 'binlog.000002', position '156'\n. . .\n210715 14:15:00 completed OK!\n
    "},{"location":"working-with-binary-logs.html#point-in-time-recovery","title":"Point-in-time recovery","text":"

    To perform a point-in-time recovery from an xtrabackup backup, you should prepare and restore the backup, and then replay binary logs from the point shown in the xtrabackup_binlog_info file.

    Find a more detailed procedure in the Point-in-time recovery document.

    "},{"location":"working-with-binary-logs.html#set-up-a-new-replication-replica","title":"Set up a new replication replica","text":"

    To set up a new replica, you should prepare the backup, and restore it to the data directory of your new replication replica. In the CHANGE_REPLICATION_SOURCE_TO with the appropriate options command, use the binary log filename and position shown in the xtrabackup_binlog_info file to start replication.

    A more detailed procedure is found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.

    "},{"location":"working-with-binary-logs.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"xbcloud-azure.html","title":"Use the xbcloud binary with Microsoft Azure Cloud Storage","text":"

    The xbcloud binary adds support for the Microsoft Azure Cloud Storage using the REST API.

    "},{"location":"xbcloud-azure.html#options","title":"Options","text":"

    The following are the options, environment variables, and descriptions for uploading a backup to Azure using the REST API. The environment variables are recognized by xbcloud, which maps them automatically to the corresponding parameters:

    Option name Environment variables Description \u2013azure-storage-account=name AZURE_STORAGE_ACCOUNT An Azure storage account is a unique namespace to access and store your Azure data objects. \u2013azure-container-name=name AZURE_CONTAINER_NAME A container name is a valid DNS name that conforms to the Azure naming rules \u2013azure-access-key=name AZURE_ACCESS_KEY A generated key that can be used to authorize access to data in your account using the Shared Key authorization. \u2013azure-endpoint=name AZURE_ENDPOINT The endpoint allows clients to securely access data \u2013azure-tier-class=name AZURE_STORAGE_CLASS Cloud tier can decrease the local storage required while maintaining the performance. When enabled, this feature has the following categories: Hot - Frequently accessed or modified data Cool - Infrequently accessed or modified data Archive - Rarely accessed or modified data

    Test your Azure applications with the Azurite open-source emulator. For testing purposes, the xbcloud binary adds the --azure-development-storage option that uses the default access_key and storage account of azurite and testcontainer for the container. You can overwrite these options, if needed.

    "},{"location":"xbcloud-azure.html#usage","title":"Usage","text":"

    All the available options for xbcloud, such as parallel, max-retries, and others, can be used. For more information, see the xbcloud binary overview.

    "},{"location":"xbcloud-azure.html#examples","title":"Examples","text":"

    An example of an xbcloud backup.

    $ xtrabackup --backup --stream=xbstream  | \nxbcloud put backup_name --azure-storage-account=pxbtesting --azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\n

    An example of restoring a backup from xbcloud.

    $ xbcloud get backup_name  --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure --parallel=10 2>download.log | xbstream -x -C restore\n

    An example of deleting a backup from xbcloud.

    $ xbcloud delete backup_name --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\n

    An example of using a shortcut restore.

    $ xbcloud get azure://operator-testing/bak22 ...\n
    "},{"location":"xbcloud-azure.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"xbcloud-binary-fifo-datasink.html","title":"FIFO data sink","text":"

    The feature is in tech preview.

    Percona XtraBackup implements a data sink that uses the first in, first out (FIFO) method. With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to an object storage.

    "},{"location":"xbcloud-binary-fifo-datasink.html#fifo-data-sink-options","title":"FIFO data sink options","text":"

    Percona XtraBackup implements the following options:

    • --fifo-streams=# - specifies the number of FIFO files to use for parallel data stream. To disable FIFO data sink and send stream to STDOUT, set --fifo-streams=1. The default value is 1 (STDOUT) to ensure the backward compatibility. The --fifo-streams value must match on both the XtraBackup and xbcloud sides.
    • --fifo-dir=name - specifies a directory to write Named Pipe.
    • --fifo-timeout=# - specifies the number of seconds to wait for the other end to open the stream for reading. The default value is 60 seconds.
    "},{"location":"xbcloud-binary-fifo-datasink.html#how-to-enable-fifo-data-sink","title":"How to enable FIFO data sink","text":"

    To use FIFO data sink, you can either run two commands in separate terminal sessions or run xtrabackup in the background.

    For example, run the following commands in separate terminal sessions:

    $ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo\n
    $ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\n

    Run xtrabackup in the background with the following commands:

    $ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo &\n$ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\n
    "},{"location":"xbcloud-binary-fifo-datasink.html#stream-to-an-object-storage","title":"Stream to an object storage","text":"

    When taking a backup, you can save the files locally or stream the files to either a different server or an object storage.

    When you stream backups to Amazon S3 compatible storage using LAN with a streaming capacity of 10Gbps, XtraBackup can use multiple FIFO streams to stream the backups faster.

    XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then multiple FIFO files are created to store the data from XtraBackup. Each XtraBackup copy thread writes the data chunks to a specific FIFO file. Xbcloud reads from the FIFO streams and uploads data to an object storage using an async request. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).

    "},{"location":"xbcloud-binary-fifo-datasink.html#performance-improvement","title":"Performance improvement","text":"

    Consider an example of using a FIFO data sink compared to the STDOUT method.

    The database has 1TB of data in multiple tables. The link speed between the source server and destination server using MinIO is ~ 9.2 Gbps.

    Both STDOUT and FIFO data sink scenarios push 1TB of data from two servers.

    For the FIFO data sink we configure 8 parallel streams with --fifo-streams=8 both for XtraBackup and xbcloud.

    The results are the following:

    • The STDOUT method takes 01:25:24 to push 1TB of data using 239 MBps (1.8 Gbps).
    • The FIFO method, using 8 streams, takes 00:16:01 to push 1TB of data using 1.15 GBps (9.2 Gbps).
    "},{"location":"xbcloud-binary-fifo-datasink.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"xbcloud-binary-overview.html","title":"The xbcloud binary overview","text":"

    The purpose of xbcloud is to download from the cloud and upload to the cloud the full or part of an xbstream archive. xbcloud will not overwrite the backup with the same name. xbcloud accepts input via a pipe from xbstream so that it can be invoked as a pipeline with xtrabackup to stream directly to the cloud without needing a local storage.

    Note

    In a Bash shell, the $? parameter returns the exit code from the last binary. If you use pipes, the ${PIPESTATUS[x]} array parameter returns the exit code for each binary in the pipe string.

    $ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ | xbcloud put [options] full_backup\n...\n$ ${PIPESTATUS[x]}\n0 0\n$ true | false\n$ echo $?\n1\n\n# with PIPESTATUS\n$ true | false\n$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}\n0 1\n

    The xbcloud binary stores each chunk as a separate object with a name backup_name/database/table.ibd.NNN..., where NNN... is a 0-padded serial number of chunk within a file. The size of chunk produced by xtrabackup and xbstream changed to 10M.

    To adjust the chunk size use --read-buffer-size. To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.

    xbcloud has three essential operations: put, get, and delete. With these operations, backups are created, stored, retrieved, restored, and deleted. xbcloud operations clearly map to similar operations within the AWS Amazon S3 API.

    The Exponential Backoff feature increases the chances for the completion of a backup or a restore operation. When taking a backup, a chunk upload or download may fail if you have an unstable network connection or other network issues. This feature adds an exponential backoff, a sleep time, and retries the operations.

    With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to object storage.

    Important

    To prevent intermittent backup failures, update the curl utility in Debian 10.

    "},{"location":"xbcloud-binary-overview.html#supported-cloud-storage-types","title":"Supported cloud storage types","text":"

    The following cloud storage types are supported:

    • OpenStack Object Storage (Swift) - see Using the xbcloud binary with Swift

    • Amazon Simple Storage (S3) - see Using the xbcloud binary with Amazon S3

    • Azure Cloud Storage - see Using the xbcloud binary with Microsoft Azure Cloud Storage

    • Google Cloud Storage (gcs) - see Using the xbcloud binary with Google Cloud Storage

    • MinIO - see Using the xbcloud binary with MinIO

    In addition to OpenStack Object Storage (Swift), which has been the only option for storing backups in a cloud storage until Percona XtraBackup 2.4.14, xbcloud supports Amazon S3, MinIO, and Google Cloud Storage. Other Amazon S3-compatible storages, such as Wasabi or Digital Ocean Spaces, are also supported.

    See also

    OpenStack Object Storage(\u201cSwift\u201d)

    Amazon Simple Storage Service

    MinIO

    Google Cloud Storage

    Wasabi

    Digital Ocean Spaces

    "},{"location":"xbcloud-binary-overview.html#usage","title":"Usage","text":"

    The following sample command creates a full backup:

    $ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ --extra-lsndirk=/storage/backups/| xbcloud \\\nput [options] full_backup\n

    An incremental backup only includes the changes since the last backup. The last backup can be either a full or incremental backup.

    The following sample command creates an incremental backup:

    $ xtrabackup --backup --stream=xbstream --incremental-basedir=/storage/backups \\\n--target-dir=/storage/inc-backup | xbcloud  put [options] inc_backup\n

    To prepare an incremental backup, you must first download the full backup with the following command:

    $ xtrabackup get [options] full_backup | xbstream -xv -C /tmp/full-backup\n

    You must prepare the full backup:

    $ xtrabackup --prepare --apply-log-only --target-dir=/tmp/full-backup\n

    After the full backup has been prepared, download the incremental backup:

    xbcloud get [options] inc_backup | xbstream -xv -C /tmp/inc-backup\n

    The downloaded backup is prepared by running the following command:

    $ xtrabackup --prepare --target-dir=/tmp/full-backup --incremental-dir=/tmp/inc-backup\n

    You do not need the full backup to restore only a specific database. You can specify only the tables to be restored:

    xbcloud get [options] ibdata1 sakila/payment.ibd /tmp/partial/partial.xbs\n
    An example of the code: 

    xbstream -xv -C /tmp/partial < /tmp/partial/partial.xbs\n
    "},{"location":"xbcloud-binary-overview.html#supplying-parameters","title":"Supplying parameters","text":"

    Each storage type has mandatory parameters that you can supply on the command line, in a configuration file, and via environment variables.

    "},{"location":"xbcloud-binary-overview.html#configuration-files","title":"Configuration files","text":"

    The parameters the values of which do not change frequently can be stored in my.cnf or in a custom configuration file. The following example is a template of configuration options under the [xbcloud] group:

    [xbcloud]\nstorage=s3\ns3-endpoint=http://localhost:9000/\ns3-access-key=minio\ns3-secret-key=minio123\ns3-bucket=backupsx\ns3-bucket-lookup=path\ns3-api-version=4\n

    Note

    If you explicitly use a parameter on the command line and in a configuration file, xbcloud uses the value provided on the command line.

    "},{"location":"xbcloud-binary-overview.html#environment-variables","title":"Environment variables","text":"

    If you explicitly use a parameter on the command line, in a configuration file, and the corresponding environment variable contains a value, xbcloud uses the value provided on the command line or in the configuration file.

    "},{"location":"xbcloud-binary-overview.html#shortcuts","title":"Shortcuts","text":"

    For all operations (put, get, and delete), you can use a shortcut to specify the storage type, bucket name, and backup name as one parameter instead of using three distinct parameters (\u2013storage, \u2013s3-bucket, and backup name per se).

    Note

    Use the following format: storage-type://bucket-name/backup-name

    In this example s3 refers to a storage type, operator-testing is a bucket name, and bak22 is the backup name. 

    $ xbcloud get s3://operator-testing/bak22 ...\n

    This shortcut expands as follows:

    $ xbcloud get --storage=s3 --s3-bucket=operator-testing bak22 ...\n

    You can supply the mandatory parameters on the command line, configuration files, and in environment variables.

    "},{"location":"xbcloud-binary-overview.html#additional-parameters","title":"Additional parameters","text":"

    xbcloud accepts additional parameters that you can use with any storage type. The --md5 parameter computes the MD5 hash value of the backup chunks. The result is stored in files that following the backup_name.md5 pattern.

    $ xtrabackup --backup --stream=xbstream \\\n--parallel=8 2>backup.log | xbcloud put s3://operator-testing/bak22 \\\n--parallel=8 --md5 2>upload.log\n

    You may use the --header parameter to pass an additional HTTP header with the server side encryption while specifying a customer key.

    An example of using the --header for AES256 encryption.

    $ xtrabackup --backup --stream=xbstream --parallel=4 | \\\nxbcloud put s3://operator-testing/bak-enc/ \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key: CuStoMerKey=\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key-MD5: CuStoMerKeyMd5==\" \\\n--parallel=8\n

    The --header parameter is also useful to set the access control list (ACL) permissions: --header=\"x-amz-acl: bucket-owner-full-control

    "},{"location":"xbcloud-binary-overview.html#incremental-backups","title":"Incremental backups","text":"

    First, you need to make the full backup on which the incremental one is going to be based:

    $ xtrabackup --backup --stream=xbstream --extra-lsndir=/storage/backups/ \\\n--target-dir=/storage/backups/ | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup\n

    Then you can make the incremental backup:

    $ xtrabackup --backup --incremental-basedir=/storage/backups \\\n--stream=xbstream --target-dir=/storage/inc_backup | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup\n
    "},{"location":"xbcloud-binary-overview.html#preparing-incremental-backups","title":"Preparing incremental backups","text":"

    To prepare a backup you first need to download the full backup:

    $ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup | xbstream -xv -C /storage/downloaded_full\n

    Once you download the full backup it should be prepared:

    $ xtrabackup --prepare --apply-log-only --target-dir=/storage/downloaded_full\n

    After the full backup has been prepared you can download the incremental backup:

    $ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup | xbstream -xv -C /storage/downloaded_inc\n

    Once the incremental backup has been downloaded you can prepare it by running:

    $ xtrabackup --prepare --apply-log-only \\\n--target-dir=/storage/downloaded_full \\\n--incremental-dir=/storage/downloaded_inc\n\n$ xtrabackup --prepare --target-dir=/storage/downloaded_full\n
    "},{"location":"xbcloud-binary-overview.html#partial-download-of-the-cloud-backup","title":"Partial download of the cloud backup","text":"

    If you do not want to download the entire backup to restore the specific database you can specify only the tables you want to restore:

    $ xbcloud get --swift-container=test_backup\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ full_backup \\\nibdata1 sakila/payment.ibd \\\n> /storage/partial/partial.xbs\n\n$ xbstream -xv -C /storage/partial < /storage/partial/partial.xbs\n
    "},{"location":"xbcloud-binary-overview.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"xbcloud-exbackoff.html","title":"Exponential backoff","text":"

    Exponential backoff increases the chances for the completion of a backup or a restore operation. For example, a chunk upload or download may fail if you have an unstable network connection or other network issues. This feature adds an exponential backoff, or sleep, time and then retries the upload or download.

    When a chunk upload or download operation fails, xbcloud checks the reason for the failure. This failure can be a CURL error or an HTTP error, or a client-specific error. If the error is listed in the Retriable errors list, xbcloud pauses for a calculated time before retrying the operation until that time reaches the --max-backoff value.

    The operation is retried until the --max-retries value is reached. If the chunk operation fails on the last retry, xbcloud aborts the process.

    The default values are the following:

    • \u2013max-backoff = 300000 (5 minutes)

    • \u2013max-retries = 10

    You can adjust the number of retries by adding the --max-retries parameter and adjust the maximum length of time between retries by adding the --max-backoff parameter to an xbcloud command.

    Since xbcloud does multiple asynchronous requests in parallel, a calculated value, measured in milliseconds, is used for max-backoff. This algorithm calculates how many milliseconds to sleep before the next retry. A number generated is based on the combining the power of two (2), the number of retries already attempted and adds a random number between 1 and 1000. This number avoids network congestion if multiple chunks have the same backoff value. If the default values are used, the final retry attempt to be approximately 17 minutes after the first try. The number is no longer calculated when the milliseconds reach the --max-backoff setting. At that point, the retries continue by using the --max-backoff setting until the max-retries parameter is reached.

    "},{"location":"xbcloud-exbackoff.html#retriable-errors","title":"Retriable errors","text":"

    We retry for the following CURL operations:

    • CURLE_GOT_NOTHING

    • CURLE_OPERATION_TIMEOUT

    • CURLE_RECV_ERROR

    • CURLE_SEND_ERROR

    • CURLE_SEND_FAIL_REWIND

    • CURLE_PARTIAL_FILE

    • CURLE_SSL_CONNECT_ERROR

    We retry for the following HTTP operation status codes:

    • 503

    • 500

    • 504

    • 408

    Each cloud provider may return a different CURL error or an HTTP error, depending on the issue. Add new errors by setting the following variables --curl-retriable-errors or --http-retriable-errors on the command line or in my.cnf or in a custom configuration file under the [xbcloud] section. You can add multiple errors using a comma-separated list of codes.

    The error handling is enhanced when using the --verbose output. This output specifies which error caused xbcloud to fail and what parameter a user must add to retry on this error.

    The following is an example of a verbose output:

    Expected output 
    210701 14:34:23 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210701 14:34:23 /work/pxb/ins/8.1/bin/xbcloud: Curl error (52) Server returned nothing (no headers, no data) is not configured as retriable. You can allow it by adding --curl-retriable-errors=52 parameter\n
    "},{"location":"xbcloud-exbackoff.html#example","title":"Example","text":"

    The following example adjusts the maximum number of retries and the maximum time between retries.

    xbcloud [options] --max-retries=5 --max-backoff=10000\n

    The following list describes the process using --max-backoff=10000:

    • The chunk xtrabackup_logfile.00000000000000000006 fails to upload the first time and sleeps for 2384 milliseconds.

    • The same chunk fails for the second time and the sleep time is increased to 4387 milliseconds.

    • The same chunk fails for the third time and the sleep time is increased to 8691 milliseconds.

    • The same chunk fails for the fourth time. The max-backoff parameter has been reached. All retries sleep the same amount of time after reaching the parameter.

    • The same chunk is successfully uploaded.

    An example of the output for this setting 
    210702 10:07:05 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:05 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 2384 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:23 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:23 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 4387 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:52 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:07:52 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 8691 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:08:47 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:08:47 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 10000 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:10:12 /work/pxb/ins/8.1/bin/xbcloud: successfully uploaded chunk: backup3/xtrabackup_logfile.00000000000000000006, size: 8388660\n
    "},{"location":"xbcloud-exbackoff.html#get-expert-help","title":"Get expert help","text":"

    If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

    Community Forum Get a Percona Expert

    "},{"location":"xbcloud-gcs.html","title":"Use the xbcloud binary with Google Cloud Storage","text":""},{"location":"xbcloud-gcs.html#create-a-full-backup-with-google-cloud-storage","title":"Create a full backup with Google Cloud Storage","text":"

    The support for Google Cloud Storage is implemented using the interoperability mode. This mode was especially designed to interact with cloud services compatible with Amazon S3.

    See also

    Cloud Storage Interoperability

    $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=google \\\n--google-endpoint=`storage.googleapis.com` \\\n--google-access-key='YOUR-ACCESSKEYID' \\\n--google-secret-key='YOUR-SECRETACCESSKEY' \\\n--google-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n

    The following options are available when using Google Cloud Storage:

    • \u2013google-access-key =

    • \u2013google-secret-key =

    • \u2013google-bucket =

    • \u2013google-storage-class=name

      • Note

      The Google storage class name options are the following:

      • STANDARD

      • NEARLINE

      • COLDLINE

      • ARCHIVE

      See also

      Google storage classes - the default Google storage class depends on the storage class of the bucket

      "},{"location":"xbcloud-gcs.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xbcloud-iam-profile.html","title":"Use the xbcloud binary with an IAM instance profile","text":"

      You can use the IAM instance profile when running xbcloud from an EC2 instance.

      An authentication system has two elements:

      • Who am I?
      • What can I do?

      A role defines \u201cwhat can I do.\u201d A role provides a method to define a collection of permissions. Roles are assigned to users, services and EC2 instances, the \u201cwho am I\u201d element.

      The IAM instance profile is the \u201cwho\u201d for an EC2 instance and assumes the IAM role, which has permissions. The instance profile has the same name as the IAM role.

      An IAM instance profile does not need the --s3-secret-key nor the --s3-access-key if they are running xbcloud from an Amazon EC2 instance. To configure or attach an instance metadata to an EC2 instance, see How can I grant my Amazon EC2 instance access to an Amazon S3 bucket.

      An example of the command:

      $ xtrabackup ... | xbcloud put --storage=s3 --s3-bucket=bucket-name backup-name\n

      The xbcloud binary outputs a connect message when successful.

      Expected output 
      221121 13:16:26 Using instance metadata for access and secret key\n221121 13:16:26 xbcloud: Successfully connected.\n

      An important consideration is that the instance metadata has a time to live (TTL) of 6 hours. A backup that takes more than that time contains Expired token errors. Use Exponential Backoff to retry the upload after fetching new keys from the instance metadata.

      Output when keys have expired 
      221121 13:04:52 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:52 xbcloud: Sleeping for 2384 ms before retrying test/mysql.ibd.00000000000000000002 [1]\n221121 13:04:55 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:55 xbcloud: Sleeping for 2887 ms before retrying test/mysql.ibd.00000000000000000003 [1]\n221121 13:04:58 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:58 xbcloud: Sleeping for 2778 ms before retrying test/undo_002.00000000000000000000 [1]\n221121 13:05:00 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:00 xbcloud: Sleeping for 2916 ms before retrying test/undo_002.00000000000000000001 [1]\n221121 13:05:03 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:03 xbcloud: Sleeping for 2794 ms before retrying test/undo_002.00000000000000000002 [1]\n221121 13:05:06 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:06 xbcloud: Sleeping for 2336 ms before retrying test/undo_001.00000000000000000000 [1]\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000002, size: 5242923\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000003, size: 23\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000000, size: 10485802\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000001, size: 6291498\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000002, size: 22\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000000, size: 10485802\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000001, size: 6291498\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000002, size: 22\n. . .\n221121 13:05:18 xbcloud: successfully uploaded chunk: test/xtrabackup_tablespaces.00000000000000000001, size: 36\n221121 13:05:19 xbcloud: Upload completed. \n
      "},{"location":"xbcloud-iam-profile.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xbcloud-minio.html","title":"Use the xbcloud binary with MinIO","text":""},{"location":"xbcloud-minio.html#create-a-full-backup-with-minio","title":"Create a full backup with MinIO","text":"
      $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='play.minio.io:9000' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n
      "},{"location":"xbcloud-minio.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xbcloud-s3.html","title":"Use the xbcloud binary with Amazon S3","text":""},{"location":"xbcloud-s3.html#create-a-full-backup-with-amazon-s3","title":"Create a full backup with Amazon S3","text":"
      $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='s3.amazonaws.com' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n

      The following options are available when using Amazon S3:

      Option Details \u2013s3-access-key Use to supply the AWS access key ID \u2013s3-secret-key Use to supply the AWS secret access key \u2013s3-bucket Use supply the AWS bucket name \u2013s3-region Use to specify the AWS region. The default value is us-east-1 \u2013s3-api-version = Select the signing algorithm. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-bucket-lookup = Specify whether to use bucket.endpoint.com or endpoint.com/bucket*style requests. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-storage-class= Specify the S3 storage class. The default storage class depends on the provider. The name options are the following:
      • STANDARD
      • STANDARD_IA
      • GLACIER
      NOTE If you use the GLACIER storage class, the object must be restored to S3 before restoring the backup. Also supports using custom S3 implementations such as MinIO or CephRadosGW."},{"location":"xbcloud-s3.html#permissions-setup","title":"Permissions setup","text":"

      Following the principle of \u201cleast-privilege\u201d, these are the minimum bucket permissions needed for xbcloud to write backups to S3: LIST/PUT/GET/DELETE.

      The following example shows the policy definition for writing to the xbcloud-testing bucket on the AWS S3 storage.

      {\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"s3:ListBucket\"\n            ],\n            \"Resource\": \"arn:aws:s3:::xbcloud-testing\"\n        },\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"s3:PutObject\",\n                \"s3:PutObjectAcl\",\n                \"s3:GetObject\",\n                \"s3:GetObjectAcl\",\n                \"s3:DeleteObject\"\n            ],\n            \"Resource\": \"arn:aws:s3:::xbcloud-testing/*\"\n        }\n    ]\n}\n
      "},{"location":"xbcloud-s3.html#environment-variables","title":"Environment variables","text":"

      The following environment variables are recognized. xbcloud maps them automatically to corresponding parameters applicable to the selected storage.

      • AWS_ACCESS_KEY_ID (or ACCESS_KEY_ID)

      • AWS_SECRET_ACCESS_KEY (or SECRET_ACCESS_KEY)

      • AWS_DEFAULT_REGION (or DEFAULT_REGION)

      • AWS_ENDPOINT (or ENDPOINT)

      • AWS_CA_BUNDLE

      "},{"location":"xbcloud-s3.html#restore-with-s3","title":"Restore with S3","text":"
      $ xbcloud get s3://operator-testing/bak22 \\\n--s3-endpoint=https://storage.googleapis.com/ \\\n--parallel=10 2>download.log | xbstream -x -C restore --parallel=8\n
      "},{"location":"xbcloud-s3.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xbcloud-swift.html","title":"Use the xbcloud binary with Swift","text":""},{"location":"xbcloud-swift.html#create-a-full-backup-with-swift","title":"Create a full backup with Swift","text":"

      The following example shows how to make a full backup and upload it to Swift.

      $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\n--parallel=10 \\\nfull_backup\n

      The following OpenStack environment variables are also recognized and mapped automatically to the corresponding swift parameters (--storage=swift):

      • OS_AUTH_URL

      • OS_TENANT_NAME

      • OS_TENANT_ID

      • OS_USERNAME

      • OS_PASSWORD

      • OS_USER_DOMAIN

      • OS_USER_DOMAIN_ID

      • OS_PROJECT_DOMAIN

      • OS_PROJECT_DOMAIN_ID

      • OS_REGION_NAME

      • OS_STORAGE_URL

      • OS_CACERT

      "},{"location":"xbcloud-swift.html#restore-with-swift","title":"Restore with Swift","text":"
      $ xbcloud get [options] <name> [<list-of-files>] | xbstream -x\n

      The following example shows how to fetch and restore the backup from Swift:

      $ xbcloud get --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\nfull_backup | xbstream -xv -C /tmp/downloaded_full\n\n$ xbcloud delete --storage=swift --swift-user=xtrabackup \\\n--swift-password=xtrabackup123! --swift-auth-version=3 \\\n--swift-auth-url=http://openstack.ci.percona.com:5000/ \\\n--swift-container=mybackup1 --swift-domain=Default\n
      "},{"location":"xbcloud-swift.html#command-line-options","title":"Command-line options","text":"

      xbcloud has the following command line options:

      "},{"location":"xbcloud-swift.html#-storageswifts3google","title":"\u2013storage(=[swift|s3|google])","text":"

      Cloud storage option. xbcloud supports Swift, MinIO, and AWS S3. The default value is swift.

      "},{"location":"xbcloud-swift.html#-swift-auth-url","title":"\u2013swift-auth-url()","text":"

      The URL of the Swift cluster

      "},{"location":"xbcloud-swift.html#-swift-storage-url","title":"\u2013swift-storage-url()","text":"

      The xbcloud tries to get the object-store URL for a given region (if any are specified) from the keystone response. You can override that URL by passing \u2013swift-storage-url=URL argument.

      "},{"location":"xbcloud-swift.html#-swift-user","title":"\u2013swift-user()","text":"

      The Swift username (X-Auth-User, specific to Swift)

      "},{"location":"xbcloud-swift.html#-swift-key","title":"\u2013swift-key()","text":"

      The Swift key/password (X-Auth-Key, specific to Swift)

      "},{"location":"xbcloud-swift.html#-swift-container","title":"\u2013swift-container()","text":"

      The container to back up into (specific to Swift)

      "},{"location":"xbcloud-swift.html#-paralleln","title":"\u2013parallel(=N)","text":"

      The maximum number of concurrent upload/download requests. The default value is 1.

      "},{"location":"xbcloud-swift.html#-cacert","title":"\u2013cacert()","text":"

      The path to the file with CA certificates

      "},{"location":"xbcloud-swift.html#-insecure","title":"\u2013insecure()","text":"

      Do not verify server\u2019s certificate

      "},{"location":"xbcloud-swift.html#swift-authentication-options","title":"Swift authentication options","text":"

      The Swift specification describes several authentication options. The xbcloud tool can authenticate against keystone with API version 2 and 3.

      "},{"location":"xbcloud-swift.html#-swift-auth-version","title":"\u2013swift-auth-version()","text":"

      Specifies the swift authentication version. The possible values are: 1.0 - TempAuth, 2.0 - Keystone v2.0, and 3 - Keystone v3. The default value is 1.0.

      For v2 additional options are:

      "},{"location":"xbcloud-swift.html#-swift-tenant","title":"\u2013swift-tenant()","text":"

      Swift tenant name

      "},{"location":"xbcloud-swift.html#-swift-tenant-id","title":"\u2013swift-tenant-id()","text":"

      Swift tenant ID

      "},{"location":"xbcloud-swift.html#-swift-region","title":"\u2013swift-region()","text":"

      Swift endpoint region

      "},{"location":"xbcloud-swift.html#-swift-password","title":"\u2013swift-password()","text":"

      Swift password for the user

      For v3 additional options are:

      "},{"location":"xbcloud-swift.html#-swift-user-id","title":"\u2013swift-user-id()","text":"

      Swift user ID

      "},{"location":"xbcloud-swift.html#-swift-project","title":"\u2013swift-project()","text":"

      Swift project name

      "},{"location":"xbcloud-swift.html#-swift-project-id","title":"\u2013swift-project-id()","text":"

      Swift project ID

      "},{"location":"xbcloud-swift.html#-swift-domain","title":"\u2013swift-domain()","text":"

      Swift domain name

      "},{"location":"xbcloud-swift.html#-swift-domain-id","title":"\u2013swift-domain-id()","text":"

      Swift domain ID

      "},{"location":"xbcloud-swift.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xbcrypt-binary-overview.html","title":"The xbcrypt binary overview","text":"

      To support encryption and decryption of the backups, a new tool xbcrypt was introduced to Percona XtraBackup.

      The XBCRYPT_ENCRYPTION_KEY environment variable is only used in place of the --encrypt_key=name option. You can use the environment variable or command line option. If you use both, the command line option takes precedence over the value specified in the environment variable.

      This utility has been modeled after The xbstream binary to perform encryption and decryption outside of Percona XtraBackup. xbcrypt has following command line options:

      "},{"location":"xbcrypt-binary-overview.html#-d-decrypt","title":"-d(, \u2013decrypt()","text":"

      Decrypt data input to output.

      "},{"location":"xbcrypt-binary-overview.html#-i-inputname","title":"-i(, \u2013input(=name)","text":"

      Optional input file. If not specified, input will be read from standard input.

      "},{"location":"xbcrypt-binary-overview.html#-o-outputname","title":"-o(, \u2013output(=name)","text":"

      Optional output file. If not specified, output will be written to standard output.

      "},{"location":"xbcrypt-binary-overview.html#-a-encrypt-algoname","title":"-a(, \u2013encrypt-algo(=name)","text":"

      Encryption algorithm.

      "},{"location":"xbcrypt-binary-overview.html#-k-encrypt-keyname","title":"-k(, \u2013encrypt-key(=name)","text":"

      Encryption key.

      "},{"location":"xbcrypt-binary-overview.html#-f-encrypt-key-filename","title":"-f(, \u2013encrypt-key-file(=name)","text":"

      File which contains encryption key.

      "},{"location":"xbcrypt-binary-overview.html#-s-encrypt-chunk-size","title":"-s(, \u2013encrypt-chunk-size(=#)","text":"

      Size of working buffer for encryption in bytes. The default value is 64K.

      "},{"location":"xbcrypt-binary-overview.html#-encrypt-threads","title":"\u2013encrypt-threads(=#)","text":"

      This option specifies the number of worker threads that will be used for parallel encryption/decryption.

      "},{"location":"xbcrypt-binary-overview.html#-v-verbose","title":"-v(, \u2013verbose()","text":"

      Display verbose status output.

      "},{"location":"xbcrypt-binary-overview.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xbstream-binary-overview.html","title":"The xbstream binary overview","text":"

      To support simultaneous compression and streaming, a new custom streaming format called xbstream was introduced to Percona XtraBackup in addition to the TAR format. That was required to overcome some limitations of traditional archive formats such as tar, cpio and others which did not allow streaming dynamically generated files, for example dynamically compressed files. Other advantages of xbstream over traditional streaming/archive format include ability to stream multiple files concurrently (so it is possible to use streaming in the xbstream format together with the \u2013parallel option) and more compact data storage.

      This utility has a tar-like interface:

      • with the -x option it extracts files from the stream read from its standard input to the current directory unless specified otherwise with the -c option. The parallel extraction is supported with the --parallel option.

      • with the -c option it streams files specified on the command line to its standard output.

      • with the --decrypt=ALGO option specified xbstream will automatically decrypt encrypted files when extracting input stream. Supported values for this option are: AES128, AES192, and AES256. Either --encrypt-key or --encrypt-key-file options must be specified to provide encryption key, but not both.

      • with the --encrypt-threads option you can specify the number of threads for parallel data encryption. The default value is 1.

      • the --encrypt-key option is used to specify the encryption key that will be used. It can\u2019t be used with --encrypt-key-file option because they are mutually exclusive.

      • the --encrypt-key-file option is used to specify the file that contains the encryption key. It can\u2019t be used with --encrypt-key option. because they are mutually exclusive.

      The utility also tries to minimize its impact on the OS page cache by using the appropriate posix_fadvise() calls when available.

      When compression is enabled with xtrabackup all data is being compressed, including the transaction log file and meta data files, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:

      "},{"location":"xbstream-binary-overview.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

      The Zstandard (ZSTD) is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. ZSTD is the default compression algorithm for the --compress option.

      To compress files using the ZSTD compression algorithm, use the --compress option:

      $ xtrabackup --backup --compress --target-dir=/data/backup\n

      The resulting files have the \\*.zst format.

      You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.

      $ xtrabackup \u2013backup \u2013compress \u2013compress-zstd-level=1 \u2013target-dir=/data/backup\n
      "},{"location":"xbstream-binary-overview.html#lz4","title":"lz4","text":"

      To compress files using the lz4 compression algorithm, set the --compress option to lz4:

      $ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\n

      The resulting files have the \\*.lz4 format.

      To decompress files, use the --decompress option.

      "},{"location":"xbstream-binary-overview.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xtrabackup-binary-overview.html","title":"The xtrabackup binary overview","text":"

      The xtrabackup binary is a compiled C program that is linked with the InnoDB libraries and the standard MySQL client libraries.

      xtrabackup enables point-in-time backups of InnoDB / XtraDB tables together with the schema definitions, MyISAM tables, and other portions of the server.

      The InnoDB libraries provide the functionality to apply a log to data files. The MySQL client libraries are used to parse command-line options and configuration file.

      The tool runs in either --backup or --prepare mode, corresponding to the two main functions it performs. There are several variations on these functions to accomplish different tasks, and there is one less commonly used mode, --print-param.

      "},{"location":"xtrabackup-binary-overview.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xtrabackup-exit-codes.html","title":"xtrabackup exit codes","text":"

      The xtrabackup binary exits with the traditional success value of 0 after a backup when no error occurs. If an error occurs during the backup, the exit value is 1.

      In certain cases, the exit value can be something other than 0 or 1, due to the command-line option code included from the MySQL libraries. An unknown command-line option, for example, will cause an exit code of 255.

      "},{"location":"xtrabackup-exit-codes.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xtrabackup-files.html","title":"Index of files created by Percona XtraBackup","text":"
      • Information related to the backup and the server
      File Name Description backup-my.cnf This file contains information to start the mini instance of InnoDB during the --prepare. This **not** a backup of the original my.cnf. The InnoDB configuration is read from the file backup-my.cnf created by xtrabackup when the backup was made. The --prepare uses InnoDB configuration from backup-my.cnf by default, or from --defaults-file, if specified. The InnoDB's configuration in this context means server variables that affect dataformat, i.e., innodb_page_size option, innodb_log_block_size, etc. Location-related variables, like innodb_log_group_home_dir or innodb_data_file_path are always ignored by --prepare, so preparing a backup always works with data files from the back directory, rather than any external ones. xtrabackup_checkpoints

      The type of the backup (for example, full or incremental), its state (for example, prepared) and the LSN range contained in it. This information is used for incremental backups. Example of the xtrabackup_checkpoints after taking a full backup:

      backup_type = full-backuped\nfrom lsn= 0\nto_lsn = 15188961605\nlast_lsn = 15188961605\n
      Example of the xtrabackup_checkpoints after taking an incremental backup: 
      backup_type = incremental\nfrom_lsn = 15188961605\nto_lsn = 15189350111\nlast_lsn = 15189350111\n
      xtrabackup_binlog_info

      The binary log file used by the server and its position at the moment of the backup. A result of the following query:

      SELECT server_uuid, local, replication, storage_engines FROM performance_schema.log_status;\n
      xtrabackup_binlog The xtrabackup binary used in the process. xtrabackup_logfile Contains data needed for running the: --prepare. The bigger this file is the --prepare process will take longer to finish. <table_name>.delta.meta

      This file is going to be created when performing the incremental backup. It contains the per-table delta metadata: page size, size of compressed page (if the value is 0 it means the tablespace isn\u2019t compressed) and space id. Example of this file:

      page_size = 16384\nzip_size = 0\nspace_id = 0\n

      • Information related to the replication environment (if using the--slave-info option):

        xtrabackup_slave_info

        The CHANGE MASTER statement needed for setting up a replica.

      • Information related to the Galera and Percona XtraDB Cluster (if using the --galera-info option):

        xtrabackup_galera_info

        Contains the values of wsrep_local_state_uuid andwsrep_last_committed status variables

      "},{"location":"xtrabackup-files.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xtrabackup-implementation-details.html","title":"xtrabackup implementation details","text":"

      This page contains notes on various internal aspects of the xtrabackup tool\u2019s operation.

      "},{"location":"xtrabackup-implementation-details.html#file-permissions","title":"File permissions","text":"

      xtrabackup opens the source data files in read-write mode, although it does not modify the files. This means that you must run xtrabackup as a user who has permission to write the data files. The reason for opening the files in read-write mode is that xtrabackup uses the embedded InnoDB libraries to open and read the files, and InnoDB opens them in read-write mode because it normally assumes it is going to write to them.

      "},{"location":"xtrabackup-implementation-details.html#tune-the-os-buffers","title":"Tune the OS buffers","text":"

      Because xtrabackup reads large amounts of data from the filesystem, it uses posix_fadvise() where possible, to instruct the operating system not to try to cache the blocks it reads from disk. Without this hint, the operating system would prefer to cache the blocks, assuming that xtrabackup is likely to need them again, which is not the case. Caching such large files can place pressure on the operating system\u2019s virtual memory and cause other processes, such as the database server, to be swapped out. The xtrabackup tool avoids this with the following hint on both the source and destination files:

      posix_fadvise(file, 0, 0, POSIX_FADV_DONTNEED)\n

      In addition, xtrabackup asks the operating system to perform more aggressive read-ahead optimizations on the source files:

      posix_fadvise(file, 0, 0, POSIX_FADV_SEQUENTIAL)\n
      "},{"location":"xtrabackup-implementation-details.html#copy-data-files","title":"Copy data files","text":"

      When copying the data files to the target directory, xtrabackup reads and writes 1 MB of data at a time. This is not configurable. When copying the log file, xtrabackup reads and writes 512 bytes at a time. This is also not possible to configure, and matches InnoDB\u2019s behavior (workaround exists in Percona Server for MySQL because it has an option to tune innodb_log_block_size for XtraDB, and in that case Percona XtraBackup will match the tuning).

      After reading from the files, xtrabackup iterates over the 1MB buffer a page at a time, and checks for page corruption on each page with InnoDB\u2019s buf_page_is_corrupted() function. If the page is corrupt, it re-reads and retries up to 10 times for each page. It skips this check on the doublewrite buffer.

      "},{"location":"xtrabackup-implementation-details.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xtrabackup-option-reference.html","title":"The xtrabackup option reference","text":"

      Here you can find all of the command-line options for the xtrabackup binary.

      "},{"location":"xtrabackup-option-reference.html#modes-of-operation","title":"Modes of operation","text":"

      You invoke xtrabackup in one of the following modes:

      • --backup mode to make a backup in a target directory

      • --prepare mode to restore data from a backup (created in --backup mode)

      • --copy-back to copy data from a backup to the location that contained the original data; to move data instead of copying use the alternate --move-back mode.

      When you intend to run xtrabackup in any of these modes, use the following syntax:

      $ xtrabackup [--defaults-file=#] --backup|--prepare|--copy-back| [OPTIONS]\n

      For example, the --prepare mode is applied as follows:

      $ xtrabackup --prepare --target-dir=/data/backup/mysql/\n

      For all modes, the default options are read from the xtrabackup and mysqld configuration groups from the following files in the given order:

      1. /etc/my.cnf

      2. /etc/mysql/my.cnf

      3. /usr/etc/my.cnf

      4. ~/.my.cnf.

      As the first parameter to xtrabackup (in place of the --defaults-file, you may supply one of the following:

      • --print-defaults to have xtrabackup print the argument list and exit.

      • --no-defaults to forbid reading options from any file but the login file.

      • --defaults-file to read the default options from the given file.

      • --defaults-extra-file to read the specified additional file after the global files have been read.

      • --defaults-group-suffix to read the configuration groups with the given suffix. The effective group name is constructed by concatenating the default configuration groups (xtrabackup and mysqld) with the given suffix.

      • --login-path to read the given path from the login file.

      "},{"location":"xtrabackup-option-reference.html#innodb-options","title":"InnoDB options","text":"

      There is a large group of InnoDB options that are normally read from the my.cnf configuration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify them explicitly. These options have the same behavior in InnoDB and XtraDB. See --innodb-miscellaneous for more information.

      "},{"location":"xtrabackup-option-reference.html#options","title":"Options","text":""},{"location":"xtrabackup-option-reference.html#-apply-log-only","title":"\u2013apply-log-only()","text":"

      This option causes only the redo stage to be performed when preparing a backup. It is very important for incremental backups.

      "},{"location":"xtrabackup-option-reference.html#-backup","title":"\u2013backup()","text":"

      Make a backup and place it in --target-dir. See Creating a backup.

      "},{"location":"xtrabackup-option-reference.html#-backup-lock-timeout","title":"\u2013backup-lock-timeout()","text":"

      The timeout in seconds for attempts to acquire metadata locks.

      "},{"location":"xtrabackup-option-reference.html#-backup-lock-retry-count","title":"\u2013backup-lock-retry-count()","text":"

      The number of attempts to acquire metadata locks.

      "},{"location":"xtrabackup-option-reference.html#-backup-locks","title":"\u2013backup-locks()","text":"

      This option controls if backup locks should be used instead of FLUSH TABLES WITH READ LOCK on the backup stage. The option has no effect when backup locks are not supported by the server. This option is enabled by default, disable with --no-backup-locks.

      "},{"location":"xtrabackup-option-reference.html#-check-privileges","title":"\u2013check-privileges()","text":"

      This option checks if Percona XtraBackup has all required privileges. If a missing privilege is required for the current operation, it will terminate and print out an error message. If a missing privilege is not required for the current operation, but may be necessary for some other XtraBackup operation, the process is not aborted and a warning is printed.

      xtrabackup: Error: missing required privilege LOCK TABLES on *.*\nxtrabackup: Warning: missing required privilege REPLICATION CLIENT on *.*\n
      "},{"location":"xtrabackup-option-reference.html#-close-files","title":"\u2013close-files()","text":"

      Do not keep files opened. When xtrabackup opens tablespace it normally doesn\u2019t close its file handle in order to handle the DDL operations correctly. However, if the number of tablespaces is really huge and can not fit into any limit, there is an option to close file handles once they are no longer accessed. Percona XtraBackup can produce inconsistent backups with this option enabled. Use at your own risk.

      "},{"location":"xtrabackup-option-reference.html#-compress","title":"\u2013compress()","text":"

      This option tells xtrabackup to compress all output data, including the transaction log file and meta data files, using either the ZSTD or lz4 compression algorithm. ZSTD is the default compression algorithm.

      --compress produces \\*.zst files. You can extract the contents of these files by using the --decompress option. You can specify ZSTD compression level with the --compress-zstd-level(=#) option.

      --compress=lz4 produces \\*.lz4 files. You can extract the contents of these files by using lz4 program.

      "},{"location":"xtrabackup-option-reference.html#-compress-chunk-size","title":"\u2013compress-chunk-size(=#)","text":"

      Size of working buffer(s) for compression threads in bytes. The default value is 64K.

      "},{"location":"xtrabackup-option-reference.html#-compress-threads","title":"\u2013compress-threads(=#)","text":"

      This option specifies the number of worker threads used by xtrabackup for parallel data compression. This option defaults to 1. Parallel compression (--compress-threads) can be used together with parallel file copying (--parallel). For example, --parallel=4 --compress --compress-threads=2 will create 4 I/O threads that will read the data and pipe it to 2 compression threads.

      "},{"location":"xtrabackup-option-reference.html#-compress-zstd-level","title":"\u2013compress-zstd-level(=#)","text":"

      This option specifies ZSTD compression level. Compression levels provide a trade-off between the speed of compression and the size of the compressed files. A lower compression level provides faster compression speed but larger file sizes. A higher compression level provides lower compression speed but smaller file sizes. For example, set level 1 if the compression speed is the most important for you. Set level 19 if the size of the compressed files is the most important.

      The default value is 1. Allowed range of values is from 1 to 19.

      "},{"location":"xtrabackup-option-reference.html#-copy-back","title":"\u2013copy-back()","text":"

      Copy all the files in a previously made backup from the backup directory to their original locations. This option will not copy over existing files unless --force-non-empty-directories option is specified.

      "},{"location":"xtrabackup-option-reference.html#-core-file","title":"\u2013core-file()","text":"

      Write core on fatal signals.

      "},{"location":"xtrabackup-option-reference.html#-databases","title":"\u2013databases(=#)","text":"

      This option specifies a list of databases and tables that should be backed up. The option accepts the list of the form \"databasename1[.table_name1] databasename2[.table_name2] . . .\".

      "},{"location":"xtrabackup-option-reference.html#-databases-excludename","title":"\u2013databases-exclude(=name)","text":"

      Excluding databases based on name, Operates the same way as --databases, but matched names are excluded from backup. Note that this option has a higher priority than --databases.

      "},{"location":"xtrabackup-option-reference.html#-databases-file","title":"\u2013databases-file(=#)","text":"

      This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form databasename1[.table_name1], one element per line.

      "},{"location":"xtrabackup-option-reference.html#-datadirdirectory","title":"\u2013datadir(=DIRECTORY)","text":"

      The source directory for the backup. This should be the same as the datadir for your MySQL server, so it should be read from my.cnf if that exists; otherwise you must specify it on the command line.

      When combined with the --copy-back or --move-back option, --datadir refers to the destination directory.

      Once connected to the server, in order to perform a backup you will need READ and EXECUTE permissions at a filesystem level in the server\u2019s datadir.

      "},{"location":"xtrabackup-option-reference.html#-debug-sleep-before-unlock","title":"\u2013debug-sleep-before-unlock(=#)","text":"

      This is a debug-only option used by the xtrabackup test suite.

      "},{"location":"xtrabackup-option-reference.html#-debug-syncname","title":"\u2013debug-sync(=name)","text":"

      The debug sync point. This option is only used by the xtrabackup test suite.

      "},{"location":"xtrabackup-option-reference.html#-decompress","title":"\u2013decompress()","text":"

      Decompresses all files in a backup previously made with the --compress option. The --parallel option will allow multiple files to be decrypted simultaneously. Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory users should use --remove-original option.

      "},{"location":"xtrabackup-option-reference.html#-decompress-threads","title":"\u2013decompress-threads(=#)","text":"

      Force xbstream to use the specified number of threads for decompressing.

      "},{"location":"xtrabackup-option-reference.html#-decryptencryption-algorithm","title":"\u2013decrypt(=ENCRYPTION-ALGORITHM)","text":"

      Decrypts all files with the .xbcrypt extension in a backup previously made with --encrypt option. The --parallel option will allow multiple files to be decrypted simultaneously. Percona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should use --remove-original option.

      "},{"location":"xtrabackup-option-reference.html#-defaults-extra-filemycnf","title":"\u2013defaults-extra-file(=[MY.CNF])","text":"

      Read this file after the global files are read. Must be given as the first option on the command-line.

      "},{"location":"xtrabackup-option-reference.html#-defaults-filemycnf","title":"\u2013defaults-file(=[MY.CNF])","text":"

      Only read default options from the given file. Must be given as the first option on the command-line. Must be a real file; it cannot be a symbolic link.

      "},{"location":"xtrabackup-option-reference.html#-defaults-groupgroup-name","title":"\u2013defaults-group(=GROUP-NAME)","text":"

      This option is to set the group which should be read from the configuration file. This is used by xtrabackup if you use the --defaults-group option. It is needed for mysqld_multi deployments.

      "},{"location":"xtrabackup-option-reference.html#-defaults-group-suffix","title":"\u2013defaults-group-suffix(=#)","text":"

      Also reads groups with concat(group, suffix).

      "},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool","title":"\u2013dump-innodb-buffer-pool()","text":"

      This option controls whether or not a new dump of buffer pool content should be done.

      With --dump-innodb-buffer-pool, xtrabackup makes a request to the server to start the buffer pool dump (it takes some time to complete and is done in background) at the beginning of a backup provided the status variable innodb_buffer_pool_dump_status reports that the dump has been completed.

      $ xtrabackup --backup --dump-innodb-buffer-pool --target-dir=/home/user/backup\n

      By default, this option is set to OFF.

      If innodb_buffer_pool_dump_status reports that there is running dump of buffer pool, xtrabackup waits for the dump to complete using the value of --dump-innodb-buffer-pool-timeout

      The file ib_buffer_pool stores tablespace ID and page ID data used to warm up the buffer pool sooner.

      "},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool-timeout","title":"\u2013dump-innodb-buffer-pool-timeout()","text":"

      This option contains the number of seconds that xtrabackup should monitor the value of innodb_buffer_pool_dump_status to determine if buffer pool dump has completed.

      This option is used in combination with --dump-innodb-buffer-pool. By default, it is set to 10 seconds.

      "},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool-pct","title":"\u2013dump-innodb-buffer-pool-pct()","text":"

      This option contains the percentage of the most recently used buffer pool pages to dump.

      This option is effective if --dump-innodb-buffer-pool option is set to ON. If this option contains a value, xtrabackup sets the MySQL system variable innodb_buffer_pool_dump_pct. As soon as the buffer pool dump completes or it is stopped (see --dump-innodb-buffer-pool-timeout), the value of the MySQL system variable is restored.

      "},{"location":"xtrabackup-option-reference.html#-encryptencryption_algorithm","title":"\u2013encrypt(=ENCRYPTION_ALGORITHM)","text":"

      This option instructs xtrabackup to encrypt backup copies of InnoDB data files using the algorithm specified in the ENCRYPTION_ALGORITHM. Currently supported algorithms are: AES128, AES192 and AES256

      "},{"location":"xtrabackup-option-reference.html#-encrypt-keyencryption_key","title":"\u2013encrypt-key(=ENCRYPTION_KEY)","text":"

      A proper length encryption key to use. It is not recommended to use this option where there is uncontrolled access to the machine as the command line and thus the key can be viewed as part of the process info.

      "},{"location":"xtrabackup-option-reference.html#-encrypt-key-fileencryption_key_file","title":"\u2013encrypt-key-file(=ENCRYPTION_KEY_FILE)","text":"

      The name of a file where the raw key of the appropriate length can be read from. The file must be a simple binary (or text) file that contains exactly the key to be used.

      It is passed directly to the xtrabackup child process. See the xtrabackup documentation for more details.

      "},{"location":"xtrabackup-option-reference.html#-encrypt-threads","title":"\u2013encrypt-threads(=#)","text":"

      This option specifies the number of worker threads that will be used for parallel encryption/decryption. See the xtrabackup documentation for more details.

      "},{"location":"xtrabackup-option-reference.html#-encrypt-chunk-size","title":"\u2013encrypt-chunk-size(=#)","text":"

      This option specifies the size of the internal working buffer for each encryption thread, measured in bytes. It is passed directly to the xtrabackup child process.

      To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.

      "},{"location":"xtrabackup-option-reference.html#-estimate-memory","title":"\u2013estimate-memory(=#)","text":"

      This option is in tech preview. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

      Implemented in Percona XtraBackup 8.0.32-26, the option lets you enable or disable the Smart memory estimation feature. The default value is OFF. Enable the feature by setting --estimate-memory=ON in the backup phase and setting the --use-free-memory-pct option in the --prepare phase. If the --estimate-memory setting is disabled, the --use-free-memory-pct setting is ignored.

      An example of how to enable the Smart memory estimation feature:

      $ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n
      $ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\n
      "},{"location":"xtrabackup-option-reference.html#-export","title":"\u2013export()","text":"

      Create files necessary for exporting tables. See Restoring Individual Tables.

      "},{"location":"xtrabackup-option-reference.html#-extra-lsndirdirectory","title":"\u2013extra-lsndir(=DIRECTORY)","text":"

      (for \u2013backup): save an extra copy of the xtrabackup_checkpoints and xtrabackup_info files in this directory.

      "},{"location":"xtrabackup-option-reference.html#-force-non-empty-directories","title":"\u2013force-non-empty-directories()","text":"

      When specified, it makes --copy-back and --move-back option transfer files to non-empty directories. No existing files will be overwritten. If files that need to be copied/moved from the backup directory already exist in the destination directory, it will still fail with an error.

      "},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-timeoutseconds","title":"\u2013ftwrl-wait-timeout(=SECONDS)","text":"

      This option specifies time in seconds that xtrabackup should wait for queries that would block FLUSH TABLES WITH READ LOCK before running it. If there are still such queries when the timeout expires, xtrabackup terminates with an error. Default is 0, in which case it does not wait for queries to complete and starts FLUSH TABLES WITH READ LOCK immediately. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

      "},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-thresholdseconds","title":"\u2013ftwrl-wait-threshold(=SECONDS)","text":"

      This option specifies the query run time threshold which is used by xtrabackup to detect long-running queries with a non-zero value of --ftwrl-wait-timeout. FLUSH TABLES WITH READ LOCK is not started until such long-running queries exist. This option has no effect if --ftwrl-wait-timeout is 0. Default value is 60 seconds. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

      "},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-query-typeallupdate","title":"\u2013ftwrl-wait-query-type(=all|update)","text":"

      This option specifies which types of queries are allowed to complete before xtrabackup will issue the global lock. Default is all.

      "},{"location":"xtrabackup-option-reference.html#-galera-info","title":"\u2013galera-info()","text":"

      This option creates the xtrabackup_galera_info file which contains the local node state at the time of the backup. Option should be used when performing the backup of Percona XtraDB Cluster. It has no effect when backup locks are used to create the backup.

      "},{"location":"xtrabackup-option-reference.html#-generate-new-master-key","title":"\u2013generate-new-master-key()","text":"

      Generate a new master key when doing a copy-back.

      "},{"location":"xtrabackup-option-reference.html#-generate-transition-key","title":"\u2013generate-transition-key()","text":"

      xtrabackup needs to access the same keyring file or vault server during prepare and copy-back but it should not depend on whether the server keys have been purged.

      --generate-transition-key creates and adds to the keyring a transition key for xtrabackup to use if the master key used for encryption is not found because it has been rotated and purged.

      "},{"location":"xtrabackup-option-reference.html#-get-server-public-key","title":"\u2013get-server-public-key()","text":"

      Get the server public key

      "},{"location":"xtrabackup-option-reference.html#-help","title":"\u2013help()","text":"

      When run with this option or without any options xtrabackup displays information about how to run the program on the command line along with all supported options and variables with default values where appropriate.

      "},{"location":"xtrabackup-option-reference.html#-historyname","title":"\u2013history(=NAME)","text":"

      This option enables the tracking of backup history in the PERCONA_SCHEMA.xtrabackup_history table. An optional history series name may be specified that will be placed with the history record for the current backup being taken.

      "},{"location":"xtrabackup-option-reference.html#-hosthost","title":"\u2013host(=HOST)","text":"

      This option accepts a string argument that specifies the host to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql \u2013help for details.

      "},{"location":"xtrabackup-option-reference.html#-incremental-basedirdirectory","title":"\u2013incremental-basedir(=DIRECTORY)","text":"

      When creating an incremental backup, this is the directory containing the full backup that is the base dataset for the incremental backups.

      "},{"location":"xtrabackup-option-reference.html#-incremental-dirdirectory","title":"\u2013incremental-dir(=DIRECTORY)","text":"

      When preparing an incremental backup, this is the directory where the incremental backup is combined with the full backup to make a new full backup.

      "},{"location":"xtrabackup-option-reference.html#-incremental-force-scan","title":"\u2013incremental-force-scan()","text":"

      When creating an incremental backup, force a full scan of the data pages in that instance.

      "},{"location":"xtrabackup-option-reference.html#-incremental-history-namename","title":"\u2013incremental-history-name(=name)","text":"

      This option specifies the name of the backup series stored in the PERCONA_SCHEMA.xtrabackup_history history record to base an incremental backup on. xtrabackup will search the history table looking for the most recent (highest innodb_to_lsn), successful backup in the series and take the to_lsn value to use as the starting lsn for the incremental backup. This will be mutually exclusive with --incremental-history-uuid, --incremental-basedir and --incremental-lsn. If no valid lsn can be found (no series by that name, no successful backups by that name) xtrabackup will return with an error. It is used with the --incremental option.

      "},{"location":"xtrabackup-option-reference.html#-incremental-history-uuidname","title":"\u2013incremental-history-uuid(=name)","text":"

      This option specifies the UUID of the specific history record stored in the PERCONA_SCHEMA.xtrabackup_history to base an incremental backup on. --incremental-history-name, --incremental-basedir and --incremental-lsn. If no valid lsn can be found (no success record with that UUID) xtrabackup will return with an error. It is used with the \u2013incremental option.

      "},{"location":"xtrabackup-option-reference.html#-incremental-lsnlsn","title":"\u2013incremental-lsn(=LSN)","text":"

      When creating an incremental backup, you can specify the log sequence number (LSN) instead of specifying --incremental-basedir. For databases created in 5.1 and later, specify the LSN as a single 64-bit integer.

      Important

      If a wrong LSN value is specified (a user error which Percona XtraBackup does not detect), the backup is unusable. Be careful!

      "},{"location":"xtrabackup-option-reference.html#-innodbname","title":"\u2013innodb([=name])","text":"

      This option is ignored for MySQL option compatibility.

      "},{"location":"xtrabackup-option-reference.html#-innodb-miscellaneous","title":"\u2013innodb-miscellaneous()","text":"

      There is a large group of InnoDB options that are normally read from the my.cnf configuration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify these explicitly. These options have the same behavior in InnoDB and XtraDB:

      "},{"location":"xtrabackup-option-reference.html#-keyring-file-datafilename","title":"\u2013keyring-file-data(=FILENAME)","text":"

      The path to the keyring file. Combine this option with --xtrabackup-plugin-dir.

      "},{"location":"xtrabackup-option-reference.html#-kill-long-queries-timeoutseconds","title":"\u2013kill-long-queries-timeout(=SECONDS)","text":"

      This option specifies the number of seconds xtrabackup waits between starting FLUSH TABLES WITH READ LOCK and killing those queries that block it. Default is 0 seconds, which means xtrabackup will not attempt to kill any queries. In order to use this option xtrabackup user should have the PROCESS and SUPER privileges. Where supported, xtrabackup automatically uses Backup locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

      "},{"location":"xtrabackup-option-reference.html#-kill-long-query-typeallselect","title":"\u2013kill-long-query-type(=all|select)","text":"

      This option specifies which types of queries should be killed to unblock the global lock. Default is \u201cselect\u201d.

      "},{"location":"xtrabackup-option-reference.html#-lock-ddl","title":"\u2013lock-ddl()","text":"

      Issue LOCK TABLES FOR BACKUP if it is supported by server (otherwise use LOCK INSTANCE FOR BACKUP) at the beginning of the backup to block all DDL operations.

      Note

      Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.

      As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.

      "},{"location":"xtrabackup-option-reference.html#-lock-ddl-per-table","title":"\u2013lock-ddl-per-table()","text":"

      Lock DDL for each table before xtrabackup starts to copy it and until the backup is completed.

      Note

      As of Percona XtraBackup 8.0.15, the \u2013lock-ddl-per-table option is deprecated. Use the \u2013lock-ddl option instead.

      "},{"location":"xtrabackup-option-reference.html#-lock-ddl-timeout","title":"\u2013lock-ddl-timeout()","text":"

      If LOCK TABLES FOR BACKUP or LOCK INSTANCE FOR BACKUP does not return within given timeout, abort the backup.

      "},{"location":"xtrabackup-option-reference.html#-log","title":"\u2013log()","text":"

      This option is ignored for MySQL

      "},{"location":"xtrabackup-option-reference.html#-log-bin","title":"\u2013log-bin()","text":"

      The base name for the log sequence.

      "},{"location":"xtrabackup-option-reference.html#-log-bin-indexname","title":"\u2013log-bin-index(=name)","text":"

      File that holds the names for binary log files.

      "},{"location":"xtrabackup-option-reference.html#-log-copy-interval","title":"\u2013log-copy-interval(=#)","text":"

      This option specifies the time interval between checks done by the log copying thread in milliseconds (default is 1 second).

      "},{"location":"xtrabackup-option-reference.html#-login-path","title":"\u2013login-path()","text":"

      Read the given path from the login file.

      "},{"location":"xtrabackup-option-reference.html#-move-back","title":"\u2013move-back()","text":"

      Move all the files in a previously made backup from the backup directory to their original locations. As this option removes backup files, it must be used with caution.

      "},{"location":"xtrabackup-option-reference.html#-no-backup-locks","title":"\u2013no-backup-locks()","text":"

      Explicity disables the --backup-locks option which is enabled by default.

      "},{"location":"xtrabackup-option-reference.html#-no-defaults","title":"\u2013no-defaults()","text":"

      The default options are only read from the login file.

      "},{"location":"xtrabackup-option-reference.html#-no-lock","title":"\u2013no-lock()","text":"

      Disables table lock with FLUSH TABLES WITH READ LOCK. Use it only if all your tables are InnoDB and you do not care about the binary log position of the backup. This option shouldn\u2019t be used if there are any DDL statements being executed or if any updates are happening on non-InnoDB tables (this includes the system MyISAM tables in the mysql database), otherwise it could lead to an inconsistent backup. Where supported xtrabackup will automatically use Backup locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables. If you are considering to use this because your backups are failing to acquire the lock, this could be because of incoming replication events are preventing the lock from succeeding. Please try using --safe-slave-backup to momentarily stop the replication replica thread, this may help the backup to succeed and you do not need to use this option.

      "},{"location":"xtrabackup-option-reference.html#-no-server-version-check","title":"\u2013no-server-version-check()","text":"

      Implemented in Percona XtraBackup 8.0.21.

      The --no-server-version-check option disables the server version check.

      The default behavior runs a check that compares the source system version to the Percona XtraBackup version. If the source system version is higher than the XtraBackup version, the backup is aborted with a message.

      Adding the option overrides this check, and the backup proceeds, but there may be issues with the backup.

      See Server Version and Backup Version Comparison for more information.

      "},{"location":"xtrabackup-option-reference.html#-no-version-check","title":"\u2013no-version-check()","text":"

      This option disables the version check. If you do not pass this option, the automatic version check is enabled implicitly when xtrabackup runs in the --backup mode. To disable the version check, you should pass explicitly the --no-version-check option when invoking xtrabackup.

      When the automatic version check is enabled, xtrabackup performs a version check against the server on the backup stage after creating a server connection. xtrabackup sends the following information to the server:

      • MySQL flavour and version

      • Operating system name

      • Percona Toolkit version

      • Perl version

      Each piece of information has a unique identifier. This is a MD5 hash value that Percona Toolkit uses to obtain statistics about how it is used. This is a random UUID; no client information is either collected or stored.

      "},{"location":"xtrabackup-option-reference.html#-open-files-limit","title":"\u2013open-files-limit(=#)","text":"

      The maximum number of file descriptors to reserve with setrlimit().

      "},{"location":"xtrabackup-option-reference.html#-parallel","title":"\u2013parallel(=#)","text":"

      This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer). In Percona XtraBackup 2.3.10 and newer, this option can be used with the --copy-back option to copy the user data files in parallel (redo logs and system tablespaces are copied in the main thread).

      "},{"location":"xtrabackup-option-reference.html#-passwordpassword","title":"\u2013password(=PASSWORD)","text":"

      This option specifies the password to use when connecting to the database. It accepts a string argument. See mysql \u2013help for details.

      "},{"location":"xtrabackup-option-reference.html#-plugin-load","title":"\u2013plugin-load()","text":"

      List of plugins to load.

      "},{"location":"xtrabackup-option-reference.html#-portport","title":"\u2013port(=PORT)","text":"

      This option accepts a string argument that specifies the port to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql \u2013help for details.

      "},{"location":"xtrabackup-option-reference.html#-prepare","title":"\u2013prepare()","text":"

      Makes xtrabackup perform a recovery on a backup created with --backup, so that it is ready to use. See preparing a backup.

      "},{"location":"xtrabackup-option-reference.html#-print-defaults","title":"\u2013print-defaults()","text":"

      Print the program argument list and exit. Must be given as the first option on the command-line.

      "},{"location":"xtrabackup-option-reference.html#-print-param","title":"\u2013print-param()","text":"

      Makes xtrabackup print out parameters that can be used for copying the data files back to their original locations to restore them.

      "},{"location":"xtrabackup-option-reference.html#-read-buffer-size","title":"\u2013read-buffer-size()","text":"

      Set the read buffer size. The given value is scaled up to page size. The default size is 10MB. Use this option to increase the xbcloud/xbstream chunk size from the default size. To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.

      "},{"location":"xtrabackup-option-reference.html#-rebuild-indexes","title":"\u2013rebuild-indexes()","text":"

      Rebuilds indexes in a compact backup. This option only has effect when the --prepare and --rebuild-threads options are provided.

      "},{"location":"xtrabackup-option-reference.html#-rebuild-threads","title":"\u2013rebuild-threads(=#)","text":"

      Uses the given number of threads to rebuild indexes in a compact backup. This option only has effect with the --prepare and --rebuild-indexes options.

      "},{"location":"xtrabackup-option-reference.html#-register-redo-log-consumer","title":"\u2013register-redo-log-consumer()","text":"

      The --register-redo-log-consumer parameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.

      "},{"location":"xtrabackup-option-reference.html#-remove-original","title":"\u2013remove-original()","text":"

      This option when specified will remove .qp, .xbcrypt and .qp.xbcrypt files after decryption and decompression.

      "},{"location":"xtrabackup-option-reference.html#-rocksdb-datadir","title":"\u2013rocksdb-datadir()","text":"

      RocksDB data directory

      "},{"location":"xtrabackup-option-reference.html#-rocksdb-wal-dir","title":"\u2013rocksdb-wal-dir()","text":"

      RocksDB WAL directory.

      "},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-age","title":"\u2013rocksdb-checkpoint-max-age()","text":"

      The checkpoint cannot be older than this number of seconds when the backup completes.

      "},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-count","title":"\u2013rocksdb-checkpoint-max-count()","text":"

      Complete the backup even if the checkpoint age requirement has not been met after this number of checkpoints.

      "},{"location":"xtrabackup-option-reference.html#-rollback-prepared-trx","title":"\u2013rollback-prepared-trx()","text":"

      Force rollback prepared InnoDB transactions.

      "},{"location":"xtrabackup-option-reference.html#-rsync","title":"\u2013rsync()","text":"

      Uses the rsync utility to optimize local file transfers. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. This option cannot be used together with --stream.

      "},{"location":"xtrabackup-option-reference.html#-safe-slave-backup","title":"\u2013safe-slave-backup()","text":"

      When specified, xtrabackup will stop the replica SQL thread just before running FLUSH TABLES WITH READ LOCK and wait to start backup until Slave_open_temp_tables in SHOW STATUS is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds. The replication SQL thread will be restarted when the backup finishes. This option is implemented in order to deal with replicating temporary tables and isn\u2019t necessary with Row-Based-Replication.

      Note

      Using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.

      "},{"location":"xtrabackup-option-reference.html#-safe-slave-backup-timeoutseconds","title":"\u2013safe-slave-backup-timeout(=SECONDS)","text":"

      How many seconds --safe-slave-backup should wait for Slave_open_temp_tables to become zero. Defaults to 300 seconds.

      "},{"location":"xtrabackup-option-reference.html#-secure-auth","title":"\u2013secure-auth()","text":"

      Refuse client connecting to server if it uses old (pre-4.1.1) protocol. (Enabled by default; use \u2013skip-secure-auth to disable.)

      "},{"location":"xtrabackup-option-reference.html#-server-id","title":"\u2013server-id(=#)","text":"

      The server instance being backed up.

      "},{"location":"xtrabackup-option-reference.html#-server-public-key-path","title":"\u2013server-public-key-path()","text":"

      The file path to the server public RSA key in the PEM format.

      "},{"location":"xtrabackup-option-reference.html#-skip-tables-compatibility-check","title":"\u2013skip-tables-compatibility-check()","text":"

      See --tables-compatibility-check.

      "},{"location":"xtrabackup-option-reference.html#-slave-info","title":"\u2013slave-info()","text":"

      This option is useful when backing up a replication replica server. It prints the binary log position of the source server. It also writes the binary log coordinates to the xtrabackup_slave_info file as a CHANGE MASTER command. A new replica for this source can be set up by starting a replica server on this backup and issuing a CHANGE MASTER command with the binary log position saved in the xtrabackup_slave_info file.

      "},{"location":"xtrabackup-option-reference.html#-socket","title":"\u2013socket()","text":"

      This option accepts a string argument that specifies the socket to use when connecting to the local database server with a UNIX domain socket. It is passed to the mysql child process without alteration. See mysql \u2013help for details.

      "},{"location":"xtrabackup-option-reference.html#-ssl","title":"\u2013ssl()","text":"

      Enable secure connection.

      "},{"location":"xtrabackup-option-reference.html#-ssl-ca","title":"\u2013ssl-ca()","text":"

      Path of the file which contains list of trusted SSL CAs.

      "},{"location":"xtrabackup-option-reference.html#-ssl-capath","title":"\u2013ssl-capath()","text":"

      Directory path that contains trusted SSL CA certificates in PEM format.

      "},{"location":"xtrabackup-option-reference.html#-ssl-cert","title":"\u2013ssl-cert()","text":"

      Path of the file which contains X509 certificate in PEM format.

      "},{"location":"xtrabackup-option-reference.html#-ssl-cipher","title":"\u2013ssl-cipher()","text":"

      List of permitted ciphers to use for connection encryption.

      "},{"location":"xtrabackup-option-reference.html#-ssl-crl","title":"\u2013ssl-crl()","text":"

      Path of the file that contains certificate revocation lists. MySQL server documentation.

      "},{"location":"xtrabackup-option-reference.html#-ssl-crlpath","title":"\u2013ssl-crlpath()","text":"

      Path of directory that contains certificate revocation list files.

      "},{"location":"xtrabackup-option-reference.html#-ssl-fips-mode","title":"\u2013ssl-fips-mode()","text":"

      SSL FIPS mode (applies only for OpenSSL); permitted values are: OFF, ON, STRICT.

      "},{"location":"xtrabackup-option-reference.html#-ssl-key","title":"\u2013ssl-key()","text":"

      Path of file that contains X509 key in PEM format.

      "},{"location":"xtrabackup-option-reference.html#-ssl-mode","title":"\u2013ssl-mode()","text":"

      Security state of connection to server.

      "},{"location":"xtrabackup-option-reference.html#-ssl-verify-server-cert","title":"\u2013ssl-verify-server-cert()","text":"

      Verify server certificate Common Name value against host name used when connecting to server.

      "},{"location":"xtrabackup-option-reference.html#-streamformat","title":"\u2013stream(=FORMAT)","text":"

      Stream all backup files to the standard output in the specified format. Currently, this option only supports the xbstream format.

      "},{"location":"xtrabackup-option-reference.html#-strict","title":"\u2013strict()","text":"

      If this option is specified, xtrabackup fails with an error when invalid parameters are passed.

      "},{"location":"xtrabackup-option-reference.html#-tablesname","title":"\u2013tables(=name)","text":"

      A regular expression against which the full tablename, in databasename.tablename format, is matched. If the name matches, the table is backed up. See partial backups.

      "},{"location":"xtrabackup-option-reference.html#-tables-compatibility-check","title":"\u2013tables-compatibility-check()","text":"

      Enables the engine compatibility warning. The default value is ON. To disable the engine compatibility warning use --skip-tables-compatibility-check.

      "},{"location":"xtrabackup-option-reference.html#-tables-excludename","title":"\u2013tables-exclude(=name)","text":"

      Filtering by regexp for table names. Operates the same way as --tables, but matched names are excluded from backup. Note that this option has a higher priority than --tables.

      "},{"location":"xtrabackup-option-reference.html#-tables-filename","title":"\u2013tables-file(=name)","text":"

      A file containing one table name per line, in databasename.tablename format. The backup will be limited to the specified tables.

      "},{"location":"xtrabackup-option-reference.html#-target-dirdirectory","title":"\u2013target-dir(=DIRECTORY)","text":"

      This option specifies the destination directory for the backup. If the directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup will succeed. xtrabackup will not overwrite existing files, however; it will fail with operating system error 17, file exists.

      If this option is a relative path, it is interpreted as being relative to the current working directory from which xtrabackup is executed.

      In order to perform a backup, you need READ, WRITE, and EXECUTE permissions at a filesystem level for the directory that you supply as the value of --target-dir.

      "},{"location":"xtrabackup-option-reference.html#-innodb-temp-tablespaces-dirdirectory","title":"\u2013innodb-temp-tablespaces-dir(=DIRECTORY)","text":"

      Directory where temp tablespace files live, this path can be absolute.

      "},{"location":"xtrabackup-option-reference.html#-throttle","title":"\u2013throttle(=#)","text":"

      This option limits the number of chunks copied per second. The chunk size is 10 MB.

      To limit the bandwidth to 10 MB/s, set the option to 1.

      "},{"location":"xtrabackup-option-reference.html#-tls-ciphersuites","title":"\u2013tls-ciphersuites()","text":"

      TLS v1.3 cipher to use.

      "},{"location":"xtrabackup-option-reference.html#-tls-version","title":"\u2013tls-version()","text":"

      TLS version to use, permitted values are: TLSv1, TLSv1.1, TLSv1.2, TLSv1.3.

      "},{"location":"xtrabackup-option-reference.html#-tmpdirname","title":"\u2013tmpdir(=name)","text":"

      Specify the directory that will be used to store temporary files during the backup

      "},{"location":"xtrabackup-option-reference.html#-transition-keyname","title":"\u2013transition-key(=name)","text":"

      This option is used to enable processing the backup without accessing the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and uses it to encrypt tablespace keys of tablespaces being backed up.

      If --transition-key does not have any value, xtrabackup will ask for it. The same passphrase should be specified for the --prepare command.

      "},{"location":"xtrabackup-option-reference.html#-use-free-memory-pct","title":"\u2013use-free-memory-pct()","text":"

      The --use-free-memory-pct is a tech preview option. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

      Implemented in Percona XtraBackup 8.0.30-23, this option lets you configure the Smart memory estimation feature. The option controls the amount of free memory that can be used to --prepare a backup. The default value is 0 (zero) which defines the option as disabled. For example, if you set --use-free-memory-pct=50, then 50% of the free memory is used to prepare a backup. The maximum allowed value is 100.

      This option works, only if --estimate-memory option is enabled. If the --estimate-memory option is disabled, the --use-free-memory-pct setting is ignored.

      An example of how to enable the Smart memory estimation feature:

      $ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n
      $ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\n
      "},{"location":"xtrabackup-option-reference.html#-use-memory","title":"\u2013use-memory()","text":"

      This option affects how much memory is allocated and is similar to innodb_buffer_pool_size. This option is only relevant in the --prepare phase. The default value is 100MB. The recommended value is between 1GB to 2GB. Multiple values are supported if you provide the unit (for example, 1MB, 1M, 1GB, 1G).

      "},{"location":"xtrabackup-option-reference.html#-userusername","title":"\u2013user(=USERNAME)","text":"

      This option specifies the MySQL username used when connecting to the server, if that\u2019s not the current user. The option accepts a string argument. See mysql \u2013help for details.

      "},{"location":"xtrabackup-option-reference.html#-v","title":"-v()","text":"

      See --version

      "},{"location":"xtrabackup-option-reference.html#-version","title":"\u2013version()","text":"

      This option prints xtrabackup version and exits.

      "},{"location":"xtrabackup-option-reference.html#-xtrabackup-plugin-dirdirname","title":"\u2013xtrabackup-plugin-dir(=DIRNAME)","text":"

      The absolute path to the directory that contains the keyring plugin.

      "},{"location":"xtrabackup-option-reference.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"xtrabackup-version-numbers.html","title":"Understand version numbers","text":"

      A version number identifies the innovtion product release. The product contains the latest features, improvements, and bug fixes at the time of that release.

      8.1.0 -1 Base version Minor build version

      Percona uses semantic version numbering, which follows the pattern of base version and build version. Percona assigns unique, non-negative integers in increasing order for each version release. The version number combines the base MySQL 8.1 version number and the minor build version.

      The version numbers for Percona XtraBackup 8.1.0-1 define the following information:

      • Base version - the leftmost numbers indicate the MySQL 8.1 version used as a base. An increase in base version resets the minor build version to 0.

      • Minor build version - an internal number that denotes the version of the software. A build version increases by one each time the Percona XtraBackup is released.

      "},{"location":"xtrabackup-version-numbers.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"yum-download-rpm.html","title":"Install Percona XtraBackup 8.1 using downloaded RPM packages","text":"

      Download RPM packages of the desired series for your architecture from the download page.

      The following example downloads Percona XtraBackup 8.1.0-1 release package for CentOS 7:

      $ wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-8.1.0-1/binary/redhat/7/x86_64/percona-xtrabackup-81-8.1.0-1.el7.x86_64.rpm\n

      Install Percona XtraBackup by running:

      $ yum localinstall percona-xtrabackup-81-8.1.0-1.el7.x86_64.rpm\n

      When installing packages manually like this, you\u2019ll need to make sure to resolve all the dependencies and install missing packages yourself.

      "},{"location":"yum-download-rpm.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"yum-files.html","title":"Files in the RPM package built for Percona XtraBackup 8.1","text":"

      The following tables show what data each RPM package contains:

      Package Contains percona-xtrabackup-81 The latest Percona XtraBackup GA binaries and associated files percona-xtrabackup-81-debuginfo The debug symbols for binaries in percona-xtrabackup-81 percona-xtrabackup-test-81 The test suite for Percona XtraBackup percona-xtrabackup The older version of the Percona XtraBackup"},{"location":"yum-files.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"yum-repo.html","title":"Use a YUM repository to install Percona XtraBackup","text":"

      Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page. The Percona yum repository supports popular RPM-based operating systems, including the Amazon Linux AMI.

      The easiest way to install the Percona Yum repository is to install an RPM that configures yum and installs the Percona GPG key.

      Specific information on the supported platforms, products, and versions is described in Percona Software and Platform Lifecycle.

      "},{"location":"yum-repo.html#install-percona-xtrabackup-from-percona-yum-repository","title":"Install Percona XtraBackup from Percona yum repository","text":"

      To install Percona XtraBackup from Percona yum repository, do the following steps:

      1. Install the Percona yum repository by running the following command as the root user or with sudo: 

        $ sudo yum install \\\nhttps://repo.percona.com/yum/percona-release-latest.\\\nnoarch.rpm\n

      2. Enable the repository: 

        $ sudo percona-release enable-only tools release\n

        If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the `tools repository: 

        $ sudo percona-release enable-only tools\n

      3. Install Percona XtraBackup by running:

        $ sudo yum install percona-xtrabackup-81\n

        Warning

        Make sure that you have the libev package installed before installing Percona XtraBackup on CentOS 6. For this operating system, the libev package is available from the EPEL repositories.

      4. To decompress backups made using LZ4 or ZSTD compression algorithm, install the corresponding package:

        Install the lz4 packageInstall the zstd package 
        $ sudo yum install lz4\n
        $ sudo yum install zstd\n

      See also

      To install Percona XtraBackup using downloaded rpm packages, see Install with package manager.

      To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup

      "},{"location":"yum-repo.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"yum-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.1 on Red Hat Enterprise Linux and CentOS","text":"

      To completely uninstall Percona XtraBackup, remove all the installed packages:

      $ yum remove percona-xtrabackup\n
      "},{"location":"yum-uninstall-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"release-notes/8.1.0-1.html","title":"Percona XtraBackup 8.1.0-1 (2023-11-02)","text":"

      Get started with Quickstart Guide for Percona XtraBackup.

      Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.

      This is an Innovation release. This type of release is only supported for a short time and is designed to be used in an environment with fast upgrade cycles. Developers and DBAs are exposed to the latest features and improvements.

      "},{"location":"release-notes/8.1.0-1.html#release-highlights","title":"Release highlights","text":"

      Percona XtraBackup 8.1.0-1 is based on MySQL 8.1 and fully supports the Percona Server for MySQL 8.1 Innovation series and the MySQL 8.1 Innovation series. This release allows taking backups of Percona Server 8.1.0-1 and MySQL 8.1.

      Use the Percona XtraBackup 8.0 series to take backups of Percona Server for MySQL 8.0.x or MySQL 8.0.x. Percona XtraBackup 8.1.0-1 does not take a backup of the Percona Server for MySQL 8.0 or the MySQL 8.0.x series.

      "},{"location":"release-notes/8.1.0-1.html#improvement","title":"Improvement","text":"
      • PXB-3155 : Moved the vault from a plugin to a component.
      "},{"location":"release-notes/8.1.0-1.html#deprecated-or-removed","title":"Deprecated or removed","text":"

      The --stats mode of operation for the xtrabackup binary has been removed.

      Only the keyring_vault component, the KMIP component, and the AWS KMS component versions are supported.

      For the keyring_file both the plugin and component are supported. The keyring_file plugin is deprecated and may be removed in the future.

      "},{"location":"release-notes/8.1.0-1.html#useful-links","title":"Useful links","text":"

      Install Percona XtraBackup 8.1

      The Percona XtraBackup GitHub location

      Download product binaries, packages, and tarballs at Percona Product Downloads

      "},{"location":"release-notes/8.1.0-1.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"release-notes/release-notes.html","title":"Percona XtraBackup 8.1 release notes index","text":"
      • Percona XtraBackup 8.1.0-1 (2023-11-02)
      "},{"location":"release-notes/release-notes.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-,:!=\\[\\]()\"`/]+|\\.(?!\\d)|&[lg]t;|(?!\\b)(?=[A-Z][a-z])","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Percona XtraBackup 8.1 Documentation","text":"

      This documentation is for the latest release: Percona XtraBackup 8.1 (Release Notes). This is an Innovation release. This type of release is only supported for a short time and is designed to be used in an environment with fast upgrade cycles. Developers and DBAs are exposed to the latest features and improvements.

      Percona XtraBackup is an open source hot backup utility for MySQL-based servers that keep your database fully available during planned maintenance windows.

      Whether it is a 24x7 highly loaded server or a low-transaction-volume Percona XtraBackup is designed to make backups seamless without disrupting the server\u2019s performance in a production environment. Percona XtraBackup (PXB) is a 100% open source backup solution with commercial support available for organizations who want to benefit from comprehensive, responsive, and cost-flexible database support for MySQL.

      This is an Innovation release. This type of release is only supported for a short time and is designed to be used in an environment with fast upgrade cycles. Developers and DBAs are exposed to the latest features and improvements.

      "},{"location":"index.html#supported-storage-engines","title":"Supported storage engines","text":"

      Percona XtraBackup can back up data from InnoDB, XtraDB, MyISAM, MyRocks tables on MySQL 8.1 servers and Percona Server for MySQL with XtraDB, Percona Server for MySQL 8.1, and Percona XtraDB Cluster 8.1.

      Percona XtraBackup 8.1 supports the MyRocks storage engine. An incremental backup on the MyRocks storage engine does not determine if an earlier full or incremental backup contains duplicate files. Percona XtraBackup copies all MyRocks files each time it takes a backup.

      "},{"location":"index.html#limitations","title":"Limitations","text":"

      Percona XtraBackup 8.1 does not support making backups of databases created in versions before 8.1 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster.

      "},{"location":"index.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"404.html","title":"404 - Not Found","text":"

      We can\u2019t find the page you are looking for. Try using the Search or return to the homepage.

      "},{"location":"404.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"about-xtrabackup.html","title":"About Percona XtraBackup","text":"

      Percona XtraBackup is the world\u2019s only open source, free MySQL hot backup software that performs non-blocking backups for InnoDB and XtraDB databases.

      Percona XtraBackup has the following benefits:

      • Complete a backup quickly and reliably

      • Process transactions uninterrupted during a back up

      • Save on disk space and network bandwidth

      • Verify backup automatically

      Percona XtraBackup makes hot backups for Percona Server for MySQL and MySQL, takes streaming, compressed, and incremental MySQL backups, and supports encryption.

      Percona\u2019s enterprise-grade commercial MySQL Support contracts include support for Percona XtraBackup. We recommend support for critical production deployments.

      "},{"location":"about-xtrabackup.html#supported-storage-engines","title":"Supported storage engines","text":"

      Percona XtraBackup can back up data from InnoDB, XtraDB, MyISAM, and MyRocks tables on MySQL 8.1 servers as well as Percona Server for MySQL with XtraDB, and Percona XtraDB Cluster.

      Percona XtraBackup supports the MyRocks storage engine. An incremental backup on the MyRocks storage engine does not determine if an earlier full or incremental backup contains the same files. Percona XtraBackup copies all MyRocks files each time it takes a backup. Percona XtraBackup does not support the TokuDB storage engine.

      "},{"location":"about-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"accelerate-backup-process.html","title":"Accelerate backup process","text":""},{"location":"accelerate-backup-process.html#accelerate-the-backup-process","title":"Accelerate the backup process","text":""},{"location":"accelerate-backup-process.html#copy-with-the-parallel-and-compress-threads-options","title":"Copy with the --parallel and --compress-threads options","text":"

      When making a local or streaming backup with xbstream binary, multiple files can be copied at the same time when using the --parallel option. This option specifies the number of threads created by xtrabackup to copy data files.

      To take advantage of this option either the multiple tablespaces option must be enabled (innodb_file_per_table) or the shared tablespace must be stored in multiple ibdata files with the innodb_data_file_path option. Having multiple files for the database (or splitting one into many) doesn\u2019t have a measurable impact on performance.

      As this feature is implemented at the file level, concurrent file transfer can sometimes increase I/O throughput when doing a backup on highly fragmented data files, due to the overlap of a greater number of random read requests. You should consider tuning the filesystem also to obtain the maximum performance (e.g. checking fragmentation).

      If the data is stored on a single file, this option has no effect.

      To use this feature, simply add the option to a local backup, for example:

      $ xtrabackup --backup --parallel=4 --target-dir=/path/to/backup\n

      By using the xbstream in streaming backups, you can additionally speed up the compression process with the --compress-threads option. This option specifies the number of threads created by xtrabackup for for parallel data compression. The default value for this option is 1.

      To use this feature, simply add the option to a local backup, for example:

      $ xtrabackup --backup --stream=xbstream --compress --compress-threads=4 --target-dir=./ > backup.xbstream\n

      Before applying logs, compressed files will need to be uncompressed.

      "},{"location":"accelerate-backup-process.html#the-rsync-option","title":"The --rsync option","text":"

      In order to speed up the backup process and to minimize the time FLUSH TABLES WITH READ LOCK is blocking the writes, the option --rsync should be used. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. xtrabackup will call the rsync twice, once before the FLUSH TABLES WITH READ LOCK and once during to minimize the time the read lock is being held. During the second rsync call, it will only synchronize the changes to non-transactional data (if any) since the first call performed before the FLUSH TABLES WITH READ LOCK. Note that Percona XtraBackup will use Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK.

      Percona XtraBackup uses these locks automatically to copy non-InnoDB data to avoid blocking Data manipulation language (DML) queries that modify InnoDB tables.

      Note

      This option cannot be used together with the --stream option.

      "},{"location":"accelerate-backup-process.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"apt-download-deb.html","title":"Install using downloaded DEB packages","text":"

      Download DEB packages of the desired series for your architecture from Percona Product Downloads. This method requires you to resolve all dependencies and install any missing packages.

      The following example downloads Percona XtraBackup 8.1.0-1 release package for Ubuntu 20.04:

      $ wget https://downloads.percona.com/downloads/Percona-XtraBackup-LATEST/Percona-XtraBackup-8.1.0-1/binary/debian/focal/x86_64/percona-xtrabackup-81_8.1.0-1.focal_amd64.deb\n

      Install Percona XtraBackup by using dpkg. Run this command as root or use the sudo command:

      $ sudo dpkg -i percona-xtrabackup-81_8.1.0-1.focal_amd64.deb\n
      "},{"location":"apt-download-deb.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"apt-files.html","title":"Files in the DEB package","text":"

      The following tables show what data each DEB package contains.

      Package Contains percona-xtrabackup-81 The latest Percona XtraBackup GA binaries and associated files percona-xtrabackup-dbg-81 The debug symbols for binaries in percona-xtrabackup-81 percona-xtrabackup-test-81 The test suite for Percona XtraBackup percona-xtrabackup The older version of the Percona XtraBackup"},{"location":"apt-files.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"apt-pinning.html","title":"Apt pinning packages","text":"

      In some cases you might need to pin the selected packages to avoid the upgrades from the distribution repositories.

      The pinning takes place in the preference file. To pin a package, set the Pin-Priority to higher numbers.

      Make a new file /etc/apt/preferences.d/00percona.pref. For example, add the following to the preference file:

      Package:\nPin: release o=Percona Development Team\nPin-Priority: 1001\n

      For more information about the pinning, check the official debian wiki.

      "},{"location":"apt-pinning.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"apt-repo.html","title":"Install with an APT repository","text":"

      Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page.

      Specific information on the supported platforms, products, and versions is described in Percona Release Lifecycle Overview.

      "},{"location":"apt-repo.html#install-percona-xtrabackup-through-percona-release","title":"Install Percona XtraBackup through percona-release","text":"

      Percona XtraBackup, like many other Percona products, is installed with the percona-release package configuration tool.

      1. Download a DEB package for percona-release the repository packages from Percona web:

        $ wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb\n

      2. Install the downloaded package with dpkg. To do that, run the following commands as root or with sudo: dpkg -i percona-release_latest.$(lsb_release -sc)_all.deb

        Once you install this package the Percona repositories should be added. You can check the repository setup in the /etc/apt/sources.list.d/percona-release.list file.

      3. Enable the repository: percona-release enable-only tools release

        If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the tools repository: percona-release enable-only tools.

      4. Refresh the local cache to update the package information:

        $ sudo apt update\n

      5. Install the percona-xtrabackup-81 package:

        $ sudo apt install percona-xtrabackup-81\n

      6. To decompress backups made using LZ4 or ZSTD compression algorithm, install the corresponding package:

        Install the lz4 packageInstall the zstd package 
        $ sudo apt install lz4\n
        $ sudo apt install zstd\n

      Note

      For AppArmor profile information, see Working with AppArmor.

      See also

      To install Percona XtraBackup using downloaded deb packages, see Install Percona XtraBackup 8.1.

      To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup 8.1

      "},{"location":"apt-repo.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"apt-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.1 on Debian and Ubuntu","text":"

      To completely uninstall Percona XtraBackup, remove all the installed packages:

      $ sudo apt remove percona-xtrabackup-81\n
      "},{"location":"apt-uninstall-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"binaries-overview.html","title":"Percona XtraBackup binaries overview","text":"

      Percona XtraBackup contains a set of the following binaries:

      • xtrabackup - a compiled C binary that provides functionality to backup a whole MySQL database instance with MyISAM, InnoDB, and XtraDB tables.

      • xbcrypt - a utility used for encrypting and decrypting backup files.

      • xbstream - a utility that allows streaming and extracting files to or from the xbstream format.

      • xbcloud - a utility used for downloading and uploading full or part of xbstream archive from or to cloud.

      The recommended way to take the backup is by using the xtrabackup script. For more information on script options, see xtrabackup.

      "},{"location":"binaries-overview.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"binary-tarball-names.html","title":"Binary tarball file names available","text":"

      Download the binary tarballs from Percona Product Downloads.

      The following table lists the tarball types available in Linux - Generic. Select the Percona XtraBackup 8.1 version, the software or the operating system, and the type of tarball for your installation. Binary tarballs support all distributions.

      After you have downloaded the binary tarballs, extract the tarball in the file location of your choice.

      Type Name Operating systems Description Full percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.12.tar.gz Built for CentOS 6 Contains binary files, libraries, test files, and debug symbols Minimal percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.12-minimal.tar.gz Built for CentOS 6 Contains binary files, and libraries but does not include test files, or debug symbols Full percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.17.tar.gz Compatible with any operating system except for CentOS 6 Contains binary files, libraries, test files, and debug symbols Minimal percona-xtrabackup-8.1.0-1-Linux.x86_64.glibc2.17-minimal.tar.gz Compatible with any operating system except for CentOS 6 Contains binary files, and libraries but does not include test files, or debug symbols

      Select a different software, such as Ubuntu 20.04 (Focal Fossa), for a tarball for that operating system. You can download the packages together or separately.

      "},{"location":"binary-tarball-names.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"binary-tarball.html","title":"Install from a binary tarball","text":"

      Binary tarballs contain precompiled executable files, libraries, and other dependencies and are compressed tar archives. Extract the binary tarballs to any path.

      Download the binary tarballs from the Linux - Generic section on Percona Product Downloads.

      The following example downloads the a tarball:

      $ wget https://downloads.percona.com/downloads/Percona-XtraBackup-8.1/Percona-XtraBackup-8.1.0-1/binary/tarball/percona-xtrabackup-8.1.0-1-Linux-x86_64.glibc2.17.tar.gz\n

      The output displays the following information:

      Expected output 
      --2023-10-20 05:56:30--  https://downloads.percona.com/downloads/Percona-XtraBackup-8.1/Percona-XtraBackup-8.1.0-1/binary/tarball/percona-xtrabackup-8.1.0-1-Linux-x86_64.glibc2.17.tar.gz\nResolving downloads.percona.com (downloads.percona.com)... 2604:2dc0:200:69f::2, 147.135.54.159\nConnecting to downloads.percona.com (downloads.percona.com)|2604:2dc0:200:69f::2|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 470358258 (449M) [application/gzip]\nSaving to: \u2018percona-xtrabackup-8.1.0-1-Linux-x86_64.glibc2.17.tar.gz\u2019\n\npercona-xtrabackup   0%[                       ]   3.17M  1.54MB/s\n
      "},{"location":"binary-tarball.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"compile-xtrabackup.html","title":"Compile xtrabackup","text":""},{"location":"compile-xtrabackup.html#compile-and-install-from-source","title":"Compile and install from source","text":"

      The following instructions install Percona XtraBackup 8.1.

      "},{"location":"compile-xtrabackup.html#1-install-percona-xtrabackup-from-the-git-source-tree","title":"1. Install Percona XtraBackup from the Git Source Tree","text":"

      Percona uses the Github revision control system for development. To build the latest Percona Server for MySQL from the source tree, you will need git installed on your system.

      You can now fetch the latest Percona XtraBackup 8.1 sources:

      $ git clone https://github.com/percona/percona-xtrabackup.git\n$ cd percona-xtrabackup\n$ git checkout trunk\n$ git submodule update --init --recursive\n
      "},{"location":"compile-xtrabackup.html#2-installation-prerequisites","title":"2. Installation prerequisites","text":"

      The following packages and tools must be installed to compile Percona XtraBackup from source. These might vary from system to system.

      Important

      To build **Percona XtraBackup 8.1 from source, you must use cmake version 3. To check which version is currently installed, run cmake --version at a command prompt. If the version is not 3, install cmake3.

      This cmake version may be available in your distribution as a separate package cmake3. For more information, see cmake.org.

      Debian or Ubuntu using aptCentOS or Red Hat using yum 
      sudo apt install bison pkg-config cmake devscripts debconf \\\ndebhelper automake bison ca-certificates libprocps-dev \\\nlibcurl4-openssl-dev cmake debhelper libaio-dev \\\nlibncurses-dev libssl-dev libtool libz-dev libgcrypt-dev libev-dev libprocps-dev \\\nlsb-release build-essential rsync libdbd-mysql-perl \\\nlibnuma1 socat librtmp-dev libtinfo5 vim-common \\\nliblz4-tool liblz4-1 liblz4-dev zstd python-docutils\n

      To install the man pages, install the python3-sphinx package first:

      $ sudo apt install python3-sphinx\n

      Percona Xtrabackup requires GCC version 5.3 or higher. If the version of GCC installed on your system is lower then you may need to install and enable the Developer Toolset on RPM-based distributions to make sure that you use the latest GCC compiler and development tools. Then, install cmake and other dependencies:

      $ sudo yum install cmake openssl-devel libaio libaio-devel automake autoconf \\\nbison libtool ncurses-devel libgcrypt-devel libev-devel libcurl-devel zlib-devel \\\nzstd vim-common procps-ng-devel\n

      To install the man pages, install the python3-sphinx package first:

      $ sudo yum install python3-sphinx\n
      "},{"location":"compile-xtrabackup.html#3-generate-the-build-pipeline","title":"3. Generate the build pipeline","text":"

      At this step, you have cmake run the commands in the CMakeList.txt file to generate the build pipeline, i.e., a native build environment that will be used to compile the source code).

      1. Change to the directory where you cloned the Percona XtraBackup repository

        $ cd percona-xtrabackup\n

      2. Create a directory to store the compiled files and then change to that directory:

        $ mkdir build\n$ cd build\n

      3. Run cmake or cmake3. In either case, the options you need to use are the same.

      Note

      You can build Percona XtraBackup with man pages but this requires python-sphinx package which isn\u2019t available from that main repositories for every distribution. If you installed the python-sphinx package you need to remove the -DWITH_MAN_PAGES=OFF from previous command.

      $ cmake -DWITH_BOOST=PATH-TO-BOOST-LIBRARY -DDOWNLOAD_BOOST=ON \\\n-DBUILD_CONFIG=xtrabackup_release -DWITH_MAN_PAGES=OFF -B ..\n
      "},{"location":"compile-xtrabackup.html#parameter-information","title":"Parameter Information","text":"Parameter Description -DWITH_BOOST For the -DWITH_BOOST parameter, specify the name of a directory to download the boost library to. This directory is created automatically in your current directory. -DWITH_MAN_PAGES To build Percona XtraBackup man pages, use ON or remove this parameter from the command line (it is ON by default). To install the man pages, install the python3-sphinx package first. -B (\u2013build) Percona XtraBackup is configured to forbid generating the build pipeline for make in the same directory where you store your sources. The -B parameter refers to the directory that contains the source code. In this example, we use the relative path to the parent directory (..).

      Important

      CMake Error at CMakeLists.txt:367 (MESSAGE): Please do not build in-source. Out-of source builds are highly recommended: you can have multiple builds for the same source, and there is an easy way to do cleanup, simply remove the build directory (note that \u2018make clean\u2019 or \u2018make distclean\u2019 does not work)

      You can force in-source build by invoking cmake with -DFORCE_INSOURCE_BUILD=1.

      "},{"location":"compile-xtrabackup.html#4-compile-the-source-code","title":"4. Compile the source code","text":"

      To compile the source code in your build directory, use the make command.

      1. Change to the build directory (created at Step 2: Generating the build pipeline).

      2. Run the make command. This command may take a long time to complete.

        $ make\n

        To use all CPU threads and make compilation faster please use:

        $ make -j$(nproc --all)\n
      "},{"location":"compile-xtrabackup.html#5-install-on-the-target-system","title":"5. Install on the target system","text":"

      The following command installs all Percona XtraBackup binaries xtrabackup and tests to default location on the target system: /usr/local/xtrabackup.

      Run make install to install Percona XtraBackup to the default location.

      $ sudo make install\n
      "},{"location":"compile-xtrabackup.html#install-to-a-non-default-location","title":"Install to a non-default location","text":"

      You may use the DESTDIR parameter with make install to install Percona XtraBackup to another location. Make sure that the effective user is able to write to the destination you choose.

      $ sudo make DESTDIR=<DIR_NAME> install\n

      In fact, the destination directory is determined by the installation layout (-DINSTALL_LAYOUT) that cmake applies (see Step 2: Generating the build pipeline). In addition to the installation directory, this parameter controls a number of other destinations that you can adjust for your system.

      By default, this parameter is set to STANDALONE, which implies the installation directory to be /usr/local/xtrabackup.

      See also

      MySQL Documentation: -DINSTALL_LAYOUT

      "},{"location":"compile-xtrabackup.html#6-run-percona-xtrabackup","title":"6. Run Percona XtraBackup","text":"

      After Percona XtraBackup is installed on your system, you may run it by using the full path to the xtrabackup command:

      $ /usr/local/xtrabackup/bin/xtrabackup\n

      Update your PATH environment variable if you would like to use the command on the command line directly.

      $# Setting $PATH on the command line\n$ PATH=$PATH:/usr/local/xtrabackup/bin/xtrabackup\n\n$# Run xtrabackup directly\n$ xtrabackup\n

      Alternatively, you may consider placing a soft link (using ln -s) to one of the locations listed in your PATH environment variable.

      To view the documentation with man, update the MANPATH variable.

      "},{"location":"compile-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"configure-xtrabackup.html","title":"Configure xtrabackup","text":"

      All the xtrabackup configuration is done through options, which behave exactly like standard MySQL program options: they can be specified either at the command-line, or through a file such as /etc/my.cnf.

      The xtrabackup binary reads the [mysqld] and [xtrabackup] sections from any configuration files, in that order. That is so that it can read its options from your existing MySQL installation, such as the datadir or some of the InnoDB options. If you want to override these, just specify them in the [xtrabackup] section, and because it is read later, it will take precedence.

      You don\u2019t need to put any configuration in your my.cnf. You can specify the options on the command-line. Normally, the only thing you may find convenient to place in the [xtrabackup] section of your my.cnf file is the target_dir option. This options adds a default path to the directory for backups.

      The following is an example:

      [xtrabackup]\ntarget_dir = /data/backups/\n

      This manual assumes you do not have any file-based configuration for xtrabackup and shows the command-line options.

      Please see the option and variable reference for details on all the configuration options.

      The xtrabackup binary does not accept exactly the same syntax in the my.cnf file as the mysqld server binary does. For historical reasons, the mysqld server binary accepts parameters with a --set-variable=<variable>=<value> syntax, which xtrabackup does not understand. If your my.cnf file has such configuration directives, you should rewrite them in the --variable=value syntax.

      "},{"location":"configure-xtrabackup.html#system-configuration-and-nfs-volumes","title":"System configuration and NFS volumes","text":"

      The xtrabackup tool requires no special configuration on most systems. However, the storage where the --target-dir is located must behave properly when fsync() is called. In particular, we have noticed that if you mount the NFS volume without the sync option the NFS volume does not sync the data. As a result, if you back up to an NFS volume mounted with the async option, and then try to prepare the backup from a different server that also mounts that volume, the data might appear to be corrupt. Use the sync mount option to avoid this issue.

      "},{"location":"configure-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"copyright-and-licensing-information.html","title":"Copyright and licensing information","text":""},{"location":"copyright-and-licensing-information.html#documentation-licensing","title":"Documentation licensing","text":"

      Percona XtraBackup documentation is (C)2009-2023 Percona LLC and/or its affiliates and is distributed under the Creative Commons Attribution 4.0 International License.

      "},{"location":"copyright-and-licensing-information.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"create-compressed-backup.html","title":"Create a compressed backup","text":"

      Percona XtraBackup supports compressed backups. To make a compressed backup, use the --compress option along with the --backup and --target-dir options. A local or streaming backup can be compressed or decompressed with xbstream.

      By default, the --compress option uses the zstandard tool that you can install with the percona-release package configuration tool as follows:

      $ sudo percona-release enable tools\n$ sudo apt update\n$ sudo apt install zstandard\n

      Note

      Enable the repository: percona-release enable-only tools release.

      If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the tools repository: percona-release enable-only tools.

      Percona XtraBackup supports the following compression algorithms:

      "},{"location":"create-compressed-backup.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

      The Zstandard (ZSTD) is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. ZSTD is the default compression algorithm for the --compress option.

      To compress files using the ZSTD compression algorithm, use the --compress option:

      $ xtrabackup --backup --compress --target-dir=/data/backup\n

      The resulting files have the \\*.zst format.

      You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.

      $ xtrabackup \u2013backup \u2013compress \u2013compress-zstd-level=1 \u2013target-dir=/data/backup\n
      "},{"location":"create-compressed-backup.html#lz4","title":"lz4","text":"

      To compress files using the lz4 compression algorithm, set the --compress option to lz4:

      $ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\n

      The resulting files have the \\*.lz4 format.

      If you want to speed up the compression you can use the parallel compression, which can be enabled with --compress-threads option. Following example will use four threads for compression:

      $ xtrabackup --backup --compress --compress-threads=4 \\\n--target-dir=/data/compressed/\n
      Expected output 
      ...\n170223 13:00:38 [01] Compressing ./test/sbtest1.frm to /tmp/compressed/test/sbtest1.frm.qp\n170223 13:00:38 [01]        ...done\n170223 13:00:38 [01] Compressing ./test/sbtest2.frm to /tmp/compressed/test/sbtest2.frm.qp\n170223 13:00:38 [01]        ...done\n...\n170223 13:00:39 [00] Compressing xtrabackup_info\n170223 13:00:39 [00]        ...done\nxtrabackup: Transaction log of lsn (9291934) to (9291934) was copied.\n170223 13:00:39 completed OK!\n
      "},{"location":"create-compressed-backup.html#next-step","title":"Next step","text":"

      Prepare the backup

      "},{"location":"create-compressed-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"create-full-backup.html","title":"Create a full backup","text":"

      To create a backup, run xtrabackup with the --backup option. You also must specify the --target-dir option, which is where the backup is stored. If the InnoDB data or log files are not stored in the same directory, you must specify their location. If the target directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup succeeds.

      xtrabackup does not overwrite existing files. it will fail with operating system error 17, file exists.

      The following command starts the process:

      $ xtrabackup --backup --target-dir=/data/backups/\n

      This operation stores the backup at /data/backups/. If you specify a relative path, the target directory is relative to the current directory.

      During the backup process, the output shows the copied data files, and a log file thread that scans and copies from the log files.

      The following is an example of the output:

      Expected output 
      160906 10:19:17 Finished backing up non-InnoDB tables and files\n160906 10:19:17 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...\nxtrabackup: The latest check point (for incremental): '62988944'\nxtrabackup: Stopping log copying thread.\n.160906 10:19:18 >> log scanned up to (137343534)\n160906 10:19:18 Executing UNLOCK TABLES\n160906 10:19:18 All tables unlocked\n160906 10:19:18 Backup created in directory '/data/backups/'\n160906 10:19:18 [00] Writing backup-my.cnf\n160906 10:19:18 [00]        ...done\n160906 10:19:18 [00] Writing xtrabackup_info\n160906 10:19:18 [00]        ...done\nxtrabackup: Transaction log of lsn (26970807) to (137343534) was copied.\n160906 10:19:18 completed OK!\n

      The process ends with the following statement; the value of the <LSN> depends on your system:

      $ xtrabackup: Transaction log of lsn (<LSN>) to (<LSN>) was copied.\n

      Note

      Log copying thread checks the transactional log every second to see if there were any new log records written that need to be copied, but there is a chance that the log copying thread might not be able to keep up with the amount of writes that go to the transactional logs, and will hit an error when the log records are overwritten before they could be read.

      After the backup is finished, the target directory will contain files such as the following, assuming you have a single InnoDB table test.tbl1 and you are using MySQL\u2019s innodb_file_per_table option:

      $ ls -lh /data/backups/\n

      The result should look like this:

      Expected output 
      total 182M\ndrwx------  7 root root 4.0K Sep  6 10:19 .\ndrwxrwxrwt 11 root root 4.0K Sep  6 11:05 ..\n-rw-r-----  1 root root  387 Sep  6 10:19 backup-my.cnf\n-rw-r-----  1 root root  76M Sep  6 10:19 ibdata1\ndrwx------  2 root root 4.0K Sep  6 10:19 mysql\ndrwx------  2 root root 4.0K Sep  6 10:19 performance_schema\ndrwx------  2 root root 4.0K Sep  6 10:19 sbtest\ndrwx------  2 root root 4.0K Sep  6 10:19 test\ndrwx------  2 root root 4.0K Sep  6 10:19 world2\n-rw-r-----  1 root root  116 Sep  6 10:19 xtrabackup_checkpoints\n-rw-r-----  1 root root  433 Sep  6 10:19 xtrabackup_info\n-rw-r-----  1 root root 106M Sep  6 10:19 xtrabackup_logfile\n

      The backup can take a long time, depending on how large the database is. It is safe to cancel at any time, because xtrabackup does not modify the database.

      "},{"location":"create-full-backup.html#next-step","title":"Next step","text":"

      Prepare the backup

      "},{"location":"create-full-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"create-gtid-replica.html","title":"How to create a new (or repair a broken) GTID-based replica","text":"

      Percona XtraBackup automatically stores the GTID value in the xtrabackup_binlog_info when doing the backup of MySQL and Percona Server for MySQL 8.1 with the GTID mode enabled. This information can be used to create a new (or repair a broken) GTID-based replica.

      "},{"location":"create-gtid-replica.html#1-take-a-backup-from-any-server-on-the-replication-environment-source-or-replica","title":"1. Take a backup from any server on the replication environment, source or replica","text":"

      The following command takes a backup and saves it in the /data/backups/$TIMESTAMP folder:

      $ xtrabackup --backup --target-dir=/data/backups/\n

      In the destination folder, there will be a file with the name xtrabackup_binlog_info. This file contains both binary log coordinates and the GTID information.

      $ cat xtrabackup_binlog_info\n
      The result could look like this:

      Expected output 
      mysql-bin.000002    1232        c777888a-b6df-11e2-a604-080027635ef5:1-4\n

      That information is also printed by xtrabackup after taking the backup:

      Expected output 
      xtrabackup: MySQL binlog position: filename 'mysql-bin.000002', position 1232, GTID of the last change 'c777888a-b6df-11e2-a604-080027635ef5:1-4'\n
      "},{"location":"create-gtid-replica.html#2-prepare-the-backup","title":"2. Prepare the backup","text":"

      The backup will be prepared with the following command on the Source:

      $ xtrabackup --prepare --target-dir=/data/backup\n

      You need to select the path where your snapshot has been taken, for example /data/backups/2023-05-07_08-33-33. If everything is ok you should get the same OK message. Now, the transaction logs are applied to the data files, and new ones are created: your data files are ready to be used by the MySQL server.

      "},{"location":"create-gtid-replica.html#3-move-the-backup-to-the-destination-server","title":"3. Move the backup to the destination server","text":"

      Use rsync or scp to copy the data to the destination server. If you are synchronizing the data directly to the already running replica\u2019s data directory it is advised to stop the MySQL server there.

      $ rsync -avprP -e ssh /path/to/backupdir/$TIMESTAMP NewSlave:/path/to/mysql/\n

      After you copy the data over, make sure MySQL has proper permissions to access them.

      $ chown mysql:mysql /path/to/mysql/datadir\n
      "},{"location":"create-gtid-replica.html#4-configure-and-start-replication","title":"4. Configure and start replication","text":"

      Set the gtid_purged variable to the GTID from xtrabackup_binlog_info. Then, update the information about the source node and, finally, start the replica.

      Note

      The example above is applicable to Percona XtraDB Cluster. The wsrep_on variable is set to 0 before resetting the source (RESET MASTER). The reason is that Percona XtraDB Cluster will not allow resetting the source if wsrep_on=1.

      # Using the mysql shell\n > SET SESSION wsrep_on = 0;\n > RESET MASTER;\n > SET SESSION wsrep_on = 1;\n > SET GLOBAL gtid_purged='<gtid_string_found_in_xtrabackup_binlog_info>';\n > CHANGE REPLICATION SOURCE TO\n             SOURCE_HOST=\"$masterip\",\n             SOURCE_USER=\"repl\",\n             SOURCE_PASSWORD=\"$slavepass\",\n             SOURCE_AUTO_POSITION = 1;\n > START REPLICA;\n
      "},{"location":"create-gtid-replica.html#5-check-the-replication-status","title":"5. Check the replication status","text":"

      The following command returns the replica status:

      mysql> SHOW REPLICA STATUS\\G\n
      The results should be similar to the following:

      Expected output 
      [..]\nSlave_IO_Running: Yes\nSlave_SQL_Running: Yes\n[...]\nRetrieved_Gtid_Set: c777888a-b6df-11e2-a604-080027635ef5:5\nExecuted_Gtid_Set: c777888a-b6df-11e2-a604-080027635ef5:1-5\n

      We can see that the replica has retrieved a new transaction with step 5, so transactions from 1 to 5 are already on the replica.

      We have created a new replica in our GTID based replication environment.

      "},{"location":"create-gtid-replica.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"create-incremental-backup.html","title":"Create an incremental backup","text":"

      xtrabackup supports incremental backups, which means that they can copy only all the data that has changed since the last backup.

      Note

      Incremental backups on the MyRocks storage engine do not determine if an earlier full backup or incremental backup contains the same files. Percona XtraBackup copies all the MyRocks files each time it takes a backup.

      You can perform many incremental backups between each full backup, so you can set up a backup process such as a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.

      Incremental backups work because each InnoDB page contains a log sequence number, or LSN. The LSN is the system version number for the entire database. Each page\u2019s LSN shows how recently it was changed.

      An incremental backup copies each page which LSN is newer than the previous incremental or full backup\u2019s LSN. An algorithm finds the pages that match the criteria. The algorithm reads the data pages and checks the page LSN.

      "},{"location":"create-incremental-backup.html#create-an-incremental-backup_1","title":"Create an incremental backup","text":"

      To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. Create the full backup with a following command:

      $ xtrabackup --backup --target-dir=/data/backups/base\n

      If you look at the xtrabackup_checkpoints file, you should see similar content depending on your LSN nuber:

      Expected output 
      backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 1626007\nlast_lsn = 1626007\ncompact = 0\nrecover_binlog_info = 1\n

      Now that you have a full backup, you can make an incremental backup based on it. Use the following command:

      $ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base\n

      The /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 1626007. If you examine the xtrabackup_checkpoints file in this directory, you should see similar content to the following:

      Expected output 
      backup_type = incremental\nfrom_lsn = 1626007\nto_lsn = 4124244\nlast_lsn = 4124244\ncompact = 0\nrecover_binlog_info = 1\n

      from_lsn is the starting LSN of the backup and for incremental it has to be the same as to_lsn (if it is the last checkpoint) of the previous/base backup.

      It\u2019s now possible to use this directory as the base for yet another incremental backup:

      $ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1\n

      This folder also contains the xtrabackup_checkpoints:

      Expected output 
      backup_type = incremental\nfrom_lsn = 4124244\nto_lsn = 6938371\nlast_lsn = 7110572\ncompact = 0\nrecover_binlog_info = 1\n

      Note

      In this case you can see that there is a difference between the to_lsn (last checkpoint LSN) and last_lsn (last copied LSN), this means that there was some traffic on the server during the backup process.

      "},{"location":"create-incremental-backup.html#next-step","title":"Next step","text":"

      Prepare the backup

      "},{"location":"create-incremental-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"create-individual-partition-backup.html","title":"Create an individual partitions backup","text":"

      Percona XtraBackup lets you back up individual partitions because partitions are regular tables with specially formatted names. The only requirement for this feature is to enable the innodb_file_per_table option on the server.

      There is one caveat about using this kind of backup: you can not copy back the prepared backup. Restoring partial backups should be done by importing the tables.

      There are three ways of specifying which part of the whole data will be backed up: regular expressions ( \u2013tables), enumerating the tables in a file (\u2013tables) or providing a list of databases (\u2013databases).

      The regular expression provided to this option will be matched against the fully qualified database name and table name, in the form of database-name.table-name.

      If the partition 0 is not backed up, Percona XtraBackup cannot generate a .cfg file. MySQL 8.0 stores the table metadata in partition 0.

      For example, this operation takes a back-up of the partition p4 from the table name located in the database imdb:

      $ xtrabackup --tables=^imdb[.]name#p#p4 --backup\n

      If partition 0 is not backed up, the following errors may occur:

      The error message 
      xtrabackup: export option not specified\nxtrabackup: error: cannot find dictionary record of table imdb/name#p#p4\n

      Note that this option is passed to xtrabackup --tables and is matched against each table of each database, the directories of each database will be created even if they are empty.

      "},{"location":"create-individual-partition-backup.html#next-step","title":"Next step","text":"

      Prepare an individual partitions backup

      "},{"location":"create-individual-partition-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"create-partial-backup.html","title":"Create a partial backup","text":"

      xtrabackup supports taking partial backups when the innodb_file_per_table option is enabled.

      Warning

      Do not copy back the prepared backup.

      Restoring partial backups should be done by importing the tables. We do not by using the \u2013copy-back option. This operation may lead to database inconsistencies.

      We do not recommend running incremental backups after taking a partial backup.

      The xtrabackup binary fails if you delete any of the matched or listed tables during the backup.

      There are multiple ways of specifying which part of the whole data is backed up:

      • Use the --tables option to list the table names

      • Use the --tables-file option to list the tables in a file

      • Use the --databases option to list the databases

      • Use the --databases-file option to list the databases

      The following examples assume a database named test that contains tables named t1 and t2.

      \u2013-tables option-\u2013tables-file option--databases option--databases-file option

      The first method involves the xtrabackup \u2013tables option. The option\u2019s value is a regular expression that is matched against the fully-qualified database name and table name using the databasename.tablename format.

      To back up only tables in the test database, use the following command:

      $ xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \\\n--tables=\"^test[.].*\"\n

      To back up only the test.t1 table, use the following command:

      $ xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \\\n--tables=\"^test[.]t1\"\n

      The --tables-file option specifies a file that can contain multiple table names, one table name per line in the file. Only the tables named in the file will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching. The table names must be fully-qualified in databasename.tablename format.

      $ echo \"mydatabase.mytable\" > /tmp/tables.txt\n$ xtrabackup --backup --tables-file=/tmp/tables.txt\n

      The ` \u2013databases` option accepts a space-separated list of the databases and tables to backup in the databasename[.tablename] format. In addition to this list, make sure to specify the mysql, sys, and

      performance_schema databases. These databases are required when restoring the databases using xtrabackup \u2013copy-back.

      Note

      Tables processed during the \u2013prepare step may also be added to the backup even if they are not explicitly listed by the parameter if they were created after the backup started.

      $ xtrabackup --databases='mysql sys performance_schema test ...'\n

      The \u2013databases-file option specifies a file that can contain multiple databases and tables in the databasename[.tablename] format, one element name per line in the file. Names are matched exactly, case-sensitive, with no pattern or regular expression matching.

      Note

      Tables processed during the \u2013prepare step may also be added to the backup even if they are not explicitly listed by the parameter if they were created after the backup started.

      "},{"location":"create-partial-backup.html#next-step","title":"Next step","text":"

      Prepare the backup

      "},{"location":"create-partial-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"dictionary-cache.html","title":"Dictionary cache","text":"

      Percona XtraBackup is based on how [crash recovery] works. Percona XtraBackup copies the InnoDB data files, which results in data that is internally inconsistent; then the prepare phase performs crash recovery on the files to make a consistent, usable database again.

      The --prepare phase has the following operations:

      • Applies the redo log
      • Applies the undo log

      As a physical operation, the changes from the redo log modifications are applied to a page. In the redo log operation, there is no concept of row or transaction. The redo apply operation does not make the database consistent with a transaction. An uncommitted transaction can be flushed or written to the redo log by the server. Percona XtraBackup applies changes recorded in the redo log.

      Percona XtraBackup physically modifies a specific offset of a page within a tablespace (IBD file) when applying a redo log.

      As a logical operation, Percona XtraBackup applies the undo log on a specific row. When the redo log completes, XtraBackup uses the undo log to roll back changes from uncommitted transactions during the backup.

      "},{"location":"dictionary-cache.html#undo-log","title":"Undo log","text":"

      There are two types of undo log records:

      • INSERT
      • UPDATE

      An undo log record contains a table_id. Percona XtraBackup uses this table_id to find the table definition, and then uses that information to parse the records on an index page. The transaction rollback reads the undo log records and applies the changes.

      After initializing the data dictionary engine and the data dictionary cache, the storage engine can request the table_id and uses this ID to fetch the table schema. An index search tuple (key) is created from the table schema and key fields from the undo log record. The server finds the record using the search tuple (key) and performs the undo operation.

      For example, InnoDB uses the table_id (also known as the se_private_id) for a table definition. Percona XtraBackup does not behave like a server and does not have access to the data dictionary. XtraBackup initializes the InnoDB engine and uses the InnoDB table object (dict_table_t) when needed. XtraBackup relies on Serialized Dictionary Information (SDI) that is stored in the tablespace. SDI is a JSON representation of a table.

      In Percona XtraBackup 8.1.0-1, tables are loaded as evictable. XtraBackup scans the B-tree index of the data dictionary tables mysql.indexes and mysql.index_partitions to establish a relationship between the table_id and the tablespace(space_id). XtraBackup uses this relationship during transaction rollback. XtraBackup does not load user tables unless there is a transaction rollback on them.

      A background thread or a Percona XtraBackup main thread handles the cache eviction when the cache size limit is reached.

      This design provides the following benefits during the --prepare phase:

      • Uses less memory
      • Uses less IO
      • Improves the time taken in the --prepare phase
      • Completes successfully, even if the --prepare phase has a huge number of tables
      • Completes the Percona XtraDB Cluster SST process faster
      "},{"location":"dictionary-cache.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"docker.html","title":"Run a Docker container","text":"

      Docker allows you to run applications in a lightweight unit called a container.

      You can run Percona XtraBackup in a Docker container without installing the product. All required libraries are available in the container. Being a lightweight execution environment, Docker containers enable creating configurations where each program runs in a separate container. You may run Percona Server for MySQL in one container and Percona XtraBackup in another. Docker images offer a range of options.

      Create a Docker container based on a Docker image. Docker images for Percona XtraBackup are hosted publicly on Docker Hub.

      $ sudo docker create ... percona/percona-xtrabackup --name xtrabackup ...\n
      "},{"location":"docker.html#scope-of-this-section","title":"Scope of this section","text":"

      This section demonstrates how to back up data on a Percona Server for MySQL running in another Dockers container.

      "},{"location":"docker.html#1-install-docker","title":"1. Install Docker","text":"

      Your operating system may already provide a package for docker. However, the versions of Docker provided by your operating system are likely to be outdated.

      Use the installation instructions for your operating system available from the Docker site to set up the latest version of docker.

      "},{"location":"docker.html#2-connect-to-a-percona-server-for-mysql-container","title":"2. Connect to a Percona Server for MySQL container","text":"

      Percona XtraBackup works in combination with a database server. When running a Docker container for Percona XtraBackup, you can make backups for a database server either installed on the host machine or running in a separate Docker container.

      To set up a database server on a host machine or in Docker container, follow the documentation of the supported product that you intend to use with Percona XtraBackup.

      See also

      Docker volumes as container persistent data storage

      More information about containers

      $ sudo docker run -d --name percona-server-mysql \\\n-e MYSQL_ROOT_PASSWORD=root percona/percona-server:8.1\n

      As soon as Percona Server for MySQL runs, add some data to it. Now, you are ready to make backups with Percona XtraBackup.

      Important

      When running Percona XtraBackup from a container and connecting to a MySQLserver container, we recommend using the \u2013user root option in the Docker command.

      "},{"location":"docker.html#3-create-a-docker-container-from-percona-xtrabackup-image","title":"3. Create a Docker container from Percona XtraBackup image","text":"

      You can create a Docker container based on Percona XtraBackup image with either docker create or the docker run command. docker create creates a Docker container and makes it available for starting later.

      Docker downloads the Percona XtraBackup image from the Docker Hub. If it is not the first time you use the selected image, Docker uses the image available locally.

      $ sudo docker create --name percona-xtrabackup --volumes-from percona-server-mysql \\\npercona/percona-xtrabackup  \\\nxtrabackup --backup --datadir=/var/lib/mysql/ --target-dir=/backup \\\n--user=root --password=mysql\n

      With parameter name you give a meaningful name to your new Docker container so that you could easily locate it among your other containers.

      The volumes-from flag refers to Percona Server for MySQL and indicates that you intend to use the same data as the Percona Server for MySQL container.

      Run the container with exactly the same parameters that were used when the container was created:

      $ sudo docker start -ai percona-xtrabackup\n

      This command starts the percona-xtrabackup container, attaches to its input/output streams, and opens an interactive shell.

      The docker run is a shortcut command that creates a Docker container and then immediately runs it.

      $ sudo docker run --name percona-xtrabackup --volumes-from percona-server-mysql \\\npercona/percona-xtrabackup\nxtrabackup --backup --data-dir=/var/lib/mysql --target-dir=/backup --user=root --password=mysql\n
      "},{"location":"docker.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"encrypt-backups.html","title":"Encrypt backups","text":"

      Percona XtraBackup supports encrypting and decrypting local and streaming backups with xbstream option adding another layer of protection. The encryption is implemented using the libgcrypt library from GnuPG.

      "},{"location":"encrypt-backups.html#create-encrypted-backups","title":"Create encrypted backups","text":"

      To make an encrypted backup the following options need to be specified (options --encrypt-key and --encrypt-key-file are mutually exclusive, i.e. just one of them needs to be provided):

      • --encrypt

      • --encrypt-key

      • --encrypt-key-file

      Both the --encrypt-key option and --encrypt-key-file option can be used to specify the encryption key. An encryption key can be generated with a command like openssl rand -base64 24

      U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\n

      This value then can be used as the encryption key

      "},{"location":"encrypt-backups.html#the-encrypt-key-option","title":"The --encrypt-key option","text":"

      Example of the xtrabackup command using the --encrypt-key should look like this:

      $  xtrabackup --backup --encrypt=AES256 --encrypt-key=\"U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\" --target-dir=/data/backup\n
      "},{"location":"encrypt-backups.html#the-encrypt-key-file-option","title":"The --encrypt-key-file option","text":"

      Use the --encrypt-key-file option as follows:

      $ xtrabackup --backup --encrypt=AES256 --encrypt-key-file=/data/backups/keyfile --target-dir=/data/backup\n

      Note

      Depending on the text editor that you use to make the KEYFILE, the editor can automatically insert the CRLF (end of line) character. This will cause the key size to grow and thus making it invalid. The suggested way to create the file is by using the command line: echo -n \u201cU2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\u201d > /data/backups/keyfile.

      "},{"location":"encrypt-backups.html#optimize-the-encryption-process","title":"Optimize the encryption process","text":"

      Two new options are available for encrypted backups that can be used to speed up the encryption process. These are --encrypt-threads and --encrypt-chunk-size. By using the --encrypt-threads option multiple threads can be specified to be used for encryption in parallel. Option --encrypt-chunk-size can be used to specify the size (in bytes) of the working encryption buffer for each encryption thread (default is 64K).

      "},{"location":"encrypt-backups.html#decrypt-encrypted-backups","title":"Decrypt encrypted backups","text":"

      Backups can be decrypted with The xbcrypt binary. The following one-liner can be used to encrypt the whole folder:

      $ for i in `find . -iname \"*\\.xbcrypt\"`; do xbcrypt -d --encrypt-key-file=/root/secret_key --encrypt-algo=AES256 < $i > $(dirname $i)/$(basename $i .xbcrypt) && rm $i; done\n

      Percona XtraBackup --decrypt option has been implemented that can be used to decrypt the backups:

      $ xtrabackup --decrypt=AES256 --encrypt-key=\"U2FsdGVkX19VPN7VM+lwNI0fePhjgnhgqmDBqbF3Bvs=\" --target-dir=/data/backup/\n

      Percona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should remove the \\*.xbcrypt files.

      Note

      --parallel can be used with --decrypt option to decrypt multiple files simultaneously.

      When the files are decrypted, the backup can be prepared.

      "},{"location":"encrypt-backups.html#prepare-encrypted-backups","title":"Prepare encrypted backups","text":"

      After the backups have been decrypted, they can be prepared in the same way as the standard full backups with the --prepare option:

      $ xtrabackup --prepare --target-dir=/data/backup/\n
      "},{"location":"encrypt-backups.html#restore-encrypted-backups","title":"Restore encrypted backups","text":"

      xtrabackup offers the --copy-back option to restore a backup to the server\u2019s datadir:

      $ xtrabackup --copy-back --target-dir=/data/backup/\n

      It will copy all the data-related files back to the server\u2019s datadir, determined by the server\u2019s my.cnf configuration file. You should check the last line of the output for a success message:

      Expected output 
      150318 11:08:13  xtrabackup: completed OK!\n
      "},{"location":"encrypt-backups.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"encrypted-innodb-tablespace-backups.html","title":"Encrypted InnoDB tablespace backups","text":"

      InnoDB supports data encryption for InnoDB tables stored in file-per-table tablespaces. This feature provides an at-rest encryption for physical tablespace data files.

      For an authenticated user or an application to access an encrypted F tablespace, InnoDB uses the master encryption key to decrypt the tablespace key. The master encryption key is stored in a keyring.

      Percona XtraBackup supports the following keyring components and plugins:

      • keyring_vault component

      • keyring_file plugin

      • keyring_file component

      • Key Management Interoperability Protocol (KMIP)

      • Amazon Key Management Service (AWS KMS)

      These components are stored in the plugin directory.

      Note

      Enable only one keyring plugin or one keyring component at a time for each server instance. Enabling multiple keyring plugins or keyring components or mixing keyring plugins or keyring components is not supported and may result in data loss.

      "},{"location":"encrypted-innodb-tablespace-backups.html#use-the-keyring-vault-component","title":"Use the keyring vault component","text":"

      The keyring_vault component extends the server capabilities. The server uses a manifest to load the component and the component has its own configuration file.

      The following example is a global manifest file that does not use local manifests:

      {\n \"read_local_manifest\": false,\n \"components\": \"file:///component_keyring_vault\"\n}\n

      The following is an example of a global manifest file that points to a local manifest file:

      {\n \"read_local_manifest\": true\n}\n

      The following is an example of a local manifest file:

      {\n \"components\": \"file:///component_keyring_vault\"\n}\n

      The configuration settings are either in a global configuration file or a local configuration file.

      Example of a configuration file in JSON format 
      {\n \"timeout\": 15,\n \"vault_url\": \"https://vault.public.com:8202\",\n \"secret_mount_point\": \"secret\",\n \"secret_mount_point_version\": \"AUTO\",\n \"token\": \"58a20c08-8001-fd5f-5192-7498a48eaf20\",\n \"vault_ca\": \"/data/keyring_vault_confs/vault_ca.crt\"\n}\n

      Find more information on configuring the keyring_vault component in Use the keyring vault component.

      The component has no special requirements for backing up a database that contains encrypted InnoDB tablespaces.

      The following command creates a backup in the /data/backup directory:

      $ xtrabackup --backup --target-dir=/data/backup --user=root\n

      After xtrabackup completes the action, the following message confirms the action:

      Confirmation message 
      xtrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\n
      "},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring-vault-component","title":"Prepare the backup with the keyring vault component","text":"

      To prepare the backup, xtrabackup must access the keyring. xtrabackup does not communicate with the MySQL server or read the default my.cnf configuration file. Specify the keyring settings in the command line:

      $ xtrabackup --prepare --target-dir=/data/backup --component-keyring-config==/etc/vault.cnf\n

      After xtrabackup completes the action, the following message confirms the action:

      Confirmation message 
      InnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\n
      "},{"location":"encrypted-innodb-tablespace-backups.html#restore-the-backup","title":"Restore the backup","text":"

      As soon as the backup is prepared, you can restore it with the --copy-back option:

      $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--component-keyring-config=/etc/vault.cnf\n
      "},{"location":"encrypted-innodb-tablespace-backups.html#use-the-keyring-file-plugin","title":"Use the keyring file plugin","text":"

      Warning

      The keyring_file plugin should not be used for regulatory compliance.

      In order to back up and prepare a database containing encrypted InnoDB tablespaces, specify the path to a keyring file as the value of the --keyring-file-data option.

      $ xtrabackup --backup --target-dir=/data/backup/ --user=root \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      After xtrabackup takes the backup, the following message confirms the action:

      Confirmation message 
      xtrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\n

      Warning

      xtrabackup does not copy the keyring file into the backup directory. To prepare the backup, you must copy the keyring file manually.

      "},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring-file-plugin","title":"Prepare the backup with the keyring file plugin","text":"

      To prepare the backup specify the keyring-file-data.

      $ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      After xtrabackup takes the backup, the following message confirms the action:

      Confirmation message 
      InnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\n

      The backup is now prepared and can be restored with the --copy-back option. In case the keyring has been rotated, you must restore the keyring used when the backup was taken and prepared.

      "},{"location":"encrypted-innodb-tablespace-backups.html#use-the-keyring-file-component","title":"Use the keyring file component","text":"

      The keyring_file component is part of the component-based MySQL infrastructure which extends the server capabilities.

      The server uses a manifest to load the component and the component has its own configuration file. See the MySQL documentation on the component installation for more information.

      An example of a manifest and a configuration file follows:

      An example of ./bin/mysqld.my:

      {\n   \"components\": \"file://component_keyring_file\"\n}\n

      An example of /lib/plugin/component_keyring_file.cnf:

      {\n   \"path\": \"/var/lib/mysql-keyring/keyring_file\", \"read_only\": false\n}\n

      For more information, see Keyring Component Installation and Using the keyring_file File-Based Keyring Plugin.

      With the appropriate privilege, you can SELECT on the performance_schema.keyring_component_status table to view the attributes and status of the installed keyring component when in use.

      The component has no special requirements for backing up a database that contains encrypted InnoDB tablespaces.

      $ xtrabackup --backup --target-dir=/data/backup --user=root\n

      After xtrabackup completes the action, the following message confirms the action:

      Confirmation message 
      xtrabackup: Transaction log of lsn (5696709) to (5696718) was copied.\n160401 10:25:51 completed OK!\n

      Warning

      xtrabackup does not copy the keyring file into the backup directory. To prepare the backup, you must copy the keyring file manually.

      "},{"location":"encrypted-innodb-tablespace-backups.html#prepare-the-backup-with-the-keyring_file-component","title":"Prepare the backup with the keyring_file component","text":"

      xtrabackup reads the keyring_file component configuration from xtrabackup_component_keyring_file.cnf. You must specify the keyring_file data path if the --component-keyring-config is not located in the attribute PATH from the xtrabackup_component_keyring_file.cnf.

      The following is an example of adding the location for the \u2013component-keyring-config:

      $ xtrabackup --prepare --target-dir=/data/backup \\\n--component-keyring-config=/var/lib/mysql-keyring/keyring\n

      xtrabackup attempts to read xtrabackup_component_keyring_file.cnf. You can assign another keyring file component configuration by passing the --component-keyring-config option.

      After xtrabackup completes preparing the backup, the following message confirms the action:

      Confirmation message 
      InnoDB: Shutdown completed; log sequence number 5697064\n160401 10:34:28 completed OK!\n

      The backup is prepared. To restore the backup use the --copy-back option. If the keyring has been rotated, you must restore the specific keyring used to take and prepare the backup.

      "},{"location":"encrypted-innodb-tablespace-backups.html#incremental-encrypted-innodb-tablespace-backups-with-keyring-file-plugin","title":"Incremental encrypted InnoDB tablespace backups with keyring file plugin","text":"

      The process of taking incremental backups with InnoDB tablespace encryption is similar to taking the Incremental Backups with unencrypted tablespace.

      "},{"location":"encrypted-innodb-tablespace-backups.html#create-an-incremental-backup","title":"Create an incremental backup","text":"

      To make an incremental backup, begin with a full backup. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup\u2019s target directory. This file contains a line showing the to_lsn, which is the database\u2019s LSN at the end of the backup. First you need to create a fullbackup with the following command:

      $ xtrabackup --backup --target-dir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      In order to prepare the backup, you must make a copy of the keyring file yourself. xtrabackup does not copy the keyring file into the backup directory. Restoring the backup after the keyring has been changed causes errors like ERROR 3185 (HY000): Can't find master key from keyring, please check keyring plugin is loaded. when the restore process tries accessing an encrypted table.

      If you look at the xtrabackup_checkpoints file, you should see the output similar to the following:

      Expected output 
      backup_type = full-backuped\nfrom_lsn = 0\nto_lsn = 7666625\nlast_lsn = 7666634\ncompact = 0\nrecover_binlog_info = 1\n

      Now that you have a full backup, you can make an incremental backup based on it. Use a command such as the following:

      $ xtrabackup --backup --target-dir=/data/backups/inc1 \\\n--incremental-basedir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      To prepare the backup, you must copy the keyring file manually. xtrabackup does not copy the keyring file into the backup directory.

      If the keyring has not been rotated you can use the same as the one you\u2019ve backed-up with the base backup. If the keyring has been rotated, or you have upgraded the plugin to a component, you must back up the keyring file. Otherwise, you cannot prepare the backup.

      The /data/backups/inc1/ directory should now contain delta files, such as ibdata1.delta and test/table1.ibd.delta. These represent the changes since the LSN 7666625. If you examine the xtrabackup_checkpoints file in this directory, you should see the output similar to the following:

      Expected output 
      backup_type = incremental\nfrom_lsn = 7666625\nto_lsn = 8873920\nlast_lsn = 8873929\ncompact = 0\nrecover_binlog_info = 1\n

      Use this directory as the base for yet another incremental backup:

      $ xtrabackup --backup --target-dir=/data/backups/inc2 \\\n--incremental-basedir=/data/backups/inc1 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
      "},{"location":"encrypted-innodb-tablespace-backups.html#prepare-incremental-backups","title":"Prepare incremental backups","text":"

      The --prepare step for incremental backups is not the same as for normal backups. In normal backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing a backup, because transactions that were uncommitted at the time of your backup may be in progress, and it\u2019s likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.

      If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups are useless. After transactions have been rolled back, further incremental backups cannot be applied.

      Beginning with the full backup you created, you can prepare it and then apply the incremental differences to it. Recall that you have the following backups:

      /data/backups/base\n/data/backups/inc1\n/data/backups/inc2\n

      To prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:

      $ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
      Expected output 
      InnoDB: Shutdown completed; log sequence number 7666643\nInnoDB: Number of pools: 1\n160401 12:31:11 completed OK!\n

      To apply the first incremental backup to the full backup, you should use the following command:

      $ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      The backup should be prepared with the keyring file and type that was used when backup was being taken. This means that if the keyring has been rotated, or you have upgraded from a plugin to a component between the base and incremental backup that you must use the keyring that was in use when the first incremental backup has been taken.

      Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:

      $ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2 \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
      Use --apply-log-only when merging all incremental backups except the last one. That\u2019s why the previous line does not contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase.

      The backup is now prepared and can be restored with --copy-back option. In case the keyring has been rotated you\u2019ll need to restore the keyring which was used to take and prepare the backup.

      "},{"location":"encrypted-innodb-tablespace-backups.html#restore-a-backup-when-the-keyring-is-not-available","title":"Restore a backup when the keyring is not available","text":"

      While the described restore method works, this method requires access to the same keyring that the server is using. It may not be possible if the backup is prepared on a different server or at a much later time, when keys in the keyring are purged, or, in the case of a malfunction, when the keyring vault server is not available at all.

      The --transition-key=<passphrase> option should be used to make it possible for xtrabackup to process the backup without access to the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and will use it to encrypt tablespace keys of tablespaces that are being backed up.

      "},{"location":"encrypted-innodb-tablespace-backups.html#create-a-backup-with-a-passphrase","title":"Create a backup with a passphrase","text":"

      The following example illustrates how the backup can be created in this case:

      $ xtrabackup --backup --user=root -p --target-dir=/data/backup \\\n--transition-key=MySecretKey\n

      If --transition-key is specified without a value, xtrabackup will ask for it.

      xtrabackup scrapes --transition-key so that its value is not visible in the ps command output.

      "},{"location":"encrypted-innodb-tablespace-backups.html#prepare-a-backup-with-a-passphrase","title":"Prepare a backup with a passphrase","text":"

      The same passphrase should be specified for the prepare command:

      $ xtrabackup --prepare --target-dir=/data/backup \\\n--transition-key=MySecretKey\n

      There are no --keyring-vault...,``\u2013keyring-file\u2026``, or --component-keyring-config options here, because xtrabackup does not talk to the keyring in this case.

      "},{"location":"encrypted-innodb-tablespace-backups.html#restore-a-backup-with-a-generated-key","title":"Restore a backup with a generated key","text":"

      When restoring a backup you need to generate a new master key.

      The example for keyring_file plugin:

      $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      The example for keyring_file component:

      $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--component-keyring-config=/var/lib/mysql-keyring/keyring\n

      The example for keyring_vault component:

      $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--transition-key=MySecretKey --generate-new-master-key \\\n--component-keyring-config=/etc/vault.cnf\n

      xtrabackup generates a new master key, stores it in the target keyring vault server and re-encrypts the tablespace keys using this key.

      "},{"location":"encrypted-innodb-tablespace-backups.html#make-a-backup-with-a-stored-transition-key","title":"Make a backup with a stored transition key","text":"

      Finally, there is an option to store a transition key in the keyring. In this case, xtrabackup will need to access the same keyring file or vault server during prepare and copy-back but does not depend on whether the server keys have been purged.

      In this scenario, the three stages of the backup process look as follows.

      • Back up
      $ xtrabackup --backup --user=root -p --target-dir=/data/backup \\\n--generate-transition-key\n

      • Prepare

        • keyring_file plugin variant:
        $ xtrabackup --prepare --target-dir=/data/backup \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n
        • keyring_file component variant:
        $ xtrabackup --prepare --target-dir=/data/backup \\\n--component-keyring-config=/var/lib/mysql-keyring/keyring\n
        • keyring_vault component variant:
        $ xtrabackup --prepare --target-dir=/data/backup \\\n--component-keyring-config=/etc/vault.cnf\n

      • Copy-back

        • keyring_file plugin variant:
        $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --keyring-file-data=/var/lib/mysql-keyring/keyring\n
        • keyring_file component variant:
        $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --component-keyring-config=/var/lib/mysql-keyring/keyring\n
        • keyring_vault component variant:
        $ xtrabackup --copy-back --target-dir=/data/backup --datadir=/data/mysql \\\n--generate-new-master-key --component-keyring-config=/etc/vault.cnf\n
      "},{"location":"encrypted-innodb-tablespace-backups.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"faq.html","title":"Faq","text":""},{"location":"faq.html#frequently-asked-questions","title":"Frequently asked questions","text":""},{"location":"faq.html#does-percona-xtrabackup-81-support-making-backups-of-databases-in-versions-prior-to-81","title":"Does Percona XtraBackup 8.1 support making backups of databases in versions prior to 8.1?","text":"

      Percona XtraBackup 8.1 does not support making backups of databases created in versions prior to 8.1 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster.

      "},{"location":"faq.html#are-you-aware-of-any-web-based-backup-management-tools-commercial-or-not-built-around-percona-xtrabackup","title":"Are you aware of any web-based backup management tools (commercial or not) built around Percona XtraBackup*?","text":"

      ZRM Community is a community tool that uses Percona XtraBackup for Non-Blocking Backups:

      \u201cZRM provides support for non-blocking backups of MySQL using Percona XtraBackup. ZRM with \\Percona XtraBackup provides resource utilization management by providing throttling based on the number of IO operations per second. Percona XtraBackup based backups also allow for table level recovery even though the backup was done at the database level (needs the recovery database server to be Percona Server for MySQL with XtraDB).\u201d*

      "},{"location":"faq.html#xtrabackup-binary-fails-with-a-floating-point-exception","title":"xtrabackup binary fails with a floating point exception","text":"

      In most of the cases this is due to not having installed the required libraries (and version) by xtrabackup. Installing the GCC suite with the supporting libraries and recompiling xtrabackup solves the issue. See Compiling and Installing from Source Code for instructions on the procedure.

      "},{"location":"faq.html#how-xtrabackup-handles-the-ibdataib_log-files-on-restore-if-they-arent-in-mysql-datadir","title":"How xtrabackup handles the ibdata/ib_log files on restore if they aren\u2019t in mysql datadir?","text":"

      In case the ibdata and ib_log files are located in different directories outside the datadir, you will have to put them in their proper place after the logs have been applied.

      "},{"location":"faq.html#backup-fails-with-error-24-too-many-open-files","title":"Backup fails with Error 24: \u2018Too many open files\u2019","text":"

      This usually happens when database being backed up contains large amount of files and Percona XtraBackup can\u2019t open all of them to create a successful backup. In order to avoid this error the operating system should be configured appropriately so that Percona XtraBackup can open all its files. On Linux, this can be done with the ulimit command for specific backup session or by editing the /etc/security/limits.conf to change it globally (NOTE: the maximum possible value that can be set up is 1048576 which is a hard-coded constant in the Linux kernel).

      "},{"location":"faq.html#how-to-deal-with-skipping-of-redo-logs-for-ddl-operations","title":"How to deal with skipping of redo logs for DDL operations?","text":"

      To prevent creating corrupted backups when running DDL operations, Percona XtraBackup aborts if it detects that redo logging is disabled. In this case, the following error is printed:

      [FATAL] InnoDB: An optimized (without redo logging) DDL operation has been performed. All modified pages may not have been flushed to the disk yet.\nPercona XtraBackup will not be able to take a consistent backup. Retry the backup operation.\n

      Note

      • Redo logging is disabled during a sorted index build. To avoid this error, Percona XtraBackup can use metadata locks on tables while they are copied:

      • To block all DDL operations, use the --lock-ddl option that issues LOCK TABLES FOR BACKUP.

      "},{"location":"faq.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"features.html","title":"Percona XtraBackup features","text":"

      The following is a short list of the Percona XtraBackup features:

      • Creates hot InnoDB backups without pausing your database

      • Makes incremental backups of MySQL

      • Streams compressed MySQL backups to another server

      • Moves tables between MySQL servers on-line

      • Creates new MySQL replication replicas easily

      • Backs up MySQL without adding load to the server

      • Performs throttling based on the number of IO operations per second

      • Skips secondary index pages and recreates them when a compact backup is prepared

      • Exports individual tables from a full InnoDB backup

      Percona XtraBackup automatically uses backup locks, a lightweight alternative to FLUSH TABLES WITH READ LOCK available in Percona Server, to copy non-InnoDB data. This operation avoids blocking DML queries that modify InnoDB tables.

      See also

      How Percona XtraBackup works

      "},{"location":"features.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"flush-tables-with-read-lock.html","title":"FLUSH TABLES WITH READ LOCK option","text":"

      The FLUSH TABLES WITH READ LOCK option does the following with a global read lock:

      • Closes all open tables

      • Locks all tables for all databases

      Release the lock with UNLOCK TABLES.

      Note

      FLUSH TABLES WITH READ LOCK does not prevent inserting rows into the log tables.

      To ensure consistent backups, use the FLUSH TABLES WITH READ LOCK option before taking a non-InnoDB file backup. The option does not affect long-running queries.

      Enabling FLUSH TABLES WITH READ LOCK when the server has long-running queries can leave the server in a read-only mode until the queries finish. If the server is in either the Waiting for table flush or the Waiting for master to send event state, stopping the FLUSH TABLES WITH READ LOCK operation does not help. Stop any long-running queries to return to normal operation.

      To prevent the server staying in a read-only mode until the queries finish, xtrabackup does the following:

      • checks if any queries run longer than specified in --ftwrl-wait-threshold. If xtrabackup finds such queries, xtrabackup waits for one second and checks again. If xtrabackup waits longer than specified in --ftwrl-wait-timeout, the backup is aborted. As soon as xtrabackup finds no queries running longer than specified in --ftwrl-wait-threshold, xtrabackup issues the global lock.

      • kills all queries or only the SELECT queries which prevent the global lock from being acquired.

      Note

      All operations described in this section have no effect when Backup locks are used.

      Percona XtraBackup uses Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. This operation automatically copies non-InnoDB data and avoids blocking DML queries that modify InnoDB tables.

      "},{"location":"flush-tables-with-read-lock.html#wait-for-queries-to-finish","title":"Wait for queries to finish","text":"

      You should issue a global lock when no long queries are running. Waiting to issue the global lock for extended period of time is not a good method. The wait can extend the time needed for backup to take place. The \u2013ftwrl-wait-timeout option can limit the waiting time. If it cannot issue the lock during this time, xtrabackup stops the option, exits with an error message, and backup is not be taken.

      The option\u2019s default value is zero (0), which turns off the option.

      Another possibility is to specify the type of query to wait on. In this case --ftwrl-wait-query-type.

      The possible values are all and update. When all is used xtrabackup will wait for all long running queries (execution time longer than allowed by --ftwrl-wait-threshold) to finish before running the FLUSH TABLES WITH READ LOCK. When update is used xtrabackup will wait on UPDATE/ALTER/REPLACE/INSERT queries to finish.

      The time needed for a specific query to complete is hard to predict. We assume that the long-running queries will not finish in a timely manner. Other queries which run for a short time finish quickly. xtrabackup uses the value of \u2013ftwrl-wait-threshold option to specify the long-running queries and will block a global lock. To use this option xtrabackup user should havePROCESSandCONNECTION_ADMIN` privileges.

      "},{"location":"flush-tables-with-read-lock.html#kill-the-blocking-queries","title":"Kill the blocking queries","text":"

      The second option is to kill all the queries which prevent from acquiring the global lock. In this case, all queries which run longer than FLUSH TABLES WITH READ LOCK are potential blockers. Although all queries can be killed, additional time can be specified for the short running queries to finish using the --kill-long-queries-timeout option. This option specifies a query time limit. After the specified time is reached, the server kills the query. The default value is zero, which turns this feature off.

      The --kill-long-query-type option can be used to specify all or only SELECT queries that are preventing global lock from being acquired. To use this option xtrabackup user should have PROCESS and CONNECTION_ADMIN privileges.

      "},{"location":"flush-tables-with-read-lock.html#options-summary","title":"Options summary","text":"

      • --ftwrl-wait-timeout (seconds) - how long to wait for a good moment. Default is 0, not to wait.

      • --ftwrl-wait-query-type - which long queries should be finished before FLUSH TABLES WITH READ LOCK is run. Default is all.

      • --ftwrl-wait-threshold (seconds) - how long query should be running before we consider it long running and potential blocker of global lock.

      • --kill-long-queries-timeout (seconds) - how many time we give for queries to complete after FLUSH TABLES WITH READ LOCK is issued before start to kill. Default if 0, not to kill.

      • --kill-long-query-type - which queries should be killed once kill-long-queries-timeout has expired.

      "},{"location":"flush-tables-with-read-lock.html#example","title":"Example","text":"

      Running the xtrabackup with the following options will cause xtrabackup to spend no longer than 3 minutes waiting for all queries older than 40 seconds to complete.

      $  xtrabackup --backup --ftwrl-wait-threshold=40 \\\n--ftwrl-wait-query-type=all --ftwrl-wait-timeout=180 \\\n--kill-long-queries-timeout=20 --kill-long-query-type=all \\\n--target-dir=/data/backups/\n

      After FLUSH TABLES WITH READ LOCK is issued, xtrabackup will wait for 20 seconds for lock to be acquired. If lock is still not acquired after 20 seconds, it will kill all queries which are running longer that the FLUSH TABLES WITH READ LOCK.

      "},{"location":"flush-tables-with-read-lock.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"glossary.html","title":"Glossary","text":""},{"location":"glossary.html#csm","title":".CSM","text":"

      Each table with the CSV Storage Engine has .CSM file which contains the metadata of it.

      "},{"location":"glossary.html#csv","title":".CSV","text":"

      Each table with the CSV Storage engine has .CSV file which contains the data of it (which is a standard Comma Separated Value file).

      "},{"location":"glossary.html#exp","title":".exp","text":"

      Files with the .exp extension are created by Percona XtraBackup per each InnoDB tablespace when the --export option is used on prepare. See restore individual tables.

      "},{"location":"glossary.html#frm","title":".frm","text":"

      For each table, the server will create a file with the .frm extension containing the table definition (for all storage engines).

      "},{"location":"glossary.html#general-availability-ga","title":"General availability (GA)","text":"

      A finalized version of the product which is made available to the general public. It is the final stage in the software release cycle.

      "},{"location":"glossary.html#ibd","title":".ibd","text":"

      On a multiple tablespace setup ([innodb_file_per_table] enabled), MySQL will store each newly created table on a file with a .ibd extension.

      "},{"location":"glossary.html#mrg","title":".MRG","text":"

      Each table using the MERGE storage engine, besides of a .frm file, will have .MRG file containing the names of the MyISAM tables associated with it.

      "},{"location":"glossary.html#myd","title":".MYD","text":"

      Each MyISAM table has .MYD (MYData) file which contains the data on it.

      "},{"location":"glossary.html#myi","title":".MYI","text":"

      Each MyISAM table has .MYI (MYIndex) file which contains the table\u2019s indexes.

      "},{"location":"glossary.html#opt","title":".opt","text":"

      MySQL stores options of a database (like charset) in a file with a .opt extension in the database directory.

      "},{"location":"glossary.html#par","title":".par","text":"

      Each partitioned table has .par file which contains metadata about the partitions.

      "},{"location":"glossary.html#trg","title":".TRG","text":"

      The file contains the triggers associated with a table, for example, \\mytable.TRG. With the .TRN file, they represent all the trigger definitions.

      "},{"location":"glossary.html#trn","title":".TRN","text":"

      The file contains the names of triggers that are associated with a table, for example, \\mytable.TRN. With the .TRG file, they represent all the trigger definitions.

      "},{"location":"glossary.html#backup","title":"backup","text":"

      The process of copying data or tables to be stored in a different location.

      "},{"location":"glossary.html#compression","title":"compression","text":"

      The method that produces backups in a reduced size.

      "},{"location":"glossary.html#configuration-file","title":"configuration file","text":"

      The file that contains the server startup options.

      "},{"location":"glossary.html#crash","title":"crash","text":"

      An unexpected shutdown which does not allow the normal server shutdown cleanup activities.

      "},{"location":"glossary.html#crash-recovery","title":"crash recovery","text":"

      The actions that occur when MySQL is restarted after a crash.

      "},{"location":"glossary.html#data-dictionary","title":"data dictionary","text":"

      The metadata for the tables, indexes, and table columns stored in the InnoDB system tablespace.

      "},{"location":"glossary.html#datadir","title":"datadir","text":"

      The directory in which the database server stores its data files. Most Linux distribution use /var/lib/mysql by default.

      "},{"location":"glossary.html#full-backup","title":"full backup","text":"

      A backup that contains the complete source data from an instance.

      "},{"location":"glossary.html#ibdata","title":"ibdata","text":"

      The default prefix for tablespace files. For example, ibdata1 is a 10MB auto-extensible file that MySQL creates for a shared tablespace by default.

      "},{"location":"glossary.html#incremental-backup","title":"incremental backup","text":"

      A backup stores data from a specific point in time.

      "},{"location":"glossary.html#innodb","title":"InnoDB","text":"

      Storage engine which provides ACID-compliant transactions and foreign key support, among others improvements over MyISAM. It is the default engine for MySQL 8.1.

      "},{"location":"glossary.html#innodb_buffer_pool_size","title":"innodb_buffer_pool_size","text":"

      The size in bytes of the memory buffer to cache data and indexes of InnoDB\u2019s tables. This aims to reduce disk access to provide better performance.

      [mysqld] innodb_buffer_pool_size=8MB

      "},{"location":"glossary.html#innodb_data_home_dir","title":"innodb_data_home_dir","text":"

      The directory (relative to datadir) where the database server stores the files in a shared tablespace setup. This option does not affect the location of innodb\\_file\\_per\\_table. For example:

      [mysqld] innodb_data_home_dir = ./

      "},{"location":"glossary.html#innodb_data_file_path","title":"innodb_data_file_path","text":"

      Specifies the names, sizes and location of shared tablespace files:

      [mysqld] innodb_data_file_path=ibdata1:50M;ibdata2:50M:autoextend

      "},{"location":"glossary.html#innodb_file_per_table","title":"innodb_file_per_table","text":"

      By default, InnoDB creates tables and indexes in a file-per-tablespace. If the innodb_file_per_table variable is disabled, you can enable the variable in your configuration file:

      [mysqld] innodb_file_per_table or start the server with --innodb_file_per_table.

      "},{"location":"glossary.html#innodb_log_group_home_dir","title":"innodb_log_group_home_dir","text":"

      Specifies the location of the InnoDB log files:

      [mysqld] innodb_log_group_home=/var/lib/mysql

      "},{"location":"glossary.html#logical-backup","title":"logical backup","text":"

      A backup which contains a set of SQL statements. The statements can be used to recreate the databases.

      "},{"location":"glossary.html#lsn","title":"LSN","text":"

      Each InnoDB page contains a log sequence number(LSN). The LSN is the system version number for the database. Each page\u2019s LSN shows how recently it was changed.

      "},{"location":"glossary.html#mycnf","title":"my.cnf","text":"

      The database server\u2019s main configuration file. Most Linux distributions place it as /etc/mysql/my.cnf or /etc/my.cnf, but the location and name depends on the particular installation. Note that this method is not the only way of configuring the server, some systems rely on the command options.

      "},{"location":"glossary.html#myisam","title":"MyISAM","text":"

      The MySQL default storage engine until version 5.5. It doesn\u2019t fully support transactions but in some scenarios may be faster than InnoDB. Each table is stored on disk in 3 files: .frm, .MYD, .MYI.

      "},{"location":"glossary.html#physical-backup","title":"physical backup","text":"

      A backup that copies the data files.

      "},{"location":"glossary.html#point-in-time-recovery","title":"point in time recovery","text":"

      This method restores the data into the state it was at any selected point of time.

      "},{"location":"glossary.html#prepared-backup","title":"prepared backup","text":"

      A consistent set of backup data that is ready to be restored.

      "},{"location":"glossary.html#restore","title":"restore","text":"

      Copies the database backups taken using the backup command to the original location or a different location. A restore returns data that has been either lost, corrupted, or stolen to the original condition at a specific point in time.

      "},{"location":"glossary.html#tech-preview","title":"Tech preview","text":"

      A tech preview item can be a feature, a variable, or a value within a variable. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment and also use an alternative backup method for redundancy. A tech preview item is included in a release for users to provide feedback. The item is either updated and released as general availability(GA) or removed if not useful. The functionality can change from tech preview to GA.

      "},{"location":"glossary.html#xbcrypt","title":"xbcrypt","text":"

      To support the encryption and the decryption of the backups. This utility has been modeled after the xbstream binary to perform encryption and decryption outside Percona XtraBackup.

      "},{"location":"glossary.html#xbstream","title":"xbstream","text":"

      To support simultaneous compression and streaming, Percona XtraBackup uses the xbstream format. For more information see xbstream

      "},{"location":"glossary.html#xtradb","title":"XtraDB","text":"

      Percona XtraDB is an enhanced version of the InnoDB storage engine, designed to better scale on modern hardware. Percona XtraDB includes features which are useful in a high performance environment. It is fully backward-compatible, and is a drop-in replacement for the standard InnoDB storage engine. For more information, see The Percona XtraDB Storage Engine.

      "},{"location":"glossary.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

      ZSTD is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios.

      "},{"location":"glossary.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"how-xtrabackup-works.html","title":"How Percona XtraBackup works","text":"

      Percona XtraBackup is based on InnoDB\u2019s crash-recovery functionality. It copies your InnoDB data files, which results in data that is internally inconsistent; but then it performs crash recovery on the files to make them a consistent, usable database again.

      This works because InnoDB maintains a redo log, also called the transaction log. This contains a record of every change to InnoDB data. When InnoDB starts, it inspects the data files and the transaction log, and performs two steps. It applies committed transaction log entries to the data files, and it performs an undo operation on any transactions that modified data but did not commit.

      The --register-redo-log-consumer parameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.

      Percona XtraBackup remembers the LSN when it starts, and then copies the data files. The operation takes time, and the files may change, then LSN reflects the state of the database at different points in time. Percona XtraBackup also runs a background process that watches the transaction log files, and copies any changes. Percona XtraBackup does this continually. The transaction logs are written in a round-robin fashion, and can be reused.

      Percona XtraBackup uses Backup locks

      where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK. MySQL 8.1 allows acquiring an instance level backup lock via the LOCK INSTANCE FOR BACKUP statement.

      Locking is only done for MyISAM and other non-InnoDB tables after Percona XtraBackup finishes backing up all InnoDB/XtraDB data and logs. Percona XtraBackup uses this automatically to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

      Important

      The BACKUP_ADMIN privilege is required to query the performance_schema_log_status for either LOCK INSTANCE FOR BACKUP or LOCK TABLES FOR BACKUP.

      xtrabackup tries to avoid backup locks and FLUSH TABLES WITH READ LOCK when the instance contains only InnoDB tables. In this case, xtrabackup obtains binary log coordinates from performance_schema.log_status. FLUSH TABLES WITH READ LOCK is still required in MySQL 8.1 when xtrabackup is started with the --slave-info. The log_status table in Percona Server for MySQL 8.1 is extended to include the relay log coordinates, so no locks are needed even with the --slave-info option.

      See also

      MySQL Documentation: LOCK INSTANCE FOR BACKUP

      When backup locks are supported by the server, xtrabackup first copies InnoDB data, runs the LOCK TABLES FOR BACKUP and then copies the MyISAM tables. Once this is done, the backup of the files will begin. It will backup .frm, .MRG, .MYD, .MYI, .CSM, .CSV, .sdi and .par files.

      After that xtrabackup will use LOCK BINLOG FOR BACKUP to block all operations that might change either binary log position or Exec_Master_Log_Pos or Exec_Gtid_Set (i.e. source binary log coordinates corresponding to the current SQL thread state on a replication replica) as reported by SHOW MASTER/SLAVE STATUS. xtrabackup will then finish copying the REDO log files and fetch the binary log coordinates. After this is completed xtrabackup will unlock the binary log and tables.

      Finally, the binary log position will be printed to STDERR and xtrabackup will exit returning 0 if all went OK.

      Note that the STDERR of xtrabackup is not written in any file. You will have to redirect it to a file, for example, xtrabackup OPTIONS 2> backupout.log.

      It will also create the following files in the directory of the backup.

      During the prepare phase, Percona XtraBackup performs crash recovery against the copied data files, using the copied transaction log file. After this is done, the database is ready to restore and use.

      The backed-up MyISAM and InnoDB tables will be eventually consistent with each other, because after the prepare (recovery) process, InnoDB\u2019s data is rolled forward to the point at which the backup completed, not rolled back to the point at which it started. This point in time matches where the FLUSH TABLES WITH READ LOCK was taken, so the MyISAM data and the prepared InnoDB data are in sync.

      The xtrabackup offers many features not mentioned in the preceding explanation. The functionality of each tool is explained in more detail further in this manual. In brief, though, the tools enable you to do operations such as streaming and incremental backups with various combinations of copying the data files, copying the log files, and applying the logs to the data.

      "},{"location":"how-xtrabackup-works.html#restoring-a-backup","title":"Restoring a backup","text":"

      To restore a backup with xtrabackup you can use the --copy-back or --move-back options.

      xtrabackup will read from the my.cnf the variables datadir, innodb_data_home_dir, innodb_data_file_path, innodb_log_group_home_dir and check that the directories exist.

      It will copy the MyISAM tables, indexes, etc. (.MRG, .MYD, .MYI, .CSM, .CSV, .sdi, and par files) first, InnoDB tables and indexes next and the log files at last. It will preserve file\u2019s attributes when copying them, you may have to change the files\u2019 ownership to mysql before starting the database server, as they will be owned by the user who created the backup.

      Alternatively, the --move-back option may be used to restore a backup. This option is similar to --copy-back with the only difference that instead of copying files it moves them to their target locations. As this option removes backup files, it must be used with caution. It is useful in cases when there is not enough free disk space to hold both data files and their backup copies.

      "},{"location":"how-xtrabackup-works.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"installation.html","title":"Install overview","text":"

      We recommend that you install Percona XtraBackup 8.1 from the official Percona software repositories using the appropriate package manager for your system:

      • Use an APT repo to install Percona XtraBackup

      • Use a YUM repo to install Percona XtraBackup

      "},{"location":"installation.html#installation-alternatives","title":"Installation alternatives","text":"

      Percona also provides the following methods:

      • Use DEB downloaded packages to install Percona XtraBackup

      • Use RPM downloaded packages to install Percona XtraBackup

      • Install Percona XtraBackup from a Binary Tarball with all the required files and binaries for a manual installation

      • Compile and install Percona XtraBackup from source code

      • Run Percona XtraBackup in a Docker container

      Before installing Percona XtraBackup with any of these methods, we recommend that you review the release notes and Server version and backup version comparison.

      "},{"location":"installation.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"limitations.html","title":"Limitations","text":"

      Percona XtraBackup 8.1 does not support making backups of databases created in versions prior to 8.1 of MySQL, Percona Server for MySQL or Percona XtraDB Cluster.

      "},{"location":"limitations.html#additional-information","title":"Additional information","text":"

      The InnoDB tables are locked while copying non-InnoDB data.

      "},{"location":"limitations.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"lock-options.html","title":"lock-ddl-per-table option improvements","text":"

      To block DDL statements on an instance, Percona Server implemented LOCK TABLES FOR BACKUP. Percona XtraBackup uses this lock for the duration of the backup. This lock does not affect DML statements.

      Percona XtraBackup has also implemented --lock-ddl-per-table, which blocks DDL statements by using metadata locks (MDL).

      The following procedures describe a simplified backup operation when using --lock-ddl-per-table:

      1. Parse and copy all redo logs after the checkpoint mark

      2. Fork a dedicated thread to continue following new redo log entries

      3. List the tablespaces required to copy

      4. Iterate through the list. The following steps occur with each listed tablespace:

      5. Query INFORMATION_SCHEMA.INNODB_TABLES to find which tables belong to the tablespace ID and take an MDL on the underlying table or tables in case there is a shared tablespace.

      6. Copy the tablespace .ibd files.

      The backup process may encounter a redo log event, generated by the bulk load operations, which notifies backup tools that data file changes have been omitted from the redo log. This event is an MLOG_INDEX_LOAD. If this event is found by the redo follow thread, the backup continues and assumes the backup is safe because the MDL protects tablespaces already copied and the MLOG_INDEX_LOAD event is for a tablespace that is not copied.

      These assumptions may not be correct and may lead to inconsistent backups.

      "},{"location":"lock-options.html#-lock-ddl-per-table-redesign","title":"--lock-ddl-per-table redesign","text":"

      The --lock-ddl-per-table option has been redesigned to minimize inconsistent backups.

      The following procedure reorders the steps:

      • The MDL lock acquired at the beginning of the backup

      • Scan the redo logs. An MLOG_INDEX_LOAD event may be recorded if a CREATE INDEX statement has occurred before the backup starts. At this time, the backup process is safe and can parse and accept the event.

      • After the first scan has completed, the follow redo log thread is initiated. This thread stops the backup process if an MLOG_INDEX_LOAD event is found.

      • Gather the tablespace list to copy

      • Copy the .ibd files.

      "},{"location":"lock-options.html#other-improvements","title":"Other improvements","text":"

      The following improvements have been added:

      • If the .ibd file belongs to a temporary table, the SELECT query is skipped.

      • For a FullText Index, an MDL is acquired on the base table.

      • A SELECT query that acquires an MDL does not retrieve any data.

      "},{"location":"lock-options.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"log-enhancements.html","title":"Improved log statements","text":"

      Percona XtraBackup is an open-source command-line utility. Command-line tools have limited interaction with the background operations and the logs provide the progress of an operation or more information about errors.

      The error logs did not have a standard structure and the log statements varied in the following ways:

      • The backup log statement header has the name of the module, xtrabackup, which generated the statement but no timestamp:

       $ xtrabackup: recognized client arguments: --parallel=4 --target-dir=/data/backups/ --backup=1\n
      The output should be similar to the following:

      Expected output 
      ./bin/xtrabackup version 8.1.0-1 based on MySQL server 8.1 Linux (x86_64) (revision id: b0f75188ca3)\n
      • The copy-back log statement has a timestamp but no module name. The timestamp is a mix of UTC and the local timezone.
      220322 19:05:13 [01] Copying undo_001 to /data/backups/undo_001\n
      • The following prepare log statements do not have header information, which makes diagnosing an issue more difficult.
      Completed space ID check of 1008 files.\nInitializing buffer pool, total size = 128.000000M, instances = 1, chunk size =128.000000M\nCompleted initialization of buffer pool\nIf the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().\n
      "},{"location":"log-enhancements.html#log-statement-structure","title":"Log statement structure","text":"

      The improved log structure is displayed in the backup, prepare, move-back/copy-back error logs.

      Each log statement has the following attributes:

      • Timestamp - a timestamp for when the event occurred in a UTC format.

      • Severity - the severity level of a statement indicates the importance of an event.

      • ID - this identifier is currently not used but may be used in future versions.

      • Context - the name of the module that issued the log statement, such as XtraBackup, InnoDB, or Server.

      • Message - a description of the event generated by the module.

      An example of a prepare log that is generated with the improved structure. The uniformity of the headers makes it easier to follow an operation\u2019s progress or review the log to diagnose issues.

      Expected output 
      2022-03-22T19:15:36.142247+05:30 0 [Note] [MY-011825] [Xtrabackup] This target seems to be not prepared yet.\n2022-03-22T19:15:36.142792+05:30 0 [Note] [MY-013251] [InnoDB] Number of pools: 1\n2022-03-22T19:15:36.149212+05:30 0 [Note] [MY-011825] [Xtrabackup] xtrabackup_logfile detected: size=8388608, start_lsn=(33311656)\n2022-03-22T19:15:36.149998+05:30 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration for recovery:\n2022-03-22T19:15:36.150023+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .\n2022-03-22T19:15:36.150036+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend\n2022-03-22T19:15:36.150078+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = .\n2022-03-22T19:15:36.150095+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 1\n2022-03-22T19:15:36.150111+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 8388608\n2022-03-22T19:15:36.151667+05:30 0 [Note] [MY-011825] [Xtrabackup] inititialize_service_handles suceeded\n2022-03-22T19:15:36.151903+05:30 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration for recovery:\n2022-03-22T19:15:36.151926+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .\n2022-03-22T19:15:36.151954+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend\n2022-03-22T19:15:36.151976+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = .\n2022-03-22T19:15:36.151991+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 1\n2022-03-22T19:15:36.152004+05:30 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 8388608\n2022-03-22T19:15:36.152021+05:30 0 [Note] [MY-011825] [Xtrabackup] Starting InnoDB instance for recovery.\n2022-03-22T19:15:36.152035+05:30 0 [Note] [MY-011825] [Xtrabackup] Using 104857600 bytes for buffer pool (set by --use-memory parameter)\n
      "},{"location":"log-enhancements.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"lru-dump-backup.html","title":"LRU dump backup","text":"

      Percona XtraBackup includes a saved buffer pool dump into a backup to enable reducing the warm up time. It restores the buffer pool state from ib_buffer_pool file after restart. Percona XtraBackup discovers ib_buffer_pool and backs it up automatically.

      If the buffer restore option is enabled in my.cnf, buffer pool will be in the warm state after backup is restored.

      Find the information on how to save and restore the buffer pool dump in Saving and Restoring the Buffer Pool State.

      "},{"location":"lru-dump-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"make-backup-in-replication-env.html","title":"Make backups in replication environments","text":"

      There are options specific to back up from a replication replica.

      "},{"location":"make-backup-in-replication-env.html#the-slave-info-option","title":"The --slave-info option","text":"

      This option is useful when backing up a replication replica server. It prints the binary log position and name of the source server. It also writes this information to the xtrabackup_slave_info file as a CHANGE MASTER statement.

      This option is useful for setting up a new replica for this source. You can start a replica server with this backup and issue the statement saved in the xtrabackup_slave_info file. More details of this procedure can be found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.

      "},{"location":"make-backup-in-replication-env.html#the-safe-slave-backup-option","title":"The --safe-slave-backup option","text":"

      In order to assure a consistent replication state, this option stops the replication SQL thread and waits to start backing up until Slave_open_temp_tables in SHOW STATUS is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds (defaults to 300 seconds). The replication SQL thread will be restarted when the backup finishes.

      Note

      Using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.

      Using this option is always recommended when taking backups from a replica server.

      Warning

      Make sure your replica is a true replica of the source before using it as a source for backup. A good tool to validate a replica is pt-table-checksum.

      "},{"location":"make-backup-in-replication-env.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"page-tracking.html","title":"Take an incremental backup using page tracking","text":"

      To create an incremental backup with page tracking, Percona XtraBackup uses the MySQL mysqlbackup component. This component provides a list of pages modified since the last backup, and Percona XtraBackup copies only those pages. This operation removes the need to scan the pages in the database. If the majority of pages have not been modified, the page tracking feature can improve the speed of incremental backups.

      "},{"location":"page-tracking.html#install-the-component","title":"Install the component","text":"

      To start using the page tracking functionality, do the following:

      1. Install the mysqlbackup component and enable it on the server:

        $ INSTALL COMPONENT \"file://component_mysqlbackup\";\n

      2. Check whether the mysqlbackup component is installed successfully:

        $ SELECT COUNT(1) FROM mysql.component WHERE component_urn='file://component_mysqlbackup';\n
      "},{"location":"page-tracking.html#use-page-tracking","title":"Use page tracking","text":"

      You can enable the page tracking functionality for the full and incremental backups with the --page-tracking option.

      The option has the following benefits:

      • Resets page tracking to the start of the backup. This reset allows the next incremental backup to use page tracking.

      • Allows the use of page tracking for an incremental backup if the page tracking data is available from the backup\u2019s start checkpoint LSN.

      Percona XtraBackup processes a list of all the tracked pages in memory. If Percona XtraBackup does not have enough available memory to process this list, the process throws an error and exits. For example, if an incremental backup uses 200GB, Percona XtraBackup can use an additional 100MB of memory to process and store the page tracking data.

      The examples of creating full and incremental backups using the --page-tracking option:

      Full backupIncremental backup 
      $ xtrabackup --backup --target-dir=$FULL_BACK --page-tracking\n
      $ xtrabackup --backup --target-dir=$INC_BACKUP  \n--incremental-basedir=$FULL_BACKUP --page-tracking\n

      After enabling the functionality, the next incremental backup finds changed pages using page tracking.

      The first full backup using page tracking, Percona XtraBackup may have a delay. The following is an example of the message:

      Expected output 
      xtrabackup: pagetracking: Sleeping for 1 second, waiting for checkpoint lsn 17852922 /\nto reach to page tracking start lsn 21353759\n

      Enable page tracking before creating the first backup to avoid this delay. This method ensures that the page tracking log sequence number (LSN) is higher than the checkpoint LSN of the server.

      "},{"location":"page-tracking.html#start-page-tracking-manually","title":"Start page tracking manually","text":"

      After the mysqlbackup component is loaded and active on the server, you can start page tracking manually with the following option:

      $ SELECT mysqlbackup_page_track_set(true);\n
      "},{"location":"page-tracking.html#check-the-lsn-value","title":"Check the LSN value","text":"

      Check the LSN value starting from which changed pages are tracked with the following option:

      $ SELECT mysqlbackup_page_track_get_start_lsn();\n
      "},{"location":"page-tracking.html#stop-page-tracking","title":"Stop page tracking","text":"

      To stop page tracking, use the following command:

      $ SELECT mysqlbackup_page_track_set(false);\n
      "},{"location":"page-tracking.html#purge-page-tracking-data","title":"Purge page tracking data","text":"

      When you start page tracking, it creates a file under the server\u2019s datadir to collect data about changed pages. This file grows until you stop the page tracking. If you stop the server and then restart it, page tracking creates a new file but also keeps the old one. The old file continues to grow until you stop the page tracking explicitly.

      If you purge the page tracking data, you should create a full backup afterward. To purge the page tracking data, do the following steps:

      $ SELECT mysqlbackup_page_track_set(false);\n$ SELECT mysqlbackup_page_track_purge_up_to(9223372036854775807);\n/* Specify the LSN up to which you want to purge page tracking data. /\n9223372036854775807 is the highest possible LSN which purges all page tracking files.*/\n$ SELECT mysqlbackup_page_track_set(true);\n
      "},{"location":"page-tracking.html#known-issue","title":"Known issue","text":"

      If the index is built in place using an exclusive algorithm and then is added to a table after the last LSN checkpoint, you may generate a bad incremental backup using page tracking. For more details see PS-8032.

      "},{"location":"page-tracking.html#uninstall-the-mysqlbackup-component","title":"Uninstall the mysqlbackup component","text":"

      To uninstall the mysqlbackup component, use the following statement:

      $ UNINSTALL COMPONENT \"file://component_mysqlbackup\"\n
      "},{"location":"page-tracking.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"permissions.html","title":"Permissions needed","text":"

      We will be referring to permissions to the ability of a user to access and perform changes on the relevant parts of the host\u2019s filesystem, starting/stopping services and installing software.

      By privileges, we refer to the abilities of a database user to perform different kinds of actions on the database server.

      There are many ways for checking the permission on a file or directory. For example, ls -ls /path/to/file or stat /path/to/file | grep Access will do the job:

      $ stat /etc/mysql | grep Access\n
      The result could look like this:

      Expected output 
      Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)\nAccess: 2011-05-12 21:19:07.129850437 -0300\n$ ls -ld /etc/mysql/my.cnf\n-rw-r--r-- 1 root root 4703 Apr  5 06:26 /etc/mysql/my.cnf\n

      As in this example, my.cnf is owned by root and not writable for anyone else. Assuming that you do not have root\u2018s password, you can check what permissions you have on these types of files with sudo -l:

      $ sudo -l\n
      The results could look like this:

      Expected output 
      Password:\nYou may run the following commands on this host:\n(root) /usr/bin/\n(root) NOPASSWD: /etc/init.d/mysqld\n(root) NOPASSWD: /bin/vi /etc/mysql/my.cnf\n(root) NOPASSWD: /usr/local/bin/top\n(root) NOPASSWD: /usr/bin/ls\n(root) /bin/tail\n

      Being able to execute with sudo scripts in /etc/init.d/, /etc/rc.d/ or /sbin/service is the ability to start and stop services.

      Also, If you can execute the package manager of your distribution, you can install or remove software with it. If not, having rwx permission over a directory will let you do a local installation of software by compiling it there. This is a typical situation in many hosting companies\u2019 services.

      There are other ways for managing permissions, such as using PolicyKit, Extended ACLs or SELinux, which may be preventing or allowing your access. You should check them in that case.

      See also

      Connection and privileges needed

      "},{"location":"permissions.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"point-in-time-recovery.html","title":"Point-in-time recovery","text":"

      Recovering up to particular moment in database\u2019s history can be done with xtrabackup and the binary logs of the server.

      Note that the binary log contains the operations that modified the database from a point in the past. You need a full datadir as a base, and then you can apply a series of operations from the binary log to make the data match what it was at the point in time you want.

      $ xtrabackup --backup --target-dir=/path/to/backup\n$ xtrabackup --prepare --target-dir=/path/to/backup\n

      For more details on these procedures, see Creating a backup and Preparing a backup.

      Now, suppose that some time has passed, and you want to restore the database to a certain point in the past, having in mind that there is the constraint of the point where the snapshot was taken.

      To find out what is the situation of binary logging in the server, execute the following queries:

      mysql> SHOW BINARY LOGS;\n
      Expected output 
      +------------------+-----------+\n| Log_name         | File_size |\n+------------------+-----------+\n| mysql-bin.000001 |       126 |\n| mysql-bin.000002 |      1306 |\n| mysql-bin.000003 |       126 |\n| mysql-bin.000004 |       497 |\n+------------------+-----------+\n

      and

      mysql> SHOW MASTER STATUS;\n
      Expected output 
      +------------------+----------+--------------+------------------+\n| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |\n+------------------+----------+--------------+------------------+\n| mysql-bin.000004 |      497 |              |                  |\n+------------------+----------+--------------+------------------+\n

      The first query will tell you which files contain the binary log and the second one which file is currently being used to record changes, and the current position within it. Those files are stored usually in the datadir (unless other location is specified when the server is started with the --log-bin= option).

      To find out the position of the snapshot taken, see the xtrabackup_binlog_info at the backup\u2019s directory:

      $ cat /path/to/backup/xtrabackup_binlog_info\n
      Expected output 
      mysql-bin.000003      57\n

      This will tell you which file was used at moment of the backup for the binary log and its position. That position will be the effective one when you restore the backup:

      $ xtrabackup --copy-back --target-dir=/path/to/backup\n

      As the restoration will not affect the binary log files (you may need to adjust file permissions, see Restoring a Backup), the next step is extracting the queries from the binary log with mysqlbinlog starting from the position of the snapshot and redirecting it to a file

      $ mysqlbinlog /path/to/datadir/mysql-bin.000003 /path/to/datadir/mysql-bin.000004 \\\n    --start-position=57 > mybinlog.sql\n

      Note that if you have multiple files for the binary log, as in the example, you have to extract the queries with one process, as shown above.

      Inspect the file with the queries to determine which position or date corresponds to the point-in-time wanted. Once determined, pipe it to the server. Assuming the point is 11-12-25 01:00:00:

      $ mysqlbinlog /path/to/datadir/mysql-bin.000003 /path/to/datadir/mysql-bin.000004 \\\n    --start-position=57 --stop-datetime=\"11-12-25 01:00:00\" | mysql -u root -p\n

      and the database will be rolled forward up to that Point-In-Time.

      "},{"location":"point-in-time-recovery.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"prepare-compressed-backup.html","title":"Prepare the backup","text":"

      Before you can prepare the backup you\u2019ll need to uncompress all the files. Percona XtraBackup has implemented --decompress option that can be used to decompress the backup.

      $ xtrabackup --decompress --target-dir=/data/compressed/\n

      Note

      --parallel can be used with --decompress option to decompress multiple files simultaneously.

      Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory you should use --remove-original option. Even if they\u2019re not removed these files will not be copied/moved over to the datadir if --copy-back or --move-back are used.

      When the files are uncompressed you can prepare the backup with the --prepare option:

      $ xtrabackup --prepare --target-dir=/data/compressed/\n
      Confirmation message 
      InnoDB: Starting shutdown...\nInnoDB: Shutdown completed; log sequence number 9293846\n170223 13:39:31 completed OK!\n

      Now the files in /data/compressed/ are ready to be used by the server.

      "},{"location":"prepare-compressed-backup.html#next-step","title":"Next step","text":"

      Restore the backup

      "},{"location":"prepare-compressed-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"prepare-full-backup.html","title":"Prepare a full backup","text":"

      After creating a backup with the --backup option, you need to prepare the backup and then restore it. Data files are not point-in-time consistent until they are prepared, because they were copied at different times as the program ran, and they might have been changed while this was happening.

      If you try to start InnoDB with these data files, it will detect corruption and stop working to avoid running on damaged data. The --prepare step makes the files perfectly consistent at a single instant in time, so you can run InnoDB on them.

      You can run the prepare operation on any machine; it does not need to be on the originating server or the server to which you intend to restore. You can copy the backup to a utility server and prepare it there.

      Note that Percona XtraBackup 8.1 can only prepare backups of MySQL 8.1 and Percona Server for MySQL 8.1 databases. Releases prior to 8.1 are not supported.

      During the prepare operation, xtrabackup boots up a kind of modified embedded InnoDB (the libraries xtrabackup was linked against). The modifications are necessary to disable InnoDB standard safety checks, such as complaining about the log file not being the right size. This warning is not appropriate for working with backups. These modifications are only for the xtrabackup binary; you do not need a modified InnoDB to use xtrabackup for your backups.

      The prepare step uses this \u201cembedded InnoDB\u201d to perform crash recovery on the copied data files, using the copied log file. The prepare step is very simple to use: you simply run xtrabackup with the --prepare option and tell it which directory to prepare, for example, to prepare the previously taken backup run:

      $ xtrabackup --prepare --target-dir=/data/backups/\n

      When this finishes, you should see an InnoDB shutdown with a message such as the following, where again the value of LSN will depend on your system:

      Expected output 
      InnoDB: Shutdown completed; log sequence number 137345046\n160906 11:21:01 completed OK!\n

      All following prepares will not change the already prepared data files, you\u2019ll see that output says:

      Expected output 
      xtrabackup: This target seems to be already prepared.\nxtrabackup: notice: xtrabackup_logfile was already used to '--prepare'.\n

      It is not recommended to interrupt xtrabackup process while preparing backup because it may cause data files corruption and backup will become unusable. Backup validity is not guaranteed if prepare process was interrupted.

      Note

      If you intend the backup to be the basis for further incremental backups, you should use the --apply-log-only option when preparing the backup, or you will not be able to apply incremental backups to it. See the documentation on preparing incremental backups for more details.

      "},{"location":"prepare-full-backup.html#next-step","title":"Next step","text":"

      Restore the backup

      "},{"location":"prepare-full-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"prepare-incremental-backup.html","title":"Prepare an incremental backup","text":"

      The --prepare step for incremental backups is not the same as for full backups. In full backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. You must skip the rollback of uncommitted transactions when preparing an incremental backup, because transactions that were uncommitted at the time of your backup may be in progress, and it\u2019s likely that they will be committed in the next incremental backup. You should use the --apply-log-only option to prevent the rollback phase.

      Warning

      If you do not use the --apply-log-only option to prevent the rollback phase, then your incremental backups will be useless. After transactions have been rolled back, further incremental backups cannot be applied.

      Beginning with the full backup you created, you can prepare it, and then apply the incremental differences to it. Recall that you have the following backups:

      /data/backups/base\n/data/backups/inc1\n/data/backups/inc2\n

      To prepare the base backup, you need to run --prepare as usual, but prevent the rollback phase:

      $ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base\n

      The output should end with text similar to the following:

      Expected output 
      InnoDB: Shutdown completed; log sequence number 1626007\n161011 12:41:04 completed OK!\n

      The log sequence number should match the to_lsn of the base backup, which you saw previously.

      Warning

      This backup is actually safe to restore as-is now, even though the rollback phase has been skipped. If you restore it and start MySQL, InnoDB will detect that the rollback phase was not performed, and it will do that in the background, as it usually does for a crash recovery upon start. It will notify you that the database was not shut down normally.

      To apply the first incremental backup to the full backup, run the following command:

      $ xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc1\n

      This applies the delta files to the files in /data/backups/base, which rolls them forward in time to the time of the incremental backup. It then applies the redo log as usual to the result. The final data is in /data/backups/base, not in the incremental directory. You should see an output similar to:

      Expected output 
      incremental backup from 1626007 is enabled.\nxtrabackup: cd to /data/backups/base\nxtrabackup: This target seems to be already prepared with --apply-log-only.\nxtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(4124244)\n...\nxtrabackup: page size for /tmp/backups/inc1/ibdata1.delta is 16384 bytes\nApplying /tmp/backups/inc1/ibdata1.delta to ./ibdata1...\n...\n161011 12:45:56 completed OK!\n

      Again, the LSN should match what you saw from your earlier inspection of the first incremental backup. If you restore the files from /data/backups/base, you should see the state of the database as of the first incremental backup.

      Warning

      Percona XtraBackup does not support using the same incremental backup directory to prepare two copies of backup. Do not run --prepare with the same incremental backup directory (the value of \u2013incremental-dir) more than once.

      Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:

      $ xtrabackup --prepare --target-dir=/data/backups/base \\\n--incremental-dir=/data/backups/inc2\n

      Note

      --apply-log-only should be used when merging the incremental backups except the last one. That\u2019s why the previous line does not contain the --apply-log-only option. Even if the --apply-log-only was used on the last step, backup would still be consistent but in that case server would perform the rollback phase.

      "},{"location":"prepare-incremental-backup.html#next-step","title":"Next step","text":"

      Restore the backup

      "},{"location":"prepare-incremental-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"prepare-individual-partitions-backup.html","title":"Prepare an individual partitions backup","text":"

      For preparing partial backups, the procedure is analogous to restoring individual tables. Apply the logs and use xtrabackup \u2013export:

      $ xtrabackup --apply-log --export /mnt/backup/2012-08-28_10-29-09\n

      You may see warnings in the output about tables that do not exist. This happens because InnoDB-based engines stores its data dictionary inside the tablespace files. xtrabackup removes the missing tables (those that haven\u2019t been selected in the partial backup) from the data dictionary in order to avoid future warnings or errors.

      "},{"location":"prepare-individual-partitions-backup.html#next-step","title":"Next step","text":"

      Restore the partition from the backup

      "},{"location":"prepare-individual-partitions-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"prepare-partial-backup.html","title":"Prepare a partial backup","text":"

      The procedure is analogous to restoring individual tables: apply the logs and use the --export option:

      $ xtrabackup --prepare --export --target-dir=/path/to/partial/backup\n

      When you use the --prepare option on a partial backup, you will see warnings about tables that don\u2019t exist. This is because these tables exist in the data dictionary inside InnoDB, but the corresponding .ibd files don\u2019t exist. They were not copied into the backup directory. These tables will be removed from the data dictionary, and when you restore the backup and start InnoDB, they will no longer exist and will not cause any errors or warnings to be printed to the log file.

      Could not find any file associated with the tablespace ID: 5

      Use --innodb-directories to find the tablespace files. If that fails then use -\u2013innodb-force-recovery=1 to ignore this and to permanently lose all changes to the missing tablespace(s).

      "},{"location":"prepare-partial-backup.html#next-step","title":"Next step","text":"

      Restore the partition from the backup

      "},{"location":"prepare-partial-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"privileges.html","title":"Connection and privileges needed","text":"

      Percona XtraBackup needs to be able to connect to the database server and perform operations on the server and the datadir when creating a backup, when preparing in some scenarios and when restoring it. In order to do so, there are privileges and permission requirements on its execution that must be fulfilled.

      Privilege refers to the operations that a system user is permitted to do in the database server. They are set at the database server and only apply to users in the database server.

      Permissions are those which permits a user to perform operations on the system, like reading, writing or executing on a certain directory or start/stop a system service. They are set at a system level and only apply to system users.

      When xtrabackup is used, there are two actors involved: the user invoking the program - a system user - and the user performing action in the database server - a database user. Note that these are different users in different places, even though they may have the same username.

      All the invocations of xtrabackup in this documentation assume that the system user has the appropriate permissions, and you are providing the relevant options for connecting the database server - besides the options for the action to be performed - and the database user has adequate privileges.

      "},{"location":"privileges.html#connect-to-the-server","title":"Connect to the server","text":"

      The database user used to connect to the server and its password are specified by the --user and --password option:

      $ xtrabackup --user=DVADER --password=14MY0URF4TH3R --backup \\\n--target-dir=/data/bkps/\n

      If you don\u2019t use the --user option, Percona XtraBackup will assume the database user whose name is the system user executing it.

      "},{"location":"privileges.html#other-connection-options","title":"Other connection options","text":"

      According to your system, you may need to specify one or more of the following options to connect to the server:

      Option Description -port Use this port when connecting to the database with TCP/IP -socket Use this socket when connecting to the local database. -host Use this host when connecting to the database server with TCP/IP

      These options are passed to the mysql child process without alteration, see mysql --help for details.

      Note

      In case of multiple server instances, the correct connection parameters (port, socket, host) must be specified in order for xtrabackup to talk to the correct server.

      "},{"location":"privileges.html#privileges-needed","title":"Privileges needed","text":"

      Once connected to the server, in order to perform a backup you need READ and EXECUTE permissions at a filesystem level in the server\u2019s datadir.

      The database user needs the following privileges to back up tables or databases:

      • RELOAD and FLUSH_TABLES in order to run FLUSH TABLES WITH READ LOCK.

      • The BACKUP_ADMIN privilege is needed to query the performance_schema.log_status table, and run LOCK INSTANCE FOR BACKUP, LOCK BINLOG FOR BACKUP, or LOCK TABLES FOR BACKUP. The BACKUP_ADMIN privilege is required to use the Page tracking feature.

      • REPLICATION CLIENT in order to obtain the binary log position,

      • CREATE TABLESPACE in order to import tables (see Restoring Individual Tables),

      • PROCESS in order to run SHOW ENGINE INNODB STATUS (which is mandatory), and optionally to see all threads which are running on the server (see FLUSH TABLES WITH READ LOCK option),

      • REPLICATION_SLAVE_ADMIN in order to start/stop the replication threads in a replication environment,

      • CREATE privilege in order to create the PERCONA_SCHEMA.xtrabackup_history database and table,

      • ALTER privilege in order to upgrade the PERCONA_SCHEMA.xtrabackup_history database and table,

      • INSERT privilege in order to add history records to the PERCONA_SCHEMA.xtrabackup_history table,

      • SELECT privilege in order to use --incremental-history-name or --incremental-history-uuid in order for the feature to look up the innodb_to_lsn values in the PERCONA_SCHEMA.xtrabackup_history table.

      • SELECT privilege on the keyring_component_status table to view the attributes and status of the installed keyring component when in use.

      • SELECT privilege on the replication_group_members table to validate if the instance is part of group replication cluster.

      A SQL example of creating a database user with the minimum privileges required to take full backups would be:

      mysql> CREATE USER 'bkpuser'@'localhost' IDENTIFIED BY 's3cr%T';\nmysql> GRANT BACKUP_ADMIN, PROCESS, RELOAD, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'bkpuser'@'localhost';\nmysql> GRANT SELECT ON performance_schema.log_status TO 'bkpuser'@'localhost';\nmysql> GRANT SELECT ON performance_schema.keyring_component_status TO bkpuser@'localhost';\nmysql> GRANT SELECT ON performance_schema.replication_group_members TO bkpuser@'localhost';\nmysql> FLUSH PRIVILEGES;\n
      "},{"location":"privileges.html#query-the-privileges","title":"Query the privileges","text":"

      To query the privileges that your database user has been granted at the console of the server execute:

      mysql> SHOW GRANTS;\n

      or for a particular user with:

      mysql> SHOW GRANTS FOR 'db-user'@'host';\n

      It will display the privileges using the same format as for the GRANT statement.

      Note that privileges may vary across versions of the server. To list the exact list of privileges that your server support (and a brief description of them) execute:

      mysql> SHOW PRIVILEGES;\n

      See also

      Permissions needed

      "},{"location":"privileges.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"quickstart-overview.html","title":"Quickstart Guide for Percona XtraBackup 8.1","text":"

      Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.

      "},{"location":"quickstart-overview.html#install-percona-xtrabackup","title":"Install Percona XtraBackup:","text":"

      You can install Percona XtraBackup using different methods:

      • Use the Percona Repositories

      • Use APT

      • Use YUM

      • Use binary tarballs

      • Use Docker

      "},{"location":"quickstart-overview.html#for-superior-and-optimized-performance","title":"For superior and optimized performance","text":"

      Percona Server for MySQL (PS) is a freely available, fully compatible, enhanced, and open source drop-in replacement for any MySQL database. It provides superior and optimized performance, greater scalability and availability, enhanced backups, increased visibility, and instrumentation. Percona Server for MySQL is trusted by thousands of enterprises to provide better performance and concurrency for their most demanding workloads.

      Install Percona Server for MySQL.

      "},{"location":"quickstart-overview.html#for-high-availability","title":"For high availability","text":"

      Percona XtraDB Cluster (PXC) is a 100% open source, enterprise-grade, highly available clustering solution for MySQL multi-master setups based on Galera. PXC helps enterprises minimize unexpected downtime and data loss, reduce costs, and improve performance and scalability of your database environments supporting your critical business applications in the most demanding public, private, and hybrid cloud environments.

      Install Percona XtraDB Cluster.

      "},{"location":"quickstart-overview.html#for-monitoring-and-management","title":"For Monitoring and Management","text":"

      Percona Monitoring and Management (PMM) monitors and provides actionable performance data for MySQL variants, including Percona Server for MySQL, Percona XtraDB Cluster, Oracle MySQL Community Edition, Oracle MySQL Enterprise Edition, and MariaDB. PMM captures metrics and data for the InnoDB, XtraDB, and MyRocks storage engines, and has specialized dashboards for specific engine details.

      Install PMM and connect your MySQL instances to it.

      "},{"location":"quickstart-overview.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"restore-a-backup.html","title":"Restore a backup","text":"

      Warning

      Backup needs to be prepared before it can be restored.

      For convenience, xtrabackup binary has the --copy-back option to copy the backup to the datadir of the server:

      $ xtrabackup --copy-back --target-dir=/data/backups/\n

      If you don\u2019t want to save your backup, you can use the --move-back option which will move the backed up data to the datadir.

      If you don\u2019t want to use any of the above options, you can additionally use rsync or cp to restore the files.

      Note

      The datadir must be empty before restoring the backup. Also, it\u2019s important to note that MySQL server needs to be shut down before restore is performed. You cannot restore to a datadir of a running mysqld instance (except when importing a partial backup).

      Example of the rsync command that can be used to restore the backup can look like this:

      $ rsync -avrP /data/backup/ /var/lib/mysql/\n

      You should check that the restored files have the correct ownership and permissions.

      As files\u2019 attributes are preserved, in most cases you must change the files\u2019 ownership to mysql before starting the database server, as the files are owned by the user who created the backup:

      $ chown -R mysql:mysql /var/lib/mysql\n

      Data is now restored, and you can start the server.

      "},{"location":"restore-a-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"restore-individual-partitions.html","title":"Restore the partition from the backup","text":"

      Restoring should be done by importing the tables in the partial backup to the server.

      First step is to create new table in which data will be restored.

      mysql> CREATE TABLE `name_p4` (\n`id` int(11) NOT NULL AUTO_INCREMENT,\n`name` text NOT NULL,\n`imdb_index` varchar(12) DEFAULT NULL,\n`imdb_id` int(11) DEFAULT NULL,\n`name_pcode_cf` varchar(5) DEFAULT NULL,\n`name_pcode_nf` varchar(5) DEFAULT NULL,\n`surname_pcode` varchar(5) DEFAULT NULL,\nPRIMARY KEY (`id`)\n) ENGINE=InnoDB AUTO_INCREMENT=2812744 DEFAULT CHARSET=utf8\n

      Note

      Generate a .cfg metadata file by running FLUSH TABLES ... FOR EXPORT. The command can only be run on a table, not on the individual table partitions. The file is located in the table schema directory and is used for schema verification when importing the tablespace.

      To restore the partition from the backup, the tablespace must be discarded for that table:

      mysql> ALTER TABLE name_p4 DISCARD TABLESPACE;\n

      The next step is to copy the .ibd file from the backup to the MySQL data directory:

      cp /mnt/backup/2012-08-28_10-29-09/imdb/name#P#p4.ibd /var/lib/mysql/imdb/name_p4.ibd\n

      Note

      Make sure that the copied files can be accessed by the user running MySQL.

      The last step is to import the tablespace:

      mysql> ALTER TABLE name_p4 IMPORT TABLESPACE;\n
      "},{"location":"restore-individual-partitions.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"restore-individual-tables.html","title":"Restore individual tables","text":"

      Percona XtraBackup can export a table that is contained in its own .ibd file. With Percona XtraBackup, you can export individual tables from any InnoDB database, and import them into Percona Server for MySQL with XtraDB or MySQL 8.1. The source doesn\u2019t have to be XtraDB or MySQL 8.1, but the destination does. This method only works on individual .ibd files.

      The following example exports and imports the following table:

      CREATE TABLE export_test (\na int(11) DEFAULT NULL\n) ENGINE=InnoDB DEFAULT CHARSET=latin1;\n
      "},{"location":"restore-individual-tables.html#export-the-table","title":"Export the table","text":"

      To generate an .ibd file in the target directory, create the table using the innodb_file_per_table mode:

      $ find /data/backups/mysql/ -name export_test.*\n/data/backups/mysql/test/export_test.ibd\n

      During the --prepare step, add the --export option to the command. For example:

      $ xtrabackup --prepare --export --target-dir=/data/backups/mysql/\n

      When restoring an encrypted InnoDB tablespace table, add the keyring file:

      $ xtrabackup --prepare --export --target-dir=/tmp/table \\\n--keyring-file-data=/var/lib/mysql-keyring/keyring\n

      The following files are the only files required to import the table into a server running Percona Server for MySQL with XtraDB or MySQL 8.1. If the server uses InnoDB Tablespace Encryption, add the .cfp file, which contains the transfer key and an encrypted tablespace key.

      The files are located in the target directory:

      /data/backups/mysql/test/export_test.ibd\n/data/backups/mysql/test/export_test.cfg\n
      "},{"location":"restore-individual-tables.html#import-the-table","title":"Import the table","text":"

      On the destination server running Percona Server for MySQL with XtraDB or MySQL 8.1, create a table with the same structure, and then perform the following steps:

      1. Run the ALTER TABLE test.export_test DISCARD TABLESPACE; command. If you see the following error message:

        Error message 
        ERROR 1809 (HY000): Table 'test/export_test' in system tablespace\n

        enable innodb_file_per_table option on the server and create the table again.

        $ set global innodb_file_per_table=ON;\n

      2. Copy the exported files to the test/ subdirectory of the destination server\u2019s data directory.

      3. Run ALTER TABLE test.export_test IMPORT TABLESPACE;

        The table is imported, and you can run a SELECT to see the imported data.

      "},{"location":"restore-individual-tables.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"restore-partial-backup.html","title":"Restore a partial backup","text":"

      Restoring should be done by restoring individual tables in the partial backup to the server.

      It can also be done by copying back the prepared backup to a \u201cclean\u201d datadir (in that case, make sure to include the mysql database) to the datadir you are moving the backup to. A system database can be created with the following:

      $ sudo mysql --initialize --user=mysql\n

      Once you start the server, you may see mysql complaining about missing tablespaces:

      Expected output 
      2021-07-19T12:42:11.077200Z 1 [Warning] [MY-012351] [InnoDB] Tablespace 4, name 'test1/t1', file './d2/test1.ibd' is missing!\n2021-07-19T12:42:11.077300Z 1 [Warning] [MY-012351] [InnoDB] Tablespace 4, name 'test1/t1', file './d2/test1.ibd' is missing!\n

      In order to clean the orphan database from the data dictionary, you must manually create the missing database directory and then DROP this database from the server.

      Example of creating the missing database:

      $ mkdir /var/lib/mysql/test1/d2\n

      Example of dropping the database from the server:

      mysql> DROP DATABASE d2;\n
      Expected output 
      Query OK, 2 rows affected (0.5 sec)\n
      "},{"location":"restore-partial-backup.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"server-backup-version-comparison.html","title":"Server version and backup version comparison","text":"

      A MySQL change to a feature, such as the structure of a redo log record, can cause older versions of Percona XtraBackup to fail. To ensure that you can back up and restore your data, use a Percona XtraBackup version that is equal to your source server major version. This means if you use Percona XtraBackup 8.1.x, you can safely back up the source server from 8.1.x to 8.1.xx.

      See also

      How XtraBackup works

      The --no-server-version-check option performs the following test. Before the backup/prepare starts, XtraBackup compares the source server version to the Percona XtraBackup version. If the source server version is greater than the XtraBackup major version, XtraBackup stops the backup and returns an error message. This comparison prevents a failed backup or a corrupted backup due to source server changes.

      The parameter checks for the following scenarios:

      • The source server and the PXB major version are the same, the backup proceeds

      • The source server is greater than the PXB major version, and the parameter is not overridden, the backup is stopped and returns an error message

      • The source server is less than the PXB major version, and the parameter is not overridden, the backup is stopped and returns an error message

      • The source server is greater than the PXB major version up to the last release in the LTS series, and the parameter is overridden, the backup proceeds

      Explicitly adding the --no-server-version-check parameter, like the example, overrides the parameter and the backup proceeds.

      $ xtrabackup --backup --no-server-version-check --target-dir=$mysql/backup1\n

      Overriding the --no-server-version-check parameter allows taking backups using a Percona XtraBackup version that is equal to a version of your source server up to the last release in the LTS series. This means if you use Percona XtraBackup 8.1.x, you can back up the source server from 8.1.x to 8.4.xx.

      When you override the parameter, the following events can happen:

      • Backup fails

      • Creates a corrupted backup

      • Backup successful

      "},{"location":"server-backup-version-comparison.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"set-up-replication.html","title":"How to set up a replica for replication in 6 simple steps with Percona XtraBackup","text":"

      Data is, by far, the most valuable part of a system. Having a backup done systematically and available for a rapid recovery in case of failure is admittedly essential to a system. However, it is not common practice because of its costs, infrastructure needed or even the boredom associated to the task. Percona XtraBackup is designed to solve this problem.

      You can have almost real-time backups in 6 simple steps by setting up a replication environment with Percona XtraBackup.

      "},{"location":"set-up-replication.html#things-you-need","title":"Things you need","text":"

      Setting up a replica for replication with Percona XtraBackup is a straightforward procedure. In order to keep it simple, here is a list of the things you need to follow the steps without hassles:

      Source

      A system with a MySQL-based server installed, configured and running. This system is called Source. The Source server stores your data and can be replicated. We assume the following about this system:

      • the MySQL server is able to communicate with others by the standard TCP/IP port;

      • the SSH server is installed and configured;

      • you have a user account in the system with the appropriate permissions;

      • you have a MySQL\u2019s user account with appropriate privileges.

      • server has binlogs enabled and server-id set up to 1.

      Replica

      Another system, with a MySQL-based server installed on it. We refer to this machine as Replica and assume the same things we did about Source, except that the server-id on Replica is 2.

      Percona XtraBackup

      We use this backup tool. Install Percona XtraBackup on both computers for convenience.

      Note

      It is not recommended to mix MySQL variants (Percona Server, MySQL) in your replication setup. This may produce incorrect xtrabackup_slave_info file when adding a new replica.

      "},{"location":"set-up-replication.html#1-make-a-backup-on-the-source-and-prepare-it","title":"1. Make a backup on the Source and prepare it","text":"

      At the Source, issue the following to a shell:

      $ xtrabackup --backup --user=yourDBuser --password=MaGiCdB1 --target-dir=/path/to/backupdir\n

      After this is finished you should get:

      Expected output 
      xtrabackup: completed OK!\n

      This operation makes a copy of your MySQL data dir to the /path/to/backupdir directory. You have told Percona XtraBackup to connect to the database server using your database user and password, and do a hot backup of all your data in it (all MyISAM, InnoDB tables and indexes in them).

      In order for snapshot to be consistent, prepare the data on the source:

      $ xtrabackup --prepare --target-dir=/path/to/backupdir\n

      Select the path where your snapshot has been taken. Apply the transaction logs to the data files and your data files are ready to be used by the MySQL server.

      Percona XtraBackup reads the my.cnf file to locate your data. If you have your configuration file in a non-standard place, you should use the flag --defaults-file =/location/of/my.cnf.

      If you want to skip writing the username and password every time you want to access MySQL, you can set it up in .mylogin.cnf as follows:

      mysql_config_editor set --login-path=client --host=localhost --user=root --password\n

      For more information, see MySQL Configuration Utility.

      This statement provides root access to MySQL.

      "},{"location":"set-up-replication.html#2-copy-backed-up-data-to-the-replica","title":"2. Copy backed up data to the Replica","text":"

      On the Source, use rsync or scp to copy the data from the Source to the Replica. If you are syncing the data directly to replica\u2019s data directory, we recommend that you stop the mysqld there.

      $ rsync -avpP -e ssh /path/to/backupdir Replica:/path/to/mysql/\n

      After data is copied, you can back up the original or previously installed MySQL datadir.

      Note

      Make sure mysqld is shut down before you move the contents of its datadir, or move the snapshot into its datadir.

      Run the following commands on the Replica:

      $ mv /path/to/mysql/datadir /path/to/mysql/datadir_bak\n

      and move the snapshot from the Source in its place:

      $ xtrabackup --move-back --target-dir=/path/to/mysql/backupdir\n

      After you copy data over, make sure the Replica MySQL has the proper permissions to access them.

      $ chown mysql:mysql /path/to/mysql/datadir\n

      If the ibdata and iblog files are located in directories outside the datadir, you must put these files in their proper place after the logs have been applied.

      "},{"location":"set-up-replication.html#3-configure-the-sources-mysql-server","title":"3. Configure the Source\u2019s MySQL server","text":"

      On the source, run the following command to add the appropriate grant. This grant allows the replica to be able to connect to source:

      mysql> GRANT REPLICATION SLAVE ON *.*  TO 'repl'@'$replicaip'\nIDENTIFIED BY '$replicapass';\n

      Also make sure that firewall rules are correct and that the Replica can connect to the Source. Run the following command on the Replica to test that you can run the mysql client on Replica, connect to the Source, and authenticate.

      mysql> mysql --host=Source --user=repl --password=$replicapass\n

      Verify the privileges.

      mysql> SHOW GRANTS;\n
      "},{"location":"set-up-replication.html#4-configure-the-replicas-mysql-server","title":"4. Configure the Replica\u2019s MySQL server","text":"

      Copy the my.cnf file from the Source to the Replica:

      $ scp user@Source:/etc/mysql/my.cnf /etc/mysql/my.cnf\n

      and change the following options in /etc/mysql/my.cnf:

      server-id=2\n

      and start/restart mysqld on the Replica.

      In case you\u2019re using init script on Debian-based system to start mysqld, be sure that the password for debian-sys-maint user has been updated, and it\u2019s the same as that user\u2019s password on the Source. Password can be seen and updated in /etc/mysql/debian.cnf.

      "},{"location":"set-up-replication.html#5-start-the-replication","title":"5. Start the replication","text":"

      On the Replica, review the content of the xtrabackup_binlog_info file:

      $ cat /var/lib/mysql/xtrabackup_binlog_info\n

      The results should resemble the following:

      Expected output 
      Source-bin.000001     481\n

      Do the following on a MySQL console and use the username and password you\u2019ve set up in STEP 3:

      Use the CHANGE_REPLICATION_SOURCE_TO statement

      CHANGE REPLICATION SOURCE TO\n    SOURCE_HOST='$sourceip',\n    SOURCE_USER='repl',\n    SOURCE_PASSWORD='$replicapass',\n    SOURCE_LOG_FILE='Source-bin.000001',\n    SOURCE_LOG_POS=481;\n

      Start the replica:

      START REPLICA;\n
      "},{"location":"set-up-replication.html#6-check","title":"6. Check","text":"

      On the Replica, check that everything went OK with:

      SHOW REPLICA STATUS\\G\n

      The result shows the status:

      Expected output 
      Slave_IO_Running: Yes\nSlave_SQL_Running: Yes\nSeconds_Behind_Master: 13\n

      Both IO and SQL threads need to be running. The Seconds_Behind_Master means the SQL currently being executed has a current_timestamp of 13 seconds ago. It is an estimation of the lag between the Source and the Replica. Note that at the beginning, a high value could be shown because the Replica has to \u201ccatch up\u201d with the Source.

      "},{"location":"set-up-replication.html#adding-more-replicas-to-the-source","title":"Adding more replicas to the Source","text":"

      You can use this procedure with slight variation to add new replicas to a source. We will use Percona XtraBackup to clone an already configured replica. We will continue using the previous scenario for convenience, but we will add a NewReplica to the plot.

      At the Replica, do a full backup:

      $ xtrabackup --user=yourDBuser --password=MaGiCiGaM \\\n   --backup --slave-info --target-dir=/path/to/backupdir\n

      By using the --slave-info Percona XtraBackup creates additional file called xtrabackup_slave_info.

      Apply the logs:

      $ xtrabackup --prepare --use-memory=2G --target-dir=/path/to/backupdir/\n

      Note

      In the prepare phase, the --use-memory parameter speeds up the process if the amount of RAM assigned to the option is available. Use the parameter only in the prepare phase. In the other phases the parameter makes the application lazy allocate this memory (reserve) but does not affect database pages.

      Copy the directory from the Replica to the NewReplica:

      Note

      Make sure mysqld is shut down on the NewReplica before you copy the contents the snapshot into its datadir.

      rsync -avprP -e ssh /path/to/backupdir NewReplica:/path/to/mysql/datadir\n

      For example, to set up a new user, user2, you add another grant on the Source:

      > GRANT REPLICATION SLAVE ON *.*  TO 'user2'@'$newreplicaip'\n IDENTIFIED BY '$replicapass';\n

      On the NewReplica, copy the configuration file from the Replica:

      $ scp user@Replica:/etc/mysql/my.cnf /etc/mysql/my.cnf\n

      Make sure you change the server-id variable in /etc/mysql/my.cnf to 3 and disable the replication on start:

      skip-slave-start\nserver-id=3\n

      After setting server_id, start mysqld.

      Fetch the source_log_file and source_log_pos from the file xtrabackup_slave_info, execute the statement for setting up the source and the log file for the NewReplica:

      CHANGE REPLICATION SOURCE TO\n    SOURCE_HOST='$sourceip',\n    SOURCE_USER='repl',\n    SOURCE_PASSWORD='$replicapass',\n    SOURCE_LOG_FILE='Source-bin.000001',\n    SOURCE_LOG_POS=481;\n

      Start the replica:

      > START REPLICA;\n

      If both IO and SQL threads are running when you check the NewReplica, server is replicating the Source.

      See also

      How to create a new (or repair a broken) GTID based slave

      "},{"location":"set-up-replication.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"smart-memory-estimation.html","title":"Smart memory estimation","text":"

      The Smart memory estimation is tech preview feature. Before using Smart memory estimation in production, we recommend that you test restoring production from physical backups in your environment and also use the alternative backup method for redundancy.

      Percona XtraBackup supports the Smart memory estimation feature. With this feature, Percona XtraBackup computes the memory required for prepare phase, while copying redo log entries during the backup phase. Percona XtraBackup also considers the number of InnoDB pages to be fetched from the disk.

      Percona XtraBackup performs the backup procedure in two steps:

      • Creates a backup

        To create a backup, Percona XtraBackup copies your InnoDB data files. While copying the files, Percona XtraBackup runs a background process that watches the InnoDB redo log, also called the transaction log, and copies changes from it.

      • Prepares a backup

        During the prepare phase, Percona XtraBackup performs crash recovery against the copied data files using the copied transaction log file. Percona XtraBackup reads all the redo log entries into memory, categorizes them by space id and page id, reads the relevant pages into memory, and checks the log sequence number (LSN) on the page and on the redo log record. If the redo log LSN is more recent than the page LSN, Percona XtraBackup applies the redo log changes to the page.

        To prepare a backup, Percona Xtrabackup uses InnoDB Buffer Pool memory. Percona Xtrabackup reserves memory to load 256 pages into the buffer pool. The remaining memory is used for hashing/categorizing the redo log entries.

        The available memory is controlled by the --use-memory option. If the available memory on the buffer pool is insufficient, the work is performed in multiple batches. After the batch is processed, the memory is freed to release space for the next batch. This process greatly impacts performance as an InnoDB page holds data from multiple rows. If a change on a page happens in different batches, that page is fetched and evicted numerous times.

      "},{"location":"smart-memory-estimation.html#how-does-smart-memory-estimation-work","title":"How does Smart memory estimation work","text":"

      To run prepare, Percona XtraBackup checks the server\u2019s available free memory and uses that memory up to the limit specified in the --use-free-memory-pct option. Due to backward compatibility, the default value for the --use-free-memory-pct option is 0 (zero), which defines the option as disabled. For example, if you set --use-free-memory-pct=50, then 50% of the free memory is used to prepare a backup.

      You can enable or disable the memory estimation during the backup phase with the --estimate-memory option. The default value is OFF. Enable the memory estimation with --estimate-memory=ON:

      $ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n

      In the prepare phase, enable the --use-free-memory-pct option by specifying the percentage of free memory to be used to prepare a backup. The --use-free-memory-pct value must be larger than 0.

      For example:

      $ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\n
      "},{"location":"smart-memory-estimation.html#example-of-smart-memory-estimation-usage","title":"Example of Smart memory estimation usage","text":"

      The examples of how Smart memory estimation can improve the time spent on prepare in Percona XtraBackup:

      We back up 16, 32, and 64 tables using sysbench. Each set contains 1M rows. In the backup phase, we enable Smart memory estimation with --estimate-memory=ON. In the prepare phase, we set --use-free-memory-pct=50, and Percona XtraBackup uses 50% of the free memory to prepare a backup. The backup is run on an ec2 c4.8xlarge instance (36 vCPUs / 60GB memory / General Purpose SSD (gp2)).

      During each --backup, the following sysbench is run:

      sysbench --db-driver=mysql --db-ps-mode=disable --mysql-user=sysbench --mysql-password=sysbench --table_size=1000000 --tables=${NUM_OF_TABLES} --threads=24 --time=0 --report-interval=1 /usr/share/sysbench/oltp_write_only.lua run\n

      The following table shows the backup details (all measurements are in Gigabytes):

      Used memory Size of XtraBackup log Size of backup 16 tables 3.375 0.7 4.7 32 tables 8.625 2.6 11 64 tables 18.5 5.6 22

      • Used memory - the amount of memory required by Percona XtraBackup with --use-free-memory-pct=50

      • Size of XtraBackup log - the size of Percona XtraBackup log file (redo log entries copied during the backup)

      • Size of backup - the size of the resulting backup folder

      Prepare executed without Smart memory estimation uses the default of 128MB for the buffer pool.

      The results are the following:

      Note

      The following results are based on tests in a specific environment. Your results may vary.

      • 16 tables result - prepare time dropped to ~5.7% of the original time. An improvement in recovery time of about 17x.

      • 32 tables result - prepare time dropped to ~8,2% of the original time. An improvement in recovery time of about 12x.

      • 64 tables result - prepare time dropped to ~9.9% of the original time. An improvement in recovery time of about 10x.

      "},{"location":"smart-memory-estimation.html#get-expert-help","title":"Get expert help","text":"

      If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

      Community Forum Get a Percona Expert

      "},{"location":"store-backup-history.html","title":"Store backup history on the server","text":"

      Percona XtraBackup supports storing the backups history on the server. Storing backup history on the server was implemented to provide users with additional information about backups that are being taken. Backup history information will be stored in the PERCONA_SCHEMA.XTRABACKUP_HISTORY table.

      To use this feature the following options are available:

      • --history = : This option enables the history feature and allows the user to specify a backup series name that will be placed within the history record.

      • --incremental-history-name = : This option allows an incremental backup to be made based on a specific history series by name. xtrabackup will search the history table looking for the most recent (highest to_lsn) backup in the series and take the to_lsn value to use as it\u2019s starting lsn. This is mutually exclusive with --incremental-history-uuid, --incremental-basedir and --incremental-lsn options. If no valid LSN can be found (no series by that name) xtrabackup will return with an error.

      • --incremental-history-uuid = : Allows an incremental backup to be made based on a specific history record identified by UUID. xtrabackup will search the history table looking for the record matching UUID and take the to_lsn value to use as it\u2019s starting LSN. This options is mutually exclusive with --incremental-basedir, --incremental-lsn and --incremental-history-name options. If no valid LSN can be found (no record by that UUID or missing to_lsn), xtrabackup will return with an error.

        Note

        Backup that\u2019s currently being performed will NOT exist in the xtrabackup_history table within the resulting backup set as the record will not be added to that table until after the backup has been taken.

        If you want access to backup history outside of your backup set in the case of some catastrophic event, you will need to either perform a mysqldump, partial backup or SELECT * on the history table after xtrabackup completes and store the results with you backup set.

        For the necessary privileges, see Permissions and Privileges Needed.

        "},{"location":"store-backup-history.html#percona_schemaxtrabackup_history-table","title":"PERCONA_SCHEMA.XTRABACKUP_HISTORY table","text":"

        This table contains the information about the previous server backups. Information about the backups will only be written if the backup was taken with --history option.

        Column Name Description uuid Unique backup id name User provided name of backup series. There may be multiple entries with the same name used to identify related backups in a series. tool_name Name of tool used to take backup tool_command Exact command line given to the tool with \u2013password and \u2013encryption_key obfuscated tool_version Version of tool used to take backup ibbackup_version Version of the xtrabackup binary used to take backup server_version Server version on which backup was taken start_time Time at the start of the backup end_time Time at the end of the backup lock_time Amount of time, in seconds, spent calling and holding locks for FLUSH TABLES WITH READ LOCK binlog_pos Binlog file and position at end of FLUSH TABLES WITH READ LOCK innodb_from_lsn LSN at beginning of backup which can be used to determine prior backups innodb_to_lsn LSN at end of backup which can be used as the starting lsn for the next incremental partial Is this a partial backup, if N that means that it\u2019s the full backup incremental Is this an incremental backup format Description of result format (xbstream) compact Is this a compact backup compressed Is this a compressed backup encrypted Is this an encrypted backup"},{"location":"store-backup-history.html#limitations","title":"Limitations","text":"

        • --history option must be specified only on the command line and not within a configuration file in order to be effective.

        • --incremental-history-name and --incremental-history-uuid options must be specified only on the command line and not within a configuration file in order to be effective.

        "},{"location":"store-backup-history.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"take-streaming-backup.html","title":"Take a streaming backup","text":"

        Percona XtraBackup supports streaming mode. Streaming mode sends a backup to STDOUT in the xbstream format instead of copying the files to the backup directory.

        This method allows you to use other programs to filter the output of the backup, providing greater flexibility for storage of the backup. For example, compression is achieved by piping the output to a compression utility. One of the benefits of streaming backups and using Unix pipes is that the backups can be automatically encrypted.

        To use the streaming feature, you must use the --stream, providing the format of the stream (xbstream ) and where to store the temporary files:

        $ xtrabackup --stream=xbstream --target-dir=/tmp\n

        xtrabackup uses xbstream to stream all of the data files to STDOUT, in a special xbstream format. After it finishes streaming all of the data files to STDOUT, it stops xtrabackup and streams the saved log file too.

        When compression is enabled, xtrabackup compresses the output data, except for the meta and non-InnoDB files which are not compressed, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:

        "},{"location":"take-streaming-backup.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

        The Zstandard (ZSTD) is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. ZSTD is the default compression algorithm for the --compress option.

        To compress files using the ZSTD compression algorithm, use the --compress option:

        $ xtrabackup --backup --compress --target-dir=/data/backup\n

        The resulting files have the \\*.zst format.

        You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.

        $ xtrabackup \u2013backup \u2013compress \u2013compress-zstd-level=1 \u2013target-dir=/data/backup\n
        "},{"location":"take-streaming-backup.html#lz4","title":"lz4","text":"

        To compress files using the lz4 compression algorithm, set the --compress option to lz4:

        $ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\n

        The resulting files have the \\*.lz4 format.

        To decompress files, use the --decompress option.

        Using xbstream as a stream option, backups can be copied and compressed in parallel. This option can significantly improve the speed of the backup process. In case backups were both compressed and encrypted, they must be decrypted before they are uncompressed.

        Task Command Stream the backup into an archived named backup.xbstream $ xtrabackup --backup --stream=xbstream --target-dir=./ > backup.xbstream Stream the backup into a compressed archive named backup.xbstream $ xtrabackup --backup --stream=xbstream --compress --target-dir=./ > backup.xbstream Encrypt the backup $ xtrabackup --backup --stream=xbstream ./ > backup.xbstream gzip -`` | openssl des3 -salt -k \u201cpassword\u201d backup.xbstream.gz.des3 Unpack the backup to the current directory $ xbstream -x < backup.xbstream Send the backup compressed directly to another host and unpack it $ xtrabackup --backup --compress --stream=xbstream --target-dir=./ | ssh user@otherhost \"xbstream -x\" Send the backup to another server using netcat On the destination host:$ nc -l 9999 | cat - > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | nc desthost 9999 Send the backup to another server using a one-liner $ ssh user@desthost \u201c( nc -l 9999 > /data/backups/backup.xbstream & )\u201d && xtrabackup --backup --stream=xbstream ./ | nc desthost 9999 Throttle the throughput to 10MB/sec using the pipe viewer tool $ xtrabackup --backup --stream=xbstream ./ | pv -q -L10m ssh user@desthost \u201ccat - > /data/backups/backup.xbstream\u201d Checksum the backup during the streaming On the destination host:$ nc -l 9999 | tee >(sha1sum > destination_checksum) > /data/backups/backup.xbstreamOn the source host:$ xtrabackup --backup --stream=xbstream ./ | tee >(sha1sum > source_checksum) | nc desthost 9999Compare the checksums on the source host:$ cat source_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbadCompare the checksums on the destination host:$ cat destination_checksum 65e4f916a49c1f216e0887ce54cf59bf3934dbad Parallel compression with parallel copying backup $ xtrabackup --backup --compress --compress-threads=8 --stream=xbstream --parallel=4 --target-dir=./ > backup.xbstream

        Important

        The streamed backup must be prepared before restoration. Streaming mode does not prepare the backup.

        "},{"location":"take-streaming-backup.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"throttling-backups.html","title":"Throttling backups","text":"

        Although xtrabackup does not block your database\u2019s operation, any backup can add load to the system being backed up. On systems that do not have much spare I/O capacity, it might be helpful to throttle the rate at which xtrabackup reads and writes data. You can do this with the --throttle option. This option limits the number of chunks copied per second. The chunk +size is 10 MB.

        The image below shows how throttling works when --throttle is set to 1.

        When specified with the --backup option, this option limits the number of pairs of read-and-write operations per second that xtrabackup will perform. If you are creating an incremental backup, then the limit is the number of read I/O operations per second.

        By default, there is no throttling, and xtrabackup reads and writes data as quickly as it can. If you set too strict of a limit on the IOPS, the backup might be so slow that it will never catch up with the transaction logs that InnoDB is writing, so the backup might never complete.

        "},{"location":"throttling-backups.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"toolkit-version-check.html","title":"Percona Toolkit version checking","text":"

        Some Percona software contains \u201cversion checking\u201d functionality which is a feature that enables Percona software users to be notified of available software updates to improve your environment security and performance. Alongside this, the version check functionality also provides Percona with information relating to which software versions you are running, coupled with the environment confirmation which the software is running within. This helps enable Percona to focus our development effort accordingly based on trends within our customer community.

        The purpose of this document is to articulate the information that is collected, as well as to provide guidance on how to disable this functionality if desired.

        "},{"location":"toolkit-version-check.html#usage","title":"Usage","text":"

        Version Check was implemented in Percona Toolkit 2.1.4, and was enabled by default in version 2.2.1. Currently, it is supported as a --[no]version-check option by a number of tools in Percona Toolkit, Percona XtraBackup, and Percona Monitoring and Management (PMM).

        When launched with Version Check enabled, the tool that supports this feature connects to a Percona\u2019s version check service via a secure HTTPS channel. It compares the locally installed version for possible updates, and also checks versions of the following software:

        • Operating System

        • Percona Monitoring and Management (PMM)

        • MySQL

        • Perl

        • MySQL driver for Perl (DBD::mysql)

        • Percona Toolkit

        Then it checks for and warns about versions with known problems if they are identified as running in the environment.

        Each version check request is logged by the server. Stored information consists of the checked system unique ID followed by the software name and version. The ID is generated either at installation or when the version checking query is submitted for the first time.

        Note

        Prior to version 3.0.7 of Percona Toolkit, the system ID was calculated as an MD5 hash of a hostname, and starting from Percona Toolkit 3.0.7 it is generated as an MD5 hash of a random number. Percona XtraBackup continues to use hostname-based MD5 hash.

        As a result, the content of the sent query is as follows:

        Expected output 
        85624f3fb5d2af8816178ea1493ed41a;DBD::mysql;4.044\nc2b6d625ef3409164cbf8af4985c48d3;MySQL;MySQL Community Server (GPL) 5.7.22-log\n85624f3fb5d2af8816178ea1493ed41a;OS;Manjaro Linux\n85624f3fb5d2af8816178ea1493ed41a;Percona::Toolkit;3.0.11-dev\n85624f3fb5d2af8816178ea1493ed41a;Perl;5.26.2\n
        "},{"location":"toolkit-version-check.html#disabling-version-check","title":"Disabling version check","text":"

        Although the version checking feature does not collect any personal information, you might prefer to disable this feature, either one time or permanently. To disable it one time, use --no-version-check option when invoking the tool from a Percona product which supports it. Here is a simple example which shows running pt-diskstats tool from the Percona Toolkit with version checking turned off:

        pt-diskstats --no-version-check\n

        Disabling version checking permanently can be done by placing no-version-check option into the configuration file of a Percona product (see correspondent documentation for exact file name and syntax). For example, in case of Percona Toolkit this can be done in a global configuration file /etc/percona-toolkit/percona-toolkit.conf:

        # Disable Version Check for all tools:\nno-version-check\n

        In case of Percona XtraBackup this can be done in its configuration file in a similar way:

        [xtrabackup]\nno-version-check\n
        "},{"location":"toolkit-version-check.html#frequently-asked-questions","title":"Frequently asked questions","text":""},{"location":"toolkit-version-check.html#why-is-this-functionality-enabled-by-default","title":"Why is this functionality enabled by default?","text":"

        We believe having this functionality enabled improves security and performance of environments running Percona Software and it is good choice for majority of the users.

        "},{"location":"toolkit-version-check.html#why-not-rely-on-operating-systems-built-in-functionality-for-software-updates","title":"Why not rely on Operating System\u2019s built in functionality for software updates?","text":"

        In many environments the Operating Systems repositories may not carry the latest version of software and newer versions of software often installed manually, so not being covered by operating system wide check for updates.

        "},{"location":"toolkit-version-check.html#why-do-you-send-more-information-than-just-the-version-of-software-being-run-as-a-part-of-version-check-service","title":"Why do you send more information than just the version of software being run as a part of version check service?","text":"

        Compatibility problems can be caused by versions of various components in the environment, for example problematic versions of Perl, DBD or MySQL could cause operational problems with Percona Toolkit.

        "},{"location":"toolkit-version-check.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"trademark-policy.html","title":"Trademark policy","text":"

        This Trademark Policy is to ensure that users of Percona-branded products or services know that what they receive has really been developed, approved, tested and maintained by Percona. Trademarks help to prevent confusion in the marketplace, by distinguishing one company\u2019s or person\u2019s products and services from another\u2019s.

        Percona owns a number of marks, including but not limited to Percona, XtraDB, Percona XtraDB, XtraBackup, Percona XtraBackup, Percona Server, and Percona Live, plus the distinctive visual icons and logos associated with these marks. Both the unregistered and registered marks of Percona are protected.

        Use of any Percona trademark in the name, URL, or other identifying characteristic of any product, service, website, or other use is not permitted without Percona\u2019s written permission with the following three limited exceptions.

        First, you may use the appropriate Percona mark when making a nominative fair use reference to a bona fide Percona product.

        Second, when Percona has released a product under a version of the GNU General Public License (\u201cGPL\u201d), you may use the appropriate Percona mark when distributing a verbatim copy of that product in accordance with the terms and conditions of the GPL.

        Third, you may use the appropriate Percona mark to refer to a distribution of GPL-released Percona software that has been modified with minor changes for the sole purpose of allowing the software to operate on an operating system or hardware platform for which Percona has not yet released the software, provided that those third party changes do not affect the behavior, functionality, features, design or performance of the software. Users who acquire this Percona-branded software receive substantially exact implementations of the Percona software.

        Percona reserves the right to revoke this authorization at any time in its sole discretion. For example, if Percona believes that your modification is beyond the scope of the limited license granted in this Policy or that your use of the Percona mark is detrimental to Percona, Percona will revoke this authorization. Upon revocation, you must immediately cease using the applicable Percona mark. If you do not immediately cease using the Percona mark upon revocation, Percona may take action to protect its rights and interests in the Percona mark. Percona does not grant any license to use any Percona mark for any other modified versions of Percona software; such use will require our prior written permission.

        Neither trademark law nor any of the exceptions set forth in this Trademark Policy permit you to truncate, modify or otherwise use any Percona mark as part of your own brand. For example, if XYZ creates a modified version of the Percona Server, XYZ may not brand that modification as \u201cXYZ Percona Server\u201d or \u201cPercona XYZ Server\u201d, even if that modification otherwise complies with the third exception noted above.

        In all cases, you must comply with applicable law, the underlying license, and this Trademark Policy, as amended from time to time. For instance, any mention of Percona trademarks should include the full trademarked name, with proper spelling and capitalization, along with attribution of ownership to Percona LLC and/or its affiliates. For example, the full proper name for XtraBackup is Percona XtraBackup. However, it is acceptable to omit the word \u201cPercona\u201d for brevity on the second and subsequent uses, where such omission does not cause confusion.

        In the event of doubt as to any of the conditions or exceptions outlined in this Trademark Policy, please contact trademarks@percona.com for assistance and we will do our very best to be helpful.

        "},{"location":"trademark-policy.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"update-curl-utility.html","title":"Update curl utility","text":""},{"location":"update-curl-utility.html#update-the-curl-utility-in-debian-10","title":"Update the curl utility in Debian 10","text":"

        The default curl version, 7.64.0, in Debian 10 has known issues when attempting to reuse an already closed connection. This issue directly affects xbcloud and users may see intermittent backup failures.

        For more details, see curl #3750 or curl #3763.

        Follow these steps to upgrade curl to version 7.74.0:

        1. Edit the /etc/apt/sources.list to add the following:

          deb http://ftp.de.debian.org/debian buster-backports main\n

        2. Refresh the apt sources:

          $ sudo apt update\n

        3. Install the version from buster-backports:

          $ sudo apt install curl/buster-backports\n

        4. Verify the version number:

          $ curl --version\n
          Expected output 
          curl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0\n
        "},{"location":"update-curl-utility.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"varify-backup.html","title":"Verify backups with replication and pt-checksum","text":"

        One way to verify if the backup is consistent is by setting up the replication and running pt-table-checksum. This can be used to verify any type of backups, but before setting up replication, backup should be prepared and be able to run (this means that incremental backups should be merged to full backups, encrypted backups decrypted etc.).

        "},{"location":"varify-backup.html#set-up-the-replication","title":"Set up the replication","text":"

        How to set up a replica for replication in 6 simple steps with Percona XtraBackup guide provides a detailed instructions on how to take the backup and set up the replication.

        For checking the backup consistency you can use either the original server where the backup was taken, or another test server created by using a different backup method (such as cold backup, mysqldump or LVM snapshots) as the source server in the replication setup.

        "},{"location":"varify-backup.html#use-pt-table-checksum","title":"Use pt-table-checksum","text":"

        This tool is part of the Percona Toolkit. It performs an online replication consistency check by executing checksum queries on the source, which produces different results on replicas that are inconsistent with the source.

        After you confirmed that replication has been set up successfully, you can install or download pt-table-checksum. This example shows downloading the latest version of pt-table-checksum:

        $ wget percona.com/get/pt-table-checksum\n

        Note

        In order for pt-table-checksum to work correctly libdbd-mysql-perl will need to be installed on Debian/Ubuntu systems or perl-DBD-MySQL on RHEL/CentOS. If you installed the percona-toolkit package from the Percona repositories package manager should install those libraries automatically.

        After this command has been run, pt-table-checksum will be downloaded to your current working directory.

        Running the pt-table-checksum on the source will create percona database with the checksums table which will be replicated to the replicas as well. Example of the pt-table-checksum will look like this:

        $ ./pt-table-checksum\n

        The results are similar to the following:

        Expected output 
        TS ERRORS  DIFFS     ROWS  CHUNKS SKIPPED    TIME TABLE\n04-30T11:31:50      0      0   633135       8       0   5.400 exampledb.aka_name\n04-30T11:31:52      0      0   290859       1       0   2.692 exampledb.aka_title\nChecksumming exampledb.user_info:  16% 02:27 remain\nChecksumming exampledb.user_info:  34% 01:58 remain\nChecksumming exampledb.user_info:  50% 01:29 remain\nChecksumming exampledb.user_info:  68% 00:56 remain\nChecksumming exampledb.user_info:  86% 00:24 remain\n04-30T11:34:38      0      0 22187768     126       0 165.216 exampledb.user_info\n04-30T11:38:09      0      0        0       1       0   0.033 mysql.time_zone_name\n04-30T11:38:09      0      0        0       1       0   0.052 mysql.time_zone_transition\n04-30T11:38:09      0      0        0       1       0   0.054 mysql.time_zone_transition_type\n04-30T11:38:09      0      0        8       1       0   0.064 mysql.user\n

        If all the values in the DIFFS column are 0 that means that backup is consistent with the current setup.

        In case backup was not consistent pt-table-checksum should spot the difference and point to the table that does not match. Following example shows adding new user on the backed up replica in order to simulate the inconsistent backup:

        mysql> grant usage on exampledb.* to exampledb@localhost identified by 'thisisnewpassword';\n

        If we run the pt-table-checksum now difference should be spotted

        $ ./pt-table-checksum\n

        The results are similar to the following:

        Expected output 
        TS ERRORS  DIFFS     ROWS  CHUNKS SKIPPED    TIME TABLE\n04-30T11:31:50      0      0   633135       8       0   5.400 exampledb.aka_name\n04-30T11:31:52      0      0   290859       1       0   2.692 exampledb.aka_title\nChecksumming exampledb.user_info:  16% 02:27 remain\nChecksumming exampledb.user_info:  34% 01:58 remain\nChecksumming exampledb.user_info:  50% 01:29 remain\nChecksumming exampledb.user_info:  68% 00:56 remain\nChecksumming exampledb.user_info:  86% 00:24 remain\n04-30T11:34:38      0      0 22187768     126       0 165.216 exampledb.user_info\n04-30T11:38:09      0      0        0       1       0   0.033 mysql.time_zone_name\n04-30T11:38:09      0      0        0       1       0   0.052 mysql.time_zone_transition\n04-30T11:38:09      0      0        0       1       0   0.054 mysql.time_zone_transition_type\n04-30T11:38:09      1      0        8       1       0   0.064 mysql.user\n

        This output shows that source and the replica aren\u2019t in consistent state and that the difference is in the mysql.user table.

        More information on different options that pt-table-checksum provides can be found in the pt-table-checksum documentation.

        "},{"location":"varify-backup.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"work-with-apparmor.html","title":"Work with AppArmor","text":"

        The Linux Security Module implements mandatory access controls (MAC) with AppArmor. Debian and Ubuntu systems install AppArmor by default. AppArmor uses profiles which define which files and permissions are needed for application.

        Percona XtraBackup does not have a profile and is not confined by AppArmor.

        For a list of common AppArmor commands, see Percona Server for MySQL - AppArmor.

        "},{"location":"work-with-apparmor.html#develop-a-profile","title":"Develop a profile","text":"

        Download the profile from:

        https://github.com/percona/percona-xtrabackup/tree/trunk/packaging/percona/apparmor/apparmor.d

        The following profile sections should be updated with your system information, such as location of the backup destination directory.

        Expected output 
        # enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\n\n# enable storing backups only in /backups directory\n# /backups/** rwk,\n\n# enable storing backups anywhere in caller user home directory\n/@{HOME}/** rwk,\n}\n

        Move the updated file:

        $ sudo mv usr.sbin.xtrabackup /etc/apparmor.d/\n

        Install the profile with the following command:

        $ sudo apparmor_parser -r -T -W /etc/apparmor.d/usr.sbin.xtrabackup\n

        Run the backup as usual.

        No additional AppArmor-related actions are required to restore a backup.

        "},{"location":"work-with-apparmor.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"work-with-selinux.html","title":"Work with SELinux","text":"

        Percona XtraBackup is installed as an unconfined process running in an undefined domain. SELinux allows unconfined processes almost all access and the processes only use Discretionary Access Control (DAC) rules.

        You find the current state of the Percona XtraBackup file with the following command:

        $ ls -Z /usr/bin | grep xtrabackup\n
        Expected output 
        -rwxr-xr-x. root root   system_u:object_r:bin_t:s0       xtrabackup\n

        The SELinux context is the following:

        • user (root)

        • role (object_r)

        • type (bin_t)

        • level (s0)

        The unconfined domain supports the network-facing services, which are protected by SELinux. These domains are not exposed. In this configuration, SELinux protects against remote intrusions but local intrusions, which require local access, are not confined.

        Percona XtraBackup works locally. The service is not network-facing and cannot be exploited externally. The service interacts only with the local user, who provides the parameters. Percona XtraBackup requires access to the target-dir location.

        "},{"location":"work-with-selinux.html#confine-xtrabackup","title":"Confine XtraBackup","text":"

        You can modify your security configuration to confine Percona XtraBackup. The first question is where to store the backup files. The service requires read and write access to the selected location.

        You can use either of the following methods:

        • Allow Percona XtraBackup to write to any location. The user provides any path to the target-dir parameter.

        • Allow Percona XtraBackup to write to a specific location, such as /backups or the user\u2019s home directory.

        The first option opens the entire system to read and write. Select the second option to harden your security.

        "},{"location":"work-with-selinux.html#install-selinux-tools","title":"Install SELinux tools","text":"

        To work with policies, you must install the SELinux tools. To find which package provides the semanage command and install the package. The following is an example on CentOS 7.

        $ yum provides *bin/semanage\n
        The result should list the packages.

        Expected output 
        ...\npolicycoreutils-python-2.5-34.el7.x86_64 : SELinux policy core python utilities\n...\n

        To install missing packages, run the following:

        $ sudo yum install -y policycoreutils-python\n

        The following is an example on CentOS 8:

        $ yum provides *bin/semanage\n
        The result should list the missing packages.

        Expected output 
        ...\npolicycoreutils-python-utils-2.8-16.1.el8.noarch : SELinux policy core python utilities\n

        Run the following to install the missing packages: 

        $ sudo yum install -y policycoreutils-python-utils\n
        "},{"location":"work-with-selinux.html#create-a-policy","title":"Create a policy","text":"

        Use a modular approach to create an SELinux policy. Create a policy module to manage XtraBackup. You must create a .te file for type enforcement, and an optional .fc file for the file contexts.

        Use $ ps -efZ | grep xtrabackup to verify the service is not confined by SELinux.

        Create the xtrabackup.fc file and add content. This file defines the security contexts.

        /usr/bin/xtrabackup    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcrypt    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbstream    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/usr/bin/xbcloud    -- gen_context(system_u:object_r:xtrabackup_exec_t,s0)\n/backups(/.*)?       system_u:object_r:xtrabackup_data_t:s0\n

        Note

        If you are using the /backups directory you must have the last line. If you are storing the backups in the user\u2019s home directory, you can omit this line.

        Download the xtrabackup.te file from the following location:

        https://github.com/percona/percona-xtrabackup/tree/trunk/packaging/percona/selinx

        Note

        In the file, the sections in bold should be modified for your system. The fc file can also be downloaded from the same location.

        Compile the policy module:

        $ make -f /usr/share/selinux/devel/Makefile xtrabackup.pp\n

        Install the module:

        $ semodule -i xtrabackup.pp\n

        Tag the PXB binaries with the proper SELinux tags, such as xtrabackup_exec_t.

        $ restorecon -v /usr/bin/*\n

        If you store your backups at /backups, restore the tag in that location:

        $ restorecon -v /backups\n

        Note

        Remember to add the standard Linux DAC permissions for this directory.

        Perform the backup in the standard way.

        "},{"location":"work-with-selinux.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"working-with-binary-logs.html","title":"Work with binary logs","text":"

        The xtrabackup binary integrates with the log_status table. This integration enables xtrabackup to print out the backup\u2019s corresponding binary log position, so that you can use this binary log position to provision a new replica or perform point-in-time recovery.

        "},{"location":"working-with-binary-logs.html#find-the-binary-log-position","title":"Find the binary log position","text":"

        You can find the binary log position corresponding to a backup after the backup has been taken. If your backup is from a server with binary logging enabled, xtrabackup creates a file named xtrabackup_binlog_info in the target directory. This file contains the binary log file name and position of the exact point when the backup was taken.

        Expected output during the backup stage 
        210715 14:14:59 Backup created in directory '/backup/'\nMySQL binlog position: filename 'binlog.000002', position '156'\n. . .\n210715 14:15:00 completed OK!\n
        "},{"location":"working-with-binary-logs.html#point-in-time-recovery","title":"Point-in-time recovery","text":"

        To perform a point-in-time recovery from an xtrabackup backup, you should prepare and restore the backup, and then replay binary logs from the point shown in the xtrabackup_binlog_info file.

        Find a more detailed procedure in the Point-in-time recovery document.

        "},{"location":"working-with-binary-logs.html#set-up-a-new-replication-replica","title":"Set up a new replication replica","text":"

        To set up a new replica, you should prepare the backup, and restore it to the data directory of your new replication replica. In the CHANGE_REPLICATION_SOURCE_TO with the appropriate options command, use the binary log filename and position shown in the xtrabackup_binlog_info file to start replication.

        A more detailed procedure is found in How to setup a replica for replication in 6 simple steps with Percona XtraBackup.

        "},{"location":"working-with-binary-logs.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"xbcloud-azure.html","title":"Use the xbcloud binary with Microsoft Azure Cloud Storage","text":"

        The xbcloud binary adds support for the Microsoft Azure Cloud Storage using the REST API.

        "},{"location":"xbcloud-azure.html#options","title":"Options","text":"

        The following are the options, environment variables, and descriptions for uploading a backup to Azure using the REST API. The environment variables are recognized by xbcloud, which maps them automatically to the corresponding parameters:

        Option name Environment variables Description \u2013azure-storage-account=name AZURE_STORAGE_ACCOUNT An Azure storage account is a unique namespace to access and store your Azure data objects. \u2013azure-container-name=name AZURE_CONTAINER_NAME A container name is a valid DNS name that conforms to the Azure naming rules \u2013azure-access-key=name AZURE_ACCESS_KEY A generated key that can be used to authorize access to data in your account using the Shared Key authorization. \u2013azure-endpoint=name AZURE_ENDPOINT The endpoint allows clients to securely access data \u2013azure-tier-class=name AZURE_STORAGE_CLASS Cloud tier can decrease the local storage required while maintaining the performance. When enabled, this feature has the following categories: Hot - Frequently accessed or modified data Cool - Infrequently accessed or modified data Archive - Rarely accessed or modified data

        Test your Azure applications with the Azurite open-source emulator. For testing purposes, the xbcloud binary adds the --azure-development-storage option that uses the default access_key and storage account of azurite and testcontainer for the container. You can overwrite these options, if needed.

        "},{"location":"xbcloud-azure.html#usage","title":"Usage","text":"

        All the available options for xbcloud, such as parallel, max-retries, and others, can be used. For more information, see the xbcloud binary overview.

        "},{"location":"xbcloud-azure.html#examples","title":"Examples","text":"

        An example of an xbcloud backup.

        $ xtrabackup --backup --stream=xbstream  | \nxbcloud put backup_name --azure-storage-account=pxbtesting --azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\n

        An example of restoring a backup from xbcloud.

        $ xbcloud get backup_name  --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure --parallel=10 2>download.log | xbstream -x -C restore\n

        An example of deleting a backup from xbcloud.

        $ xbcloud delete backup_name --azure-storage-account=pxbtesting \n--azure-access-key=$AZURE_KEY --azure-container-name=test --storage=azure\n

        An example of using a shortcut restore.

        $ xbcloud get azure://operator-testing/bak22 ...\n
        "},{"location":"xbcloud-azure.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"xbcloud-binary-fifo-datasink.html","title":"FIFO data sink","text":"

        The feature is in tech preview.

        Percona XtraBackup implements a data sink that uses the first in, first out (FIFO) method. With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to an object storage.

        "},{"location":"xbcloud-binary-fifo-datasink.html#fifo-data-sink-options","title":"FIFO data sink options","text":"

        Percona XtraBackup implements the following options:

        • --fifo-streams=# - specifies the number of FIFO files to use for parallel data stream. To disable FIFO data sink and send stream to STDOUT, set --fifo-streams=1. The default value is 1 (STDOUT) to ensure the backward compatibility. The --fifo-streams value must match on both the XtraBackup and xbcloud sides.
        • --fifo-dir=name - specifies a directory to write Named Pipe.
        • --fifo-timeout=# - specifies the number of seconds to wait for the other end to open the stream for reading. The default value is 60 seconds.
        "},{"location":"xbcloud-binary-fifo-datasink.html#how-to-enable-fifo-data-sink","title":"How to enable FIFO data sink","text":"

        To use FIFO data sink, you can either run two commands in separate terminal sessions or run xtrabackup in the background.

        For example, run the following commands in separate terminal sessions:

        $ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo\n
        $ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\n

        Run xtrabackup in the background with the following commands:

        $ xtrabackup --backup --stream --fifo-streams=2 --fifo-dir=/tmp/fifo &\n$ xbcloud put --fifo-streams=2 --fifo-dir=/tmp/fifo full\n
        "},{"location":"xbcloud-binary-fifo-datasink.html#stream-to-an-object-storage","title":"Stream to an object storage","text":"

        When taking a backup, you can save the files locally or stream the files to either a different server or an object storage.

        When you stream backups to Amazon S3 compatible storage using LAN with a streaming capacity of 10Gbps, XtraBackup can use multiple FIFO streams to stream the backups faster.

        XtraBackup spawns multiple copy threads and each copy thread reads a data chunk from a specific file. Then multiple FIFO files are created to store the data from XtraBackup. Each XtraBackup copy thread writes the data chunks to a specific FIFO file. Xbcloud reads from the FIFO streams and uploads data to an object storage using an async request. The xbcloud event handler executes the callback depending on the response from the object storage (success or failure).

        "},{"location":"xbcloud-binary-fifo-datasink.html#performance-improvement","title":"Performance improvement","text":"

        Consider an example of using a FIFO data sink compared to the STDOUT method.

        The database has 1TB of data in multiple tables. The link speed between the source server and destination server using MinIO is ~ 9.2 Gbps.

        Both STDOUT and FIFO data sink scenarios push 1TB of data from two servers.

        For the FIFO data sink we configure 8 parallel streams with --fifo-streams=8 both for XtraBackup and xbcloud.

        The results are the following:

        • The STDOUT method takes 01:25:24 to push 1TB of data using 239 MBps (1.8 Gbps).
        • The FIFO method, using 8 streams, takes 00:16:01 to push 1TB of data using 1.15 GBps (9.2 Gbps).
        "},{"location":"xbcloud-binary-fifo-datasink.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"xbcloud-binary-overview.html","title":"The xbcloud binary overview","text":"

        The purpose of xbcloud is to download from the cloud and upload to the cloud the full or part of an xbstream archive. xbcloud will not overwrite the backup with the same name. xbcloud accepts input via a pipe from xbstream so that it can be invoked as a pipeline with xtrabackup to stream directly to the cloud without needing a local storage.

        Note

        In a Bash shell, the $? parameter returns the exit code from the last binary. If you use pipes, the ${PIPESTATUS[x]} array parameter returns the exit code for each binary in the pipe string.

        $ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ | xbcloud put [options] full_backup\n...\n$ ${PIPESTATUS[x]}\n0 0\n$ true | false\n$ echo $?\n1\n\n# with PIPESTATUS\n$ true | false\n$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}\n0 1\n

        The xbcloud binary stores each chunk as a separate object with a name backup_name/database/table.ibd.NNN..., where NNN... is a 0-padded serial number of chunk within a file. The size of chunk produced by xtrabackup and xbstream changed to 10M.

        To adjust the chunk size use --read-buffer-size. To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.

        xbcloud has three essential operations: put, get, and delete. With these operations, backups are created, stored, retrieved, restored, and deleted. xbcloud operations clearly map to similar operations within the AWS Amazon S3 API.

        The Exponential Backoff feature increases the chances for the completion of a backup or a restore operation. When taking a backup, a chunk upload or download may fail if you have an unstable network connection or other network issues. This feature adds an exponential backoff, a sleep time, and retries the operations.

        With the FIFO data sink feature, users with a streaming capacity of 10Gbps (typically on a Local Area Network (LAN)) can benefit from faster backups by streaming data in parallel to object storage.

        Important

        To prevent intermittent backup failures, update the curl utility in Debian 10.

        "},{"location":"xbcloud-binary-overview.html#supported-cloud-storage-types","title":"Supported cloud storage types","text":"

        The following cloud storage types are supported:

        • OpenStack Object Storage (Swift) - see Using the xbcloud binary with Swift

        • Amazon Simple Storage (S3) - see Using the xbcloud binary with Amazon S3

        • Azure Cloud Storage - see Using the xbcloud binary with Microsoft Azure Cloud Storage

        • Google Cloud Storage (gcs) - see Using the xbcloud binary with Google Cloud Storage

        • MinIO - see Using the xbcloud binary with MinIO

        In addition to OpenStack Object Storage (Swift), which has been the only option for storing backups in a cloud storage until Percona XtraBackup 2.4.14, xbcloud supports Amazon S3, MinIO, and Google Cloud Storage. Other Amazon S3-compatible storages, such as Wasabi or Digital Ocean Spaces, are also supported.

        See also

        OpenStack Object Storage(\u201cSwift\u201d)

        Amazon Simple Storage Service

        MinIO

        Google Cloud Storage

        Wasabi

        Digital Ocean Spaces

        "},{"location":"xbcloud-binary-overview.html#usage","title":"Usage","text":"

        The following sample command creates a full backup:

        $ xtrabackup --backup --stream=xbstream --target-dir=/storage/backups/ --extra-lsndirk=/storage/backups/| xbcloud \\\nput [options] full_backup\n

        An incremental backup only includes the changes since the last backup. The last backup can be either a full or incremental backup.

        The following sample command creates an incremental backup:

        $ xtrabackup --backup --stream=xbstream --incremental-basedir=/storage/backups \\\n--target-dir=/storage/inc-backup | xbcloud  put [options] inc_backup\n

        To prepare an incremental backup, you must first download the full backup with the following command:

        $ xtrabackup get [options] full_backup | xbstream -xv -C /tmp/full-backup\n

        You must prepare the full backup:

        $ xtrabackup --prepare --apply-log-only --target-dir=/tmp/full-backup\n

        After the full backup has been prepared, download the incremental backup:

        xbcloud get [options] inc_backup | xbstream -xv -C /tmp/inc-backup\n

        The downloaded backup is prepared by running the following command:

        $ xtrabackup --prepare --target-dir=/tmp/full-backup --incremental-dir=/tmp/inc-backup\n

        You do not need the full backup to restore only a specific database. You can specify only the tables to be restored:

        xbcloud get [options] ibdata1 sakila/payment.ibd /tmp/partial/partial.xbs\n
        An example of the code: 

        xbstream -xv -C /tmp/partial < /tmp/partial/partial.xbs\n
        "},{"location":"xbcloud-binary-overview.html#supplying-parameters","title":"Supplying parameters","text":"

        Each storage type has mandatory parameters that you can supply on the command line, in a configuration file, and via environment variables.

        "},{"location":"xbcloud-binary-overview.html#configuration-files","title":"Configuration files","text":"

        The parameters the values of which do not change frequently can be stored in my.cnf or in a custom configuration file. The following example is a template of configuration options under the [xbcloud] group:

        [xbcloud]\nstorage=s3\ns3-endpoint=http://localhost:9000/\ns3-access-key=minio\ns3-secret-key=minio123\ns3-bucket=backupsx\ns3-bucket-lookup=path\ns3-api-version=4\n

        Note

        If you explicitly use a parameter on the command line and in a configuration file, xbcloud uses the value provided on the command line.

        "},{"location":"xbcloud-binary-overview.html#environment-variables","title":"Environment variables","text":"

        If you explicitly use a parameter on the command line, in a configuration file, and the corresponding environment variable contains a value, xbcloud uses the value provided on the command line or in the configuration file.

        "},{"location":"xbcloud-binary-overview.html#shortcuts","title":"Shortcuts","text":"

        For all operations (put, get, and delete), you can use a shortcut to specify the storage type, bucket name, and backup name as one parameter instead of using three distinct parameters (\u2013storage, \u2013s3-bucket, and backup name per se).

        Note

        Use the following format: storage-type://bucket-name/backup-name

        In this example s3 refers to a storage type, operator-testing is a bucket name, and bak22 is the backup name. 

        $ xbcloud get s3://operator-testing/bak22 ...\n

        This shortcut expands as follows:

        $ xbcloud get --storage=s3 --s3-bucket=operator-testing bak22 ...\n

        You can supply the mandatory parameters on the command line, configuration files, and in environment variables.

        "},{"location":"xbcloud-binary-overview.html#additional-parameters","title":"Additional parameters","text":"

        xbcloud accepts additional parameters that you can use with any storage type. The --md5 parameter computes the MD5 hash value of the backup chunks. The result is stored in files that following the backup_name.md5 pattern.

        $ xtrabackup --backup --stream=xbstream \\\n--parallel=8 2>backup.log | xbcloud put s3://operator-testing/bak22 \\\n--parallel=8 --md5 2>upload.log\n

        You may use the --header parameter to pass an additional HTTP header with the server side encryption while specifying a customer key.

        An example of using the --header for AES256 encryption.

        $ xtrabackup --backup --stream=xbstream --parallel=4 | \\\nxbcloud put s3://operator-testing/bak-enc/ \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key: CuStoMerKey=\" \\\n--header=\"X-Amz-Server-Side-Encryption-Customer-Key-MD5: CuStoMerKeyMd5==\" \\\n--parallel=8\n

        The --header parameter is also useful to set the access control list (ACL) permissions: --header=\"x-amz-acl: bucket-owner-full-control

        "},{"location":"xbcloud-binary-overview.html#incremental-backups","title":"Incremental backups","text":"

        First, you need to make the full backup on which the incremental one is going to be based:

        $ xtrabackup --backup --stream=xbstream --extra-lsndir=/storage/backups/ \\\n--target-dir=/storage/backups/ | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup\n

        Then you can make the incremental backup:

        $ xtrabackup --backup --incremental-basedir=/storage/backups \\\n--stream=xbstream --target-dir=/storage/inc_backup | xbcloud put \\\n--storage=swift --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup\n
        "},{"location":"xbcloud-binary-overview.html#preparing-incremental-backups","title":"Preparing incremental backups","text":"

        To prepare a backup you first need to download the full backup:

        $ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\nfull_backup | xbstream -xv -C /storage/downloaded_full\n

        Once you download the full backup it should be prepared:

        $ xtrabackup --prepare --apply-log-only --target-dir=/storage/downloaded_full\n

        After the full backup has been prepared you can download the incremental backup:

        $ xbcloud get --swift-container=test_backup \\\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ --parallel=10 \\\ninc_backup | xbstream -xv -C /storage/downloaded_inc\n

        Once the incremental backup has been downloaded you can prepare it by running:

        $ xtrabackup --prepare --apply-log-only \\\n--target-dir=/storage/downloaded_full \\\n--incremental-dir=/storage/downloaded_inc\n\n$ xtrabackup --prepare --target-dir=/storage/downloaded_full\n
        "},{"location":"xbcloud-binary-overview.html#partial-download-of-the-cloud-backup","title":"Partial download of the cloud backup","text":"

        If you do not want to download the entire backup to restore the specific database you can specify only the tables you want to restore:

        $ xbcloud get --swift-container=test_backup\n--swift-auth-version=2.0 --swift-user=admin \\\n--swift-tenant=admin --swift-password=xoxoxoxo \\\n--swift-auth-url=http://127.0.0.1:35357/ full_backup \\\nibdata1 sakila/payment.ibd \\\n> /storage/partial/partial.xbs\n\n$ xbstream -xv -C /storage/partial < /storage/partial/partial.xbs\n
        "},{"location":"xbcloud-binary-overview.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"xbcloud-exbackoff.html","title":"Exponential backoff","text":"

        Exponential backoff increases the chances for the completion of a backup or a restore operation. For example, a chunk upload or download may fail if you have an unstable network connection or other network issues. This feature adds an exponential backoff, or sleep, time and then retries the upload or download.

        When a chunk upload or download operation fails, xbcloud checks the reason for the failure. This failure can be a CURL error or an HTTP error, or a client-specific error. If the error is listed in the Retriable errors list, xbcloud pauses for a calculated time before retrying the operation until that time reaches the --max-backoff value.

        The operation is retried until the --max-retries value is reached. If the chunk operation fails on the last retry, xbcloud aborts the process.

        The default values are the following:

        • \u2013max-backoff = 300000 (5 minutes)

        • \u2013max-retries = 10

        You can adjust the number of retries by adding the --max-retries parameter and adjust the maximum length of time between retries by adding the --max-backoff parameter to an xbcloud command.

        Since xbcloud does multiple asynchronous requests in parallel, a calculated value, measured in milliseconds, is used for max-backoff. This algorithm calculates how many milliseconds to sleep before the next retry. A number generated is based on the combining the power of two (2), the number of retries already attempted and adds a random number between 1 and 1000. This number avoids network congestion if multiple chunks have the same backoff value. If the default values are used, the final retry attempt to be approximately 17 minutes after the first try. The number is no longer calculated when the milliseconds reach the --max-backoff setting. At that point, the retries continue by using the --max-backoff setting until the max-retries parameter is reached.

        "},{"location":"xbcloud-exbackoff.html#retriable-errors","title":"Retriable errors","text":"

        We retry for the following CURL operations:

        • CURLE_GOT_NOTHING

        • CURLE_OPERATION_TIMEOUT

        • CURLE_RECV_ERROR

        • CURLE_SEND_ERROR

        • CURLE_SEND_FAIL_REWIND

        • CURLE_PARTIAL_FILE

        • CURLE_SSL_CONNECT_ERROR

        We retry for the following HTTP operation status codes:

        • 503

        • 500

        • 504

        • 408

        Each cloud provider may return a different CURL error or an HTTP error, depending on the issue. Add new errors by setting the following variables --curl-retriable-errors or --http-retriable-errors on the command line or in my.cnf or in a custom configuration file under the [xbcloud] section. You can add multiple errors using a comma-separated list of codes.

        The error handling is enhanced when using the --verbose output. This output specifies which error caused xbcloud to fail and what parameter a user must add to retry on this error.

        The following is an example of a verbose output:

        Expected output 
        210701 14:34:23 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210701 14:34:23 /work/pxb/ins/8.1/bin/xbcloud: Curl error (52) Server returned nothing (no headers, no data) is not configured as retriable. You can allow it by adding --curl-retriable-errors=52 parameter\n
        "},{"location":"xbcloud-exbackoff.html#example","title":"Example","text":"

        The following example adjusts the maximum number of retries and the maximum time between retries.

        xbcloud [options] --max-retries=5 --max-backoff=10000\n

        The following list describes the process using --max-backoff=10000:

        • The chunk xtrabackup_logfile.00000000000000000006 fails to upload the first time and sleeps for 2384 milliseconds.

        • The same chunk fails for the second time and the sleep time is increased to 4387 milliseconds.

        • The same chunk fails for the third time and the sleep time is increased to 8691 milliseconds.

        • The same chunk fails for the fourth time. The max-backoff parameter has been reached. All retries sleep the same amount of time after reaching the parameter.

        • The same chunk is successfully uploaded.

        An example of the output for this setting 
        210702 10:07:05 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:05 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 2384 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:23 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Server returned nothing (no headers, no data)\n210702 10:07:23 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 4387 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:07:52 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:07:52 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 8691 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:08:47 /work/pxb/ins/8.1/bin/xbcloud: Operation failed. Error: Failed sending data to the peer\n210702 10:08:47 /work/pxb/ins/8.1/bin/xbcloud: Sleeping for 10000 ms before retrying backup3/xtrabackup_logfile.00000000000000000006\n. . .\n210702 10:10:12 /work/pxb/ins/8.1/bin/xbcloud: successfully uploaded chunk: backup3/xtrabackup_logfile.00000000000000000006, size: 8388660\n
        "},{"location":"xbcloud-exbackoff.html#get-expert-help","title":"Get expert help","text":"

        If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

        Community Forum Get a Percona Expert

        "},{"location":"xbcloud-gcs.html","title":"Use the xbcloud binary with Google Cloud Storage","text":""},{"location":"xbcloud-gcs.html#create-a-full-backup-with-google-cloud-storage","title":"Create a full backup with Google Cloud Storage","text":"

        The support for Google Cloud Storage is implemented using the interoperability mode. This mode was especially designed to interact with cloud services compatible with Amazon S3.

        See also

        Cloud Storage Interoperability

        $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=google \\\n--google-endpoint=`storage.googleapis.com` \\\n--google-access-key='YOUR-ACCESSKEYID' \\\n--google-secret-key='YOUR-SECRETACCESSKEY' \\\n--google-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n

        The following options are available when using Google Cloud Storage:

        • \u2013google-access-key =

        • \u2013google-secret-key =

        • \u2013google-bucket =

        • \u2013google-storage-class=name

          • Note

          The Google storage class name options are the following:

          • STANDARD

          • NEARLINE

          • COLDLINE

          • ARCHIVE

          See also

          Google storage classes - the default Google storage class depends on the storage class of the bucket

          "},{"location":"xbcloud-gcs.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xbcloud-iam-profile.html","title":"Use the xbcloud binary with an IAM instance profile","text":"

          You can use the IAM instance profile when running xbcloud from an EC2 instance.

          An authentication system has two elements:

          • Who am I?
          • What can I do?

          A role defines \u201cwhat can I do.\u201d A role provides a method to define a collection of permissions. Roles are assigned to users, services and EC2 instances, the \u201cwho am I\u201d element.

          The IAM instance profile is the \u201cwho\u201d for an EC2 instance and assumes the IAM role, which has permissions. The instance profile has the same name as the IAM role.

          An IAM instance profile does not need the --s3-secret-key nor the --s3-access-key if they are running xbcloud from an Amazon EC2 instance. To configure or attach an instance metadata to an EC2 instance, see How can I grant my Amazon EC2 instance access to an Amazon S3 bucket.

          An example of the command:

          $ xtrabackup ... | xbcloud put --storage=s3 --s3-bucket=bucket-name backup-name\n

          The xbcloud binary outputs a connect message when successful.

          Expected output 
          221121 13:16:26 Using instance metadata for access and secret key\n221121 13:16:26 xbcloud: Successfully connected.\n

          An important consideration is that the instance metadata has a time to live (TTL) of 6 hours. A backup that takes more than that time contains Expired token errors. Use Exponential Backoff to retry the upload after fetching new keys from the instance metadata.

          Output when keys have expired 
          221121 13:04:52 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:52 xbcloud: Sleeping for 2384 ms before retrying test/mysql.ibd.00000000000000000002 [1]\n221121 13:04:55 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:55 xbcloud: Sleeping for 2887 ms before retrying test/mysql.ibd.00000000000000000003 [1]\n221121 13:04:58 xbcloud: S3 error message: The provided token has expired.\n221121 13:04:58 xbcloud: Sleeping for 2778 ms before retrying test/undo_002.00000000000000000000 [1]\n221121 13:05:00 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:00 xbcloud: Sleeping for 2916 ms before retrying test/undo_002.00000000000000000001 [1]\n221121 13:05:03 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:03 xbcloud: Sleeping for 2794 ms before retrying test/undo_002.00000000000000000002 [1]\n221121 13:05:06 xbcloud: S3 error message: The provided token has expired.\n221121 13:05:06 xbcloud: Sleeping for 2336 ms before retrying test/undo_001.00000000000000000000 [1]\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000002, size: 5242923\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/mysql.ibd.00000000000000000003, size: 23\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000000, size: 10485802\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000001, size: 6291498\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_002.00000000000000000002, size: 22\n221121 13:05:09 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000000, size: 10485802\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000001, size: 6291498\n221121 13:05:10 xbcloud: successfully uploaded chunk: test/undo_001.00000000000000000002, size: 22\n. . .\n221121 13:05:18 xbcloud: successfully uploaded chunk: test/xtrabackup_tablespaces.00000000000000000001, size: 36\n221121 13:05:19 xbcloud: Upload completed. \n
          "},{"location":"xbcloud-iam-profile.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xbcloud-minio.html","title":"Use the xbcloud binary with MinIO","text":""},{"location":"xbcloud-minio.html#create-a-full-backup-with-minio","title":"Create a full backup with MinIO","text":"
          $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='play.minio.io:9000' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n
          "},{"location":"xbcloud-minio.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xbcloud-s3.html","title":"Use the xbcloud binary with Amazon S3","text":""},{"location":"xbcloud-s3.html#create-a-full-backup-with-amazon-s3","title":"Create a full backup with Amazon S3","text":"
          $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=s3 \\\n--s3-endpoint='s3.amazonaws.com' \\\n--s3-access-key='YOUR-ACCESSKEYID' \\\n--s3-secret-key='YOUR-SECRETACCESSKEY' \\\n--s3-bucket='mysql_backups'\n--parallel=10 \\\n$(date -I)-full_backup\n

          The following options are available when using Amazon S3:

          Option Details \u2013s3-access-key Use to supply the AWS access key ID \u2013s3-secret-key Use to supply the AWS secret access key \u2013s3-bucket Use supply the AWS bucket name \u2013s3-region Use to specify the AWS region. The default value is us-east-1 \u2013s3-api-version = Select the signing algorithm. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-bucket-lookup = Specify whether to use bucket.endpoint.com or endpoint.com/bucket*style requests. The default value is AUTO. In this case, xbcloud will probe. \u2013s3-storage-class= Specify the S3 storage class. The default storage class depends on the provider. The name options are the following:
          • STANDARD
          • STANDARD_IA
          • GLACIER
          NOTE If you use the GLACIER storage class, the object must be restored to S3 before restoring the backup. Also supports using custom S3 implementations such as MinIO or CephRadosGW."},{"location":"xbcloud-s3.html#permissions-setup","title":"Permissions setup","text":"

          Following the principle of \u201cleast-privilege\u201d, these are the minimum bucket permissions needed for xbcloud to write backups to S3: LIST/PUT/GET/DELETE.

          The following example shows the policy definition for writing to the xbcloud-testing bucket on the AWS S3 storage.

          {\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"s3:ListBucket\"\n            ],\n            \"Resource\": \"arn:aws:s3:::xbcloud-testing\"\n        },\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"s3:PutObject\",\n                \"s3:PutObjectAcl\",\n                \"s3:GetObject\",\n                \"s3:GetObjectAcl\",\n                \"s3:DeleteObject\"\n            ],\n            \"Resource\": \"arn:aws:s3:::xbcloud-testing/*\"\n        }\n    ]\n}\n
          "},{"location":"xbcloud-s3.html#environment-variables","title":"Environment variables","text":"

          The following environment variables are recognized. xbcloud maps them automatically to corresponding parameters applicable to the selected storage.

          • AWS_ACCESS_KEY_ID (or ACCESS_KEY_ID)

          • AWS_SECRET_ACCESS_KEY (or SECRET_ACCESS_KEY)

          • AWS_DEFAULT_REGION (or DEFAULT_REGION)

          • AWS_ENDPOINT (or ENDPOINT)

          • AWS_CA_BUNDLE

          "},{"location":"xbcloud-s3.html#restore-with-s3","title":"Restore with S3","text":"
          $ xbcloud get s3://operator-testing/bak22 \\\n--s3-endpoint=https://storage.googleapis.com/ \\\n--parallel=10 2>download.log | xbstream -x -C restore --parallel=8\n
          "},{"location":"xbcloud-s3.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xbcloud-swift.html","title":"Use the xbcloud binary with Swift","text":""},{"location":"xbcloud-swift.html#create-a-full-backup-with-swift","title":"Create a full backup with Swift","text":"

          The following example shows how to make a full backup and upload it to Swift.

          $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \\\nxbcloud put --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\n--parallel=10 \\\nfull_backup\n

          The following OpenStack environment variables are also recognized and mapped automatically to the corresponding swift parameters (--storage=swift):

          • OS_AUTH_URL

          • OS_TENANT_NAME

          • OS_TENANT_ID

          • OS_USERNAME

          • OS_PASSWORD

          • OS_USER_DOMAIN

          • OS_USER_DOMAIN_ID

          • OS_PROJECT_DOMAIN

          • OS_PROJECT_DOMAIN_ID

          • OS_REGION_NAME

          • OS_STORAGE_URL

          • OS_CACERT

          "},{"location":"xbcloud-swift.html#restore-with-swift","title":"Restore with Swift","text":"
          $ xbcloud get [options] <name> [<list-of-files>] | xbstream -x\n

          The following example shows how to fetch and restore the backup from Swift:

          $ xbcloud get --storage=swift \\\n--swift-container=test \\\n--swift-user=test:tester \\\n--swift-auth-url=http://192.168.8.80:8080/ \\\n--swift-key=testing \\\nfull_backup | xbstream -xv -C /tmp/downloaded_full\n\n$ xbcloud delete --storage=swift --swift-user=xtrabackup \\\n--swift-password=xtrabackup123! --swift-auth-version=3 \\\n--swift-auth-url=http://openstack.ci.percona.com:5000/ \\\n--swift-container=mybackup1 --swift-domain=Default\n
          "},{"location":"xbcloud-swift.html#command-line-options","title":"Command-line options","text":"

          xbcloud has the following command line options:

          "},{"location":"xbcloud-swift.html#-storageswifts3google","title":"\u2013storage(=[swift|s3|google])","text":"

          Cloud storage option. xbcloud supports Swift, MinIO, and AWS S3. The default value is swift.

          "},{"location":"xbcloud-swift.html#-swift-auth-url","title":"\u2013swift-auth-url()","text":"

          The URL of the Swift cluster

          "},{"location":"xbcloud-swift.html#-swift-storage-url","title":"\u2013swift-storage-url()","text":"

          The xbcloud tries to get the object-store URL for a given region (if any are specified) from the keystone response. You can override that URL by passing \u2013swift-storage-url=URL argument.

          "},{"location":"xbcloud-swift.html#-swift-user","title":"\u2013swift-user()","text":"

          The Swift username (X-Auth-User, specific to Swift)

          "},{"location":"xbcloud-swift.html#-swift-key","title":"\u2013swift-key()","text":"

          The Swift key/password (X-Auth-Key, specific to Swift)

          "},{"location":"xbcloud-swift.html#-swift-container","title":"\u2013swift-container()","text":"

          The container to back up into (specific to Swift)

          "},{"location":"xbcloud-swift.html#-paralleln","title":"\u2013parallel(=N)","text":"

          The maximum number of concurrent upload/download requests. The default value is 1.

          "},{"location":"xbcloud-swift.html#-cacert","title":"\u2013cacert()","text":"

          The path to the file with CA certificates

          "},{"location":"xbcloud-swift.html#-insecure","title":"\u2013insecure()","text":"

          Do not verify server\u2019s certificate

          "},{"location":"xbcloud-swift.html#swift-authentication-options","title":"Swift authentication options","text":"

          The Swift specification describes several authentication options. The xbcloud tool can authenticate against keystone with API version 2 and 3.

          "},{"location":"xbcloud-swift.html#-swift-auth-version","title":"\u2013swift-auth-version()","text":"

          Specifies the swift authentication version. The possible values are: 1.0 - TempAuth, 2.0 - Keystone v2.0, and 3 - Keystone v3. The default value is 1.0.

          For v2 additional options are:

          "},{"location":"xbcloud-swift.html#-swift-tenant","title":"\u2013swift-tenant()","text":"

          Swift tenant name

          "},{"location":"xbcloud-swift.html#-swift-tenant-id","title":"\u2013swift-tenant-id()","text":"

          Swift tenant ID

          "},{"location":"xbcloud-swift.html#-swift-region","title":"\u2013swift-region()","text":"

          Swift endpoint region

          "},{"location":"xbcloud-swift.html#-swift-password","title":"\u2013swift-password()","text":"

          Swift password for the user

          For v3 additional options are:

          "},{"location":"xbcloud-swift.html#-swift-user-id","title":"\u2013swift-user-id()","text":"

          Swift user ID

          "},{"location":"xbcloud-swift.html#-swift-project","title":"\u2013swift-project()","text":"

          Swift project name

          "},{"location":"xbcloud-swift.html#-swift-project-id","title":"\u2013swift-project-id()","text":"

          Swift project ID

          "},{"location":"xbcloud-swift.html#-swift-domain","title":"\u2013swift-domain()","text":"

          Swift domain name

          "},{"location":"xbcloud-swift.html#-swift-domain-id","title":"\u2013swift-domain-id()","text":"

          Swift domain ID

          "},{"location":"xbcloud-swift.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xbcrypt-binary-overview.html","title":"The xbcrypt binary overview","text":"

          To support encryption and decryption of the backups, a new tool xbcrypt was introduced to Percona XtraBackup.

          The XBCRYPT_ENCRYPTION_KEY environment variable is only used in place of the --encrypt_key=name option. You can use the environment variable or command line option. If you use both, the command line option takes precedence over the value specified in the environment variable.

          This utility has been modeled after The xbstream binary to perform encryption and decryption outside of Percona XtraBackup. xbcrypt has following command line options:

          "},{"location":"xbcrypt-binary-overview.html#-d-decrypt","title":"-d(, \u2013decrypt()","text":"

          Decrypt data input to output.

          "},{"location":"xbcrypt-binary-overview.html#-i-inputname","title":"-i(, \u2013input(=name)","text":"

          Optional input file. If not specified, input will be read from standard input.

          "},{"location":"xbcrypt-binary-overview.html#-o-outputname","title":"-o(, \u2013output(=name)","text":"

          Optional output file. If not specified, output will be written to standard output.

          "},{"location":"xbcrypt-binary-overview.html#-a-encrypt-algoname","title":"-a(, \u2013encrypt-algo(=name)","text":"

          Encryption algorithm.

          "},{"location":"xbcrypt-binary-overview.html#-k-encrypt-keyname","title":"-k(, \u2013encrypt-key(=name)","text":"

          Encryption key.

          "},{"location":"xbcrypt-binary-overview.html#-f-encrypt-key-filename","title":"-f(, \u2013encrypt-key-file(=name)","text":"

          File which contains encryption key.

          "},{"location":"xbcrypt-binary-overview.html#-s-encrypt-chunk-size","title":"-s(, \u2013encrypt-chunk-size(=#)","text":"

          Size of working buffer for encryption in bytes. The default value is 64K.

          "},{"location":"xbcrypt-binary-overview.html#-encrypt-threads","title":"\u2013encrypt-threads(=#)","text":"

          This option specifies the number of worker threads that will be used for parallel encryption/decryption.

          "},{"location":"xbcrypt-binary-overview.html#-v-verbose","title":"-v(, \u2013verbose()","text":"

          Display verbose status output.

          "},{"location":"xbcrypt-binary-overview.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xbstream-binary-overview.html","title":"The xbstream binary overview","text":"

          To support simultaneous compression and streaming, a new custom streaming format called xbstream was introduced to Percona XtraBackup in addition to the TAR format. That was required to overcome some limitations of traditional archive formats such as tar, cpio and others which did not allow streaming dynamically generated files, for example dynamically compressed files. Other advantages of xbstream over traditional streaming/archive format include ability to stream multiple files concurrently (so it is possible to use streaming in the xbstream format together with the \u2013parallel option) and more compact data storage.

          This utility has a tar-like interface:

          • with the -x option it extracts files from the stream read from its standard input to the current directory unless specified otherwise with the -c option. The parallel extraction is supported with the --parallel option.

          • with the -c option it streams files specified on the command line to its standard output.

          • with the --decrypt=ALGO option specified xbstream will automatically decrypt encrypted files when extracting input stream. Supported values for this option are: AES128, AES192, and AES256. Either --encrypt-key or --encrypt-key-file options must be specified to provide encryption key, but not both.

          • with the --encrypt-threads option you can specify the number of threads for parallel data encryption. The default value is 1.

          • the --encrypt-key option is used to specify the encryption key that will be used. It can\u2019t be used with --encrypt-key-file option because they are mutually exclusive.

          • the --encrypt-key-file option is used to specify the file that contains the encryption key. It can\u2019t be used with --encrypt-key option. because they are mutually exclusive.

          The utility also tries to minimize its impact on the OS page cache by using the appropriate posix_fadvise() calls when available.

          When compression is enabled with xtrabackup all data is being compressed, including the transaction log file and meta data files, using the specified compression algorithm. Percona XtraBackup supports the following compression algorithms:

          "},{"location":"xbstream-binary-overview.html#zstandard-zstd","title":"Zstandard (ZSTD)","text":"

          The Zstandard (ZSTD) is a fast lossless compression algorithm that targets real-time compression scenarios and better compression ratios. ZSTD is the default compression algorithm for the --compress option.

          To compress files using the ZSTD compression algorithm, use the --compress option:

          $ xtrabackup --backup --compress --target-dir=/data/backup\n

          The resulting files have the \\*.zst format.

          You can specify ZSTD compression level with the --compress-zstd-level(=#) option. The default value is 1.

          $ xtrabackup \u2013backup \u2013compress \u2013compress-zstd-level=1 \u2013target-dir=/data/backup\n
          "},{"location":"xbstream-binary-overview.html#lz4","title":"lz4","text":"

          To compress files using the lz4 compression algorithm, set the --compress option to lz4:

          $ xtrabackup --backup --compress=lz4 --target-dir=/data/backup\n

          The resulting files have the \\*.lz4 format.

          To decompress files, use the --decompress option.

          "},{"location":"xbstream-binary-overview.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xtrabackup-binary-overview.html","title":"The xtrabackup binary overview","text":"

          The xtrabackup binary is a compiled C program that is linked with the InnoDB libraries and the standard MySQL client libraries.

          xtrabackup enables point-in-time backups of InnoDB / XtraDB tables together with the schema definitions, MyISAM tables, and other portions of the server.

          The InnoDB libraries provide the functionality to apply a log to data files. The MySQL client libraries are used to parse command-line options and configuration file.

          The tool runs in either --backup or --prepare mode, corresponding to the two main functions it performs. There are several variations on these functions to accomplish different tasks, and there is one less commonly used mode, --print-param.

          "},{"location":"xtrabackup-binary-overview.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xtrabackup-exit-codes.html","title":"xtrabackup exit codes","text":"

          The xtrabackup binary exits with the traditional success value of 0 after a backup when no error occurs. If an error occurs during the backup, the exit value is 1.

          In certain cases, the exit value can be something other than 0 or 1, due to the command-line option code included from the MySQL libraries. An unknown command-line option, for example, will cause an exit code of 255.

          "},{"location":"xtrabackup-exit-codes.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xtrabackup-files.html","title":"Index of files created by Percona XtraBackup","text":"
          • Information related to the backup and the server
          File Name Description backup-my.cnf This file contains information to start the mini instance of InnoDB during the --prepare. This **not** a backup of the original my.cnf. The InnoDB configuration is read from the file backup-my.cnf created by xtrabackup when the backup was made. The --prepare uses InnoDB configuration from backup-my.cnf by default, or from --defaults-file, if specified. The InnoDB's configuration in this context means server variables that affect dataformat, i.e., innodb_page_size option, innodb_log_block_size, etc. Location-related variables, like innodb_log_group_home_dir or innodb_data_file_path are always ignored by --prepare, so preparing a backup always works with data files from the back directory, rather than any external ones. xtrabackup_checkpoints

          The type of the backup (for example, full or incremental), its state (for example, prepared) and the LSN range contained in it. This information is used for incremental backups. Example of the xtrabackup_checkpoints after taking a full backup:

          backup_type = full-backuped\nfrom lsn= 0\nto_lsn = 15188961605\nlast_lsn = 15188961605\n
          Example of the xtrabackup_checkpoints after taking an incremental backup: 
          backup_type = incremental\nfrom_lsn = 15188961605\nto_lsn = 15189350111\nlast_lsn = 15189350111\n
          xtrabackup_binlog_info

          The binary log file used by the server and its position at the moment of the backup. A result of the following query:

          SELECT server_uuid, local, replication, storage_engines FROM performance_schema.log_status;\n
          xtrabackup_binlog The xtrabackup binary used in the process. xtrabackup_logfile Contains data needed for running the: --prepare. The bigger this file is the --prepare process will take longer to finish. <table_name>.delta.meta

          This file is going to be created when performing the incremental backup. It contains the per-table delta metadata: page size, size of compressed page (if the value is 0 it means the tablespace isn\u2019t compressed) and space id. Example of this file:

          page_size = 16384\nzip_size = 0\nspace_id = 0\n

          • Information related to the replication environment (if using the--slave-info option):

            xtrabackup_slave_info

            The CHANGE MASTER statement needed for setting up a replica.

          • Information related to the Galera and Percona XtraDB Cluster (if using the --galera-info option):

            xtrabackup_galera_info

            Contains the values of wsrep_local_state_uuid andwsrep_last_committed status variables

          "},{"location":"xtrabackup-files.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xtrabackup-implementation-details.html","title":"xtrabackup implementation details","text":"

          This page contains notes on various internal aspects of the xtrabackup tool\u2019s operation.

          "},{"location":"xtrabackup-implementation-details.html#file-permissions","title":"File permissions","text":"

          xtrabackup opens the source data files in read-write mode, although it does not modify the files. This means that you must run xtrabackup as a user who has permission to write the data files. The reason for opening the files in read-write mode is that xtrabackup uses the embedded InnoDB libraries to open and read the files, and InnoDB opens them in read-write mode because it normally assumes it is going to write to them.

          "},{"location":"xtrabackup-implementation-details.html#tune-the-os-buffers","title":"Tune the OS buffers","text":"

          Because xtrabackup reads large amounts of data from the filesystem, it uses posix_fadvise() where possible, to instruct the operating system not to try to cache the blocks it reads from disk. Without this hint, the operating system would prefer to cache the blocks, assuming that xtrabackup is likely to need them again, which is not the case. Caching such large files can place pressure on the operating system\u2019s virtual memory and cause other processes, such as the database server, to be swapped out. The xtrabackup tool avoids this with the following hint on both the source and destination files:

          posix_fadvise(file, 0, 0, POSIX_FADV_DONTNEED)\n

          In addition, xtrabackup asks the operating system to perform more aggressive read-ahead optimizations on the source files:

          posix_fadvise(file, 0, 0, POSIX_FADV_SEQUENTIAL)\n
          "},{"location":"xtrabackup-implementation-details.html#copy-data-files","title":"Copy data files","text":"

          When copying the data files to the target directory, xtrabackup reads and writes 1 MB of data at a time. This is not configurable. When copying the log file, xtrabackup reads and writes 512 bytes at a time. This is also not possible to configure, and matches InnoDB\u2019s behavior (workaround exists in Percona Server for MySQL because it has an option to tune innodb_log_block_size for XtraDB, and in that case Percona XtraBackup will match the tuning).

          After reading from the files, xtrabackup iterates over the 1MB buffer a page at a time, and checks for page corruption on each page with InnoDB\u2019s buf_page_is_corrupted() function. If the page is corrupt, it re-reads and retries up to 10 times for each page. It skips this check on the doublewrite buffer.

          "},{"location":"xtrabackup-implementation-details.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xtrabackup-option-reference.html","title":"The xtrabackup option reference","text":"

          Here you can find all of the command-line options for the xtrabackup binary.

          "},{"location":"xtrabackup-option-reference.html#modes-of-operation","title":"Modes of operation","text":"

          You invoke xtrabackup in one of the following modes:

          • --backup mode to make a backup in a target directory

          • --prepare mode to restore data from a backup (created in --backup mode)

          • --copy-back to copy data from a backup to the location that contained the original data; to move data instead of copying use the alternate --move-back mode.

          When you intend to run xtrabackup in any of these modes, use the following syntax:

          $ xtrabackup [--defaults-file=#] --backup|--prepare|--copy-back| [OPTIONS]\n

          For example, the --prepare mode is applied as follows:

          $ xtrabackup --prepare --target-dir=/data/backup/mysql/\n

          For all modes, the default options are read from the xtrabackup and mysqld configuration groups from the following files in the given order:

          1. /etc/my.cnf

          2. /etc/mysql/my.cnf

          3. /usr/etc/my.cnf

          4. ~/.my.cnf.

          As the first parameter to xtrabackup (in place of the --defaults-file, you may supply one of the following:

          • --print-defaults to have xtrabackup print the argument list and exit.

          • --no-defaults to forbid reading options from any file but the login file.

          • --defaults-file to read the default options from the given file.

          • --defaults-extra-file to read the specified additional file after the global files have been read.

          • --defaults-group-suffix to read the configuration groups with the given suffix. The effective group name is constructed by concatenating the default configuration groups (xtrabackup and mysqld) with the given suffix.

          • --login-path to read the given path from the login file.

          "},{"location":"xtrabackup-option-reference.html#innodb-options","title":"InnoDB options","text":"

          There is a large group of InnoDB options that are normally read from the my.cnf configuration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify them explicitly. These options have the same behavior in InnoDB and XtraDB. See --innodb-miscellaneous for more information.

          "},{"location":"xtrabackup-option-reference.html#options","title":"Options","text":""},{"location":"xtrabackup-option-reference.html#-apply-log-only","title":"\u2013apply-log-only()","text":"

          This option causes only the redo stage to be performed when preparing a backup. It is very important for incremental backups.

          "},{"location":"xtrabackup-option-reference.html#-backup","title":"\u2013backup()","text":"

          Make a backup and place it in --target-dir. See Creating a backup.

          "},{"location":"xtrabackup-option-reference.html#-backup-lock-timeout","title":"\u2013backup-lock-timeout()","text":"

          The timeout in seconds for attempts to acquire metadata locks.

          "},{"location":"xtrabackup-option-reference.html#-backup-lock-retry-count","title":"\u2013backup-lock-retry-count()","text":"

          The number of attempts to acquire metadata locks.

          "},{"location":"xtrabackup-option-reference.html#-backup-locks","title":"\u2013backup-locks()","text":"

          This option controls if backup locks should be used instead of FLUSH TABLES WITH READ LOCK on the backup stage. The option has no effect when backup locks are not supported by the server. This option is enabled by default, disable with --no-backup-locks.

          "},{"location":"xtrabackup-option-reference.html#-check-privileges","title":"\u2013check-privileges()","text":"

          This option checks if Percona XtraBackup has all required privileges. If a missing privilege is required for the current operation, it will terminate and print out an error message. If a missing privilege is not required for the current operation, but may be necessary for some other XtraBackup operation, the process is not aborted and a warning is printed.

          xtrabackup: Error: missing required privilege LOCK TABLES on *.*\nxtrabackup: Warning: missing required privilege REPLICATION CLIENT on *.*\n
          "},{"location":"xtrabackup-option-reference.html#-close-files","title":"\u2013close-files()","text":"

          Do not keep files opened. When xtrabackup opens tablespace it normally doesn\u2019t close its file handle in order to handle the DDL operations correctly. However, if the number of tablespaces is really huge and can not fit into any limit, there is an option to close file handles once they are no longer accessed. Percona XtraBackup can produce inconsistent backups with this option enabled. Use at your own risk.

          "},{"location":"xtrabackup-option-reference.html#-compress","title":"\u2013compress()","text":"

          This option tells xtrabackup to compress all output data, including the transaction log file and meta data files, using either the ZSTD or lz4 compression algorithm. ZSTD is the default compression algorithm.

          --compress produces \\*.zst files. You can extract the contents of these files by using the --decompress option. You can specify ZSTD compression level with the --compress-zstd-level(=#) option.

          --compress=lz4 produces \\*.lz4 files. You can extract the contents of these files by using lz4 program.

          "},{"location":"xtrabackup-option-reference.html#-compress-chunk-size","title":"\u2013compress-chunk-size(=#)","text":"

          Size of working buffer(s) for compression threads in bytes. The default value is 64K.

          "},{"location":"xtrabackup-option-reference.html#-compress-threads","title":"\u2013compress-threads(=#)","text":"

          This option specifies the number of worker threads used by xtrabackup for parallel data compression. This option defaults to 1. Parallel compression (--compress-threads) can be used together with parallel file copying (--parallel). For example, --parallel=4 --compress --compress-threads=2 will create 4 I/O threads that will read the data and pipe it to 2 compression threads.

          "},{"location":"xtrabackup-option-reference.html#-compress-zstd-level","title":"\u2013compress-zstd-level(=#)","text":"

          This option specifies ZSTD compression level. Compression levels provide a trade-off between the speed of compression and the size of the compressed files. A lower compression level provides faster compression speed but larger file sizes. A higher compression level provides lower compression speed but smaller file sizes. For example, set level 1 if the compression speed is the most important for you. Set level 19 if the size of the compressed files is the most important.

          The default value is 1. Allowed range of values is from 1 to 19.

          "},{"location":"xtrabackup-option-reference.html#-copy-back","title":"\u2013copy-back()","text":"

          Copy all the files in a previously made backup from the backup directory to their original locations. This option will not copy over existing files unless --force-non-empty-directories option is specified.

          "},{"location":"xtrabackup-option-reference.html#-core-file","title":"\u2013core-file()","text":"

          Write core on fatal signals.

          "},{"location":"xtrabackup-option-reference.html#-databases","title":"\u2013databases(=#)","text":"

          This option specifies a list of databases and tables that should be backed up. The option accepts the list of the form \"databasename1[.table_name1] databasename2[.table_name2] . . .\".

          "},{"location":"xtrabackup-option-reference.html#-databases-excludename","title":"\u2013databases-exclude(=name)","text":"

          Excluding databases based on name, Operates the same way as --databases, but matched names are excluded from backup. Note that this option has a higher priority than --databases.

          "},{"location":"xtrabackup-option-reference.html#-databases-file","title":"\u2013databases-file(=#)","text":"

          This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form databasename1[.table_name1], one element per line.

          "},{"location":"xtrabackup-option-reference.html#-datadirdirectory","title":"\u2013datadir(=DIRECTORY)","text":"

          The source directory for the backup. This should be the same as the datadir for your MySQL server, so it should be read from my.cnf if that exists; otherwise you must specify it on the command line.

          When combined with the --copy-back or --move-back option, --datadir refers to the destination directory.

          Once connected to the server, in order to perform a backup you will need READ and EXECUTE permissions at a filesystem level in the server\u2019s datadir.

          "},{"location":"xtrabackup-option-reference.html#-debug-sleep-before-unlock","title":"\u2013debug-sleep-before-unlock(=#)","text":"

          This is a debug-only option used by the xtrabackup test suite.

          "},{"location":"xtrabackup-option-reference.html#-debug-syncname","title":"\u2013debug-sync(=name)","text":"

          The debug sync point. This option is only used by the xtrabackup test suite.

          "},{"location":"xtrabackup-option-reference.html#-decompress","title":"\u2013decompress()","text":"

          Decompresses all files in a backup previously made with the --compress option. The --parallel option will allow multiple files to be decrypted simultaneously. Percona XtraBackup does not automatically remove the compressed files. In order to clean up the backup directory users should use --remove-original option.

          "},{"location":"xtrabackup-option-reference.html#-decompress-threads","title":"\u2013decompress-threads(=#)","text":"

          Force xbstream to use the specified number of threads for decompressing.

          "},{"location":"xtrabackup-option-reference.html#-decryptencryption-algorithm","title":"\u2013decrypt(=ENCRYPTION-ALGORITHM)","text":"

          Decrypts all files with the .xbcrypt extension in a backup previously made with --encrypt option. The --parallel option will allow multiple files to be decrypted simultaneously. Percona XtraBackup doesn\u2019t automatically remove the encrypted files. In order to clean up the backup directory users should use --remove-original option.

          "},{"location":"xtrabackup-option-reference.html#-defaults-extra-filemycnf","title":"\u2013defaults-extra-file(=[MY.CNF])","text":"

          Read this file after the global files are read. Must be given as the first option on the command-line.

          "},{"location":"xtrabackup-option-reference.html#-defaults-filemycnf","title":"\u2013defaults-file(=[MY.CNF])","text":"

          Only read default options from the given file. Must be given as the first option on the command-line. Must be a real file; it cannot be a symbolic link.

          "},{"location":"xtrabackup-option-reference.html#-defaults-groupgroup-name","title":"\u2013defaults-group(=GROUP-NAME)","text":"

          This option is to set the group which should be read from the configuration file. This is used by xtrabackup if you use the --defaults-group option. It is needed for mysqld_multi deployments.

          "},{"location":"xtrabackup-option-reference.html#-defaults-group-suffix","title":"\u2013defaults-group-suffix(=#)","text":"

          Also reads groups with concat(group, suffix).

          "},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool","title":"\u2013dump-innodb-buffer-pool()","text":"

          This option controls whether or not a new dump of buffer pool content should be done.

          With --dump-innodb-buffer-pool, xtrabackup makes a request to the server to start the buffer pool dump (it takes some time to complete and is done in background) at the beginning of a backup provided the status variable innodb_buffer_pool_dump_status reports that the dump has been completed.

          $ xtrabackup --backup --dump-innodb-buffer-pool --target-dir=/home/user/backup\n

          By default, this option is set to OFF.

          If innodb_buffer_pool_dump_status reports that there is running dump of buffer pool, xtrabackup waits for the dump to complete using the value of --dump-innodb-buffer-pool-timeout

          The file ib_buffer_pool stores tablespace ID and page ID data used to warm up the buffer pool sooner.

          "},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool-timeout","title":"\u2013dump-innodb-buffer-pool-timeout()","text":"

          This option contains the number of seconds that xtrabackup should monitor the value of innodb_buffer_pool_dump_status to determine if buffer pool dump has completed.

          This option is used in combination with --dump-innodb-buffer-pool. By default, it is set to 10 seconds.

          "},{"location":"xtrabackup-option-reference.html#-dump-innodb-buffer-pool-pct","title":"\u2013dump-innodb-buffer-pool-pct()","text":"

          This option contains the percentage of the most recently used buffer pool pages to dump.

          This option is effective if --dump-innodb-buffer-pool option is set to ON. If this option contains a value, xtrabackup sets the MySQL system variable innodb_buffer_pool_dump_pct. As soon as the buffer pool dump completes or it is stopped (see --dump-innodb-buffer-pool-timeout), the value of the MySQL system variable is restored.

          "},{"location":"xtrabackup-option-reference.html#-encryptencryption_algorithm","title":"\u2013encrypt(=ENCRYPTION_ALGORITHM)","text":"

          This option instructs xtrabackup to encrypt backup copies of InnoDB data files using the algorithm specified in the ENCRYPTION_ALGORITHM. Currently supported algorithms are: AES128, AES192 and AES256

          "},{"location":"xtrabackup-option-reference.html#-encrypt-keyencryption_key","title":"\u2013encrypt-key(=ENCRYPTION_KEY)","text":"

          A proper length encryption key to use. It is not recommended to use this option where there is uncontrolled access to the machine as the command line and thus the key can be viewed as part of the process info.

          "},{"location":"xtrabackup-option-reference.html#-encrypt-key-fileencryption_key_file","title":"\u2013encrypt-key-file(=ENCRYPTION_KEY_FILE)","text":"

          The name of a file where the raw key of the appropriate length can be read from. The file must be a simple binary (or text) file that contains exactly the key to be used.

          It is passed directly to the xtrabackup child process. See the xtrabackup documentation for more details.

          "},{"location":"xtrabackup-option-reference.html#-encrypt-threads","title":"\u2013encrypt-threads(=#)","text":"

          This option specifies the number of worker threads that will be used for parallel encryption/decryption. See the xtrabackup documentation for more details.

          "},{"location":"xtrabackup-option-reference.html#-encrypt-chunk-size","title":"\u2013encrypt-chunk-size(=#)","text":"

          This option specifies the size of the internal working buffer for each encryption thread, measured in bytes. It is passed directly to the xtrabackup child process.

          To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.

          "},{"location":"xtrabackup-option-reference.html#-estimate-memory","title":"\u2013estimate-memory(=#)","text":"

          This option is in tech preview. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

          Implemented in Percona XtraBackup 8.0.32-26, the option lets you enable or disable the Smart memory estimation feature. The default value is OFF. Enable the feature by setting --estimate-memory=ON in the backup phase and setting the --use-free-memory-pct option in the --prepare phase. If the --estimate-memory setting is disabled, the --use-free-memory-pct setting is ignored.

          An example of how to enable the Smart memory estimation feature:

          $ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n
          $ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\n
          "},{"location":"xtrabackup-option-reference.html#-export","title":"\u2013export()","text":"

          Create files necessary for exporting tables. See Restoring Individual Tables.

          "},{"location":"xtrabackup-option-reference.html#-extra-lsndirdirectory","title":"\u2013extra-lsndir(=DIRECTORY)","text":"

          (for \u2013backup): save an extra copy of the xtrabackup_checkpoints and xtrabackup_info files in this directory.

          "},{"location":"xtrabackup-option-reference.html#-force-non-empty-directories","title":"\u2013force-non-empty-directories()","text":"

          When specified, it makes --copy-back and --move-back option transfer files to non-empty directories. No existing files will be overwritten. If files that need to be copied/moved from the backup directory already exist in the destination directory, it will still fail with an error.

          "},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-timeoutseconds","title":"\u2013ftwrl-wait-timeout(=SECONDS)","text":"

          This option specifies time in seconds that xtrabackup should wait for queries that would block FLUSH TABLES WITH READ LOCK before running it. If there are still such queries when the timeout expires, xtrabackup terminates with an error. Default is 0, in which case it does not wait for queries to complete and starts FLUSH TABLES WITH READ LOCK immediately. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

          "},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-thresholdseconds","title":"\u2013ftwrl-wait-threshold(=SECONDS)","text":"

          This option specifies the query run time threshold which is used by xtrabackup to detect long-running queries with a non-zero value of --ftwrl-wait-timeout. FLUSH TABLES WITH READ LOCK is not started until such long-running queries exist. This option has no effect if --ftwrl-wait-timeout is 0. Default value is 60 seconds. Where supported xtrabackup will automatically use Backup Locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

          "},{"location":"xtrabackup-option-reference.html#-ftwrl-wait-query-typeallupdate","title":"\u2013ftwrl-wait-query-type(=all|update)","text":"

          This option specifies which types of queries are allowed to complete before xtrabackup will issue the global lock. Default is all.

          "},{"location":"xtrabackup-option-reference.html#-galera-info","title":"\u2013galera-info()","text":"

          This option creates the xtrabackup_galera_info file which contains the local node state at the time of the backup. Option should be used when performing the backup of Percona XtraDB Cluster. It has no effect when backup locks are used to create the backup.

          "},{"location":"xtrabackup-option-reference.html#-generate-new-master-key","title":"\u2013generate-new-master-key()","text":"

          Generate a new master key when doing a copy-back.

          "},{"location":"xtrabackup-option-reference.html#-generate-transition-key","title":"\u2013generate-transition-key()","text":"

          xtrabackup needs to access the same keyring file or vault server during prepare and copy-back but it should not depend on whether the server keys have been purged.

          --generate-transition-key creates and adds to the keyring a transition key for xtrabackup to use if the master key used for encryption is not found because it has been rotated and purged.

          "},{"location":"xtrabackup-option-reference.html#-get-server-public-key","title":"\u2013get-server-public-key()","text":"

          Get the server public key

          "},{"location":"xtrabackup-option-reference.html#-help","title":"\u2013help()","text":"

          When run with this option or without any options xtrabackup displays information about how to run the program on the command line along with all supported options and variables with default values where appropriate.

          "},{"location":"xtrabackup-option-reference.html#-historyname","title":"\u2013history(=NAME)","text":"

          This option enables the tracking of backup history in the PERCONA_SCHEMA.xtrabackup_history table. An optional history series name may be specified that will be placed with the history record for the current backup being taken.

          "},{"location":"xtrabackup-option-reference.html#-hosthost","title":"\u2013host(=HOST)","text":"

          This option accepts a string argument that specifies the host to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql \u2013help for details.

          "},{"location":"xtrabackup-option-reference.html#-incremental-basedirdirectory","title":"\u2013incremental-basedir(=DIRECTORY)","text":"

          When creating an incremental backup, this is the directory containing the full backup that is the base dataset for the incremental backups.

          "},{"location":"xtrabackup-option-reference.html#-incremental-dirdirectory","title":"\u2013incremental-dir(=DIRECTORY)","text":"

          When preparing an incremental backup, this is the directory where the incremental backup is combined with the full backup to make a new full backup.

          "},{"location":"xtrabackup-option-reference.html#-incremental-force-scan","title":"\u2013incremental-force-scan()","text":"

          When creating an incremental backup, force a full scan of the data pages in that instance.

          "},{"location":"xtrabackup-option-reference.html#-incremental-history-namename","title":"\u2013incremental-history-name(=name)","text":"

          This option specifies the name of the backup series stored in the PERCONA_SCHEMA.xtrabackup_history history record to base an incremental backup on. xtrabackup will search the history table looking for the most recent (highest innodb_to_lsn), successful backup in the series and take the to_lsn value to use as the starting lsn for the incremental backup. This will be mutually exclusive with --incremental-history-uuid, --incremental-basedir and --incremental-lsn. If no valid lsn can be found (no series by that name, no successful backups by that name) xtrabackup will return with an error. It is used with the --incremental option.

          "},{"location":"xtrabackup-option-reference.html#-incremental-history-uuidname","title":"\u2013incremental-history-uuid(=name)","text":"

          This option specifies the UUID of the specific history record stored in the PERCONA_SCHEMA.xtrabackup_history to base an incremental backup on. --incremental-history-name, --incremental-basedir and --incremental-lsn. If no valid lsn can be found (no success record with that UUID) xtrabackup will return with an error. It is used with the \u2013incremental option.

          "},{"location":"xtrabackup-option-reference.html#-incremental-lsnlsn","title":"\u2013incremental-lsn(=LSN)","text":"

          When creating an incremental backup, you can specify the log sequence number (LSN) instead of specifying --incremental-basedir. For databases created in 5.1 and later, specify the LSN as a single 64-bit integer.

          Important

          If a wrong LSN value is specified (a user error which Percona XtraBackup does not detect), the backup is unusable. Be careful!

          "},{"location":"xtrabackup-option-reference.html#-innodbname","title":"\u2013innodb([=name])","text":"

          This option is ignored for MySQL option compatibility.

          "},{"location":"xtrabackup-option-reference.html#-innodb-miscellaneous","title":"\u2013innodb-miscellaneous()","text":"

          There is a large group of InnoDB options that are normally read from the my.cnf configuration file, so that xtrabackup boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify these explicitly. These options have the same behavior in InnoDB and XtraDB:

          "},{"location":"xtrabackup-option-reference.html#-keyring-file-datafilename","title":"\u2013keyring-file-data(=FILENAME)","text":"

          The path to the keyring file. Combine this option with --xtrabackup-plugin-dir.

          "},{"location":"xtrabackup-option-reference.html#-kill-long-queries-timeoutseconds","title":"\u2013kill-long-queries-timeout(=SECONDS)","text":"

          This option specifies the number of seconds xtrabackup waits between starting FLUSH TABLES WITH READ LOCK and killing those queries that block it. Default is 0 seconds, which means xtrabackup will not attempt to kill any queries. In order to use this option xtrabackup user should have the PROCESS and SUPER privileges. Where supported, xtrabackup automatically uses Backup locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables.

          "},{"location":"xtrabackup-option-reference.html#-kill-long-query-typeallselect","title":"\u2013kill-long-query-type(=all|select)","text":"

          This option specifies which types of queries should be killed to unblock the global lock. Default is \u201cselect\u201d.

          "},{"location":"xtrabackup-option-reference.html#-lock-ddl","title":"\u2013lock-ddl()","text":"

          Issue LOCK TABLES FOR BACKUP if it is supported by server (otherwise use LOCK INSTANCE FOR BACKUP) at the beginning of the backup to block all DDL operations.

          Note

          Prior to Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup stops the SQL replica thread after the InnoDB tables and before the non-InnoDB tables are backed up.

          As of Percona XtraBackup 8.0.22-15.0, using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.

          "},{"location":"xtrabackup-option-reference.html#-lock-ddl-per-table","title":"\u2013lock-ddl-per-table()","text":"

          Lock DDL for each table before xtrabackup starts to copy it and until the backup is completed.

          Note

          As of Percona XtraBackup 8.0.15, the \u2013lock-ddl-per-table option is deprecated. Use the \u2013lock-ddl option instead.

          "},{"location":"xtrabackup-option-reference.html#-lock-ddl-timeout","title":"\u2013lock-ddl-timeout()","text":"

          If LOCK TABLES FOR BACKUP or LOCK INSTANCE FOR BACKUP does not return within given timeout, abort the backup.

          "},{"location":"xtrabackup-option-reference.html#-log","title":"\u2013log()","text":"

          This option is ignored for MySQL

          "},{"location":"xtrabackup-option-reference.html#-log-bin","title":"\u2013log-bin()","text":"

          The base name for the log sequence.

          "},{"location":"xtrabackup-option-reference.html#-log-bin-indexname","title":"\u2013log-bin-index(=name)","text":"

          File that holds the names for binary log files.

          "},{"location":"xtrabackup-option-reference.html#-log-copy-interval","title":"\u2013log-copy-interval(=#)","text":"

          This option specifies the time interval between checks done by the log copying thread in milliseconds (default is 1 second).

          "},{"location":"xtrabackup-option-reference.html#-login-path","title":"\u2013login-path()","text":"

          Read the given path from the login file.

          "},{"location":"xtrabackup-option-reference.html#-move-back","title":"\u2013move-back()","text":"

          Move all the files in a previously made backup from the backup directory to their original locations. As this option removes backup files, it must be used with caution.

          "},{"location":"xtrabackup-option-reference.html#-no-backup-locks","title":"\u2013no-backup-locks()","text":"

          Explicity disables the --backup-locks option which is enabled by default.

          "},{"location":"xtrabackup-option-reference.html#-no-defaults","title":"\u2013no-defaults()","text":"

          The default options are only read from the login file.

          "},{"location":"xtrabackup-option-reference.html#-no-lock","title":"\u2013no-lock()","text":"

          Disables table lock with FLUSH TABLES WITH READ LOCK. Use it only if all your tables are InnoDB and you do not care about the binary log position of the backup. This option shouldn\u2019t be used if there are any DDL statements being executed or if any updates are happening on non-InnoDB tables (this includes the system MyISAM tables in the mysql database), otherwise it could lead to an inconsistent backup. Where supported xtrabackup will automatically use Backup locks as a lightweight alternative to FLUSH TABLES WITH READ LOCK to copy non-InnoDB data to avoid blocking DML queries that modify InnoDB tables. If you are considering to use this because your backups are failing to acquire the lock, this could be because of incoming replication events are preventing the lock from succeeding. Please try using --safe-slave-backup to momentarily stop the replication replica thread, this may help the backup to succeed and you do not need to use this option.

          "},{"location":"xtrabackup-option-reference.html#-no-server-version-check","title":"\u2013no-server-version-check()","text":"

          Implemented in Percona XtraBackup 8.0.21.

          The --no-server-version-check option disables the server version check.

          The default behavior runs a check that compares the source system version to the Percona XtraBackup version. If the source system version is higher than the XtraBackup version, the backup is aborted with a message.

          Adding the option overrides this check, and the backup proceeds, but there may be issues with the backup.

          See Server Version and Backup Version Comparison for more information.

          "},{"location":"xtrabackup-option-reference.html#-no-version-check","title":"\u2013no-version-check()","text":"

          This option disables the version check. If you do not pass this option, the automatic version check is enabled implicitly when xtrabackup runs in the --backup mode. To disable the version check, you should pass explicitly the --no-version-check option when invoking xtrabackup.

          When the automatic version check is enabled, xtrabackup performs a version check against the server on the backup stage after creating a server connection. xtrabackup sends the following information to the server:

          • MySQL flavour and version

          • Operating system name

          • Percona Toolkit version

          • Perl version

          Each piece of information has a unique identifier. This is a MD5 hash value that Percona Toolkit uses to obtain statistics about how it is used. This is a random UUID; no client information is either collected or stored.

          "},{"location":"xtrabackup-option-reference.html#-open-files-limit","title":"\u2013open-files-limit(=#)","text":"

          The maximum number of file descriptors to reserve with setrlimit().

          "},{"location":"xtrabackup-option-reference.html#-parallel","title":"\u2013parallel(=#)","text":"

          This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer). In Percona XtraBackup 2.3.10 and newer, this option can be used with the --copy-back option to copy the user data files in parallel (redo logs and system tablespaces are copied in the main thread).

          "},{"location":"xtrabackup-option-reference.html#-passwordpassword","title":"\u2013password(=PASSWORD)","text":"

          This option specifies the password to use when connecting to the database. It accepts a string argument. See mysql \u2013help for details.

          "},{"location":"xtrabackup-option-reference.html#-plugin-load","title":"\u2013plugin-load()","text":"

          List of plugins to load.

          "},{"location":"xtrabackup-option-reference.html#-portport","title":"\u2013port(=PORT)","text":"

          This option accepts a string argument that specifies the port to use when connecting to the database server with TCP/IP. It is passed to the mysql child process without alteration. See mysql \u2013help for details.

          "},{"location":"xtrabackup-option-reference.html#-prepare","title":"\u2013prepare()","text":"

          Makes xtrabackup perform a recovery on a backup created with --backup, so that it is ready to use. See preparing a backup.

          "},{"location":"xtrabackup-option-reference.html#-print-defaults","title":"\u2013print-defaults()","text":"

          Print the program argument list and exit. Must be given as the first option on the command-line.

          "},{"location":"xtrabackup-option-reference.html#-print-param","title":"\u2013print-param()","text":"

          Makes xtrabackup print out parameters that can be used for copying the data files back to their original locations to restore them.

          "},{"location":"xtrabackup-option-reference.html#-read-buffer-size","title":"\u2013read-buffer-size()","text":"

          Set the read buffer size. The given value is scaled up to page size. The default size is 10MB. Use this option to increase the xbcloud/xbstream chunk size from the default size. To adjust the chunk size for encrypted files, use --read-buffer-size and --encrypt-chunk-size.

          "},{"location":"xtrabackup-option-reference.html#-rebuild-indexes","title":"\u2013rebuild-indexes()","text":"

          Rebuilds indexes in a compact backup. This option only has effect when the --prepare and --rebuild-threads options are provided.

          "},{"location":"xtrabackup-option-reference.html#-rebuild-threads","title":"\u2013rebuild-threads(=#)","text":"

          Uses the given number of threads to rebuild indexes in a compact backup. This option only has effect with the --prepare and --rebuild-indexes options.

          "},{"location":"xtrabackup-option-reference.html#-register-redo-log-consumer","title":"\u2013register-redo-log-consumer()","text":"

          The --register-redo-log-consumer parameter is disabled by default. When enabled, this parameter lets Percona XtraBackup register as a redo log consumer at the start of the backup. The server does not remove a redo log that Percona XtraBackup (the consumer) has not yet copied. The consumer reads the redo log and manually advances the log sequence number (LSN). The server blocks the writes during the process. Based on the redo log consumption, the server determines when it can purge the log.

          "},{"location":"xtrabackup-option-reference.html#-remove-original","title":"\u2013remove-original()","text":"

          This option when specified will remove .qp, .xbcrypt and .qp.xbcrypt files after decryption and decompression.

          "},{"location":"xtrabackup-option-reference.html#-rocksdb-datadir","title":"\u2013rocksdb-datadir()","text":"

          RocksDB data directory

          "},{"location":"xtrabackup-option-reference.html#-rocksdb-wal-dir","title":"\u2013rocksdb-wal-dir()","text":"

          RocksDB WAL directory.

          "},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-age","title":"\u2013rocksdb-checkpoint-max-age()","text":"

          The checkpoint cannot be older than this number of seconds when the backup completes.

          "},{"location":"xtrabackup-option-reference.html#-rocksdb-checkpoint-max-count","title":"\u2013rocksdb-checkpoint-max-count()","text":"

          Complete the backup even if the checkpoint age requirement has not been met after this number of checkpoints.

          "},{"location":"xtrabackup-option-reference.html#-rollback-prepared-trx","title":"\u2013rollback-prepared-trx()","text":"

          Force rollback prepared InnoDB transactions.

          "},{"location":"xtrabackup-option-reference.html#-rsync","title":"\u2013rsync()","text":"

          Uses the rsync utility to optimize local file transfers. When this option is specified, xtrabackup uses rsync to copy all non-InnoDB files instead of spawning a separate cp for each file, which can be much faster for servers with a large number of databases or tables. This option cannot be used together with --stream.

          "},{"location":"xtrabackup-option-reference.html#-safe-slave-backup","title":"\u2013safe-slave-backup()","text":"

          When specified, xtrabackup will stop the replica SQL thread just before running FLUSH TABLES WITH READ LOCK and wait to start backup until Slave_open_temp_tables in SHOW STATUS is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds. The replication SQL thread will be restarted when the backup finishes. This option is implemented in order to deal with replicating temporary tables and isn\u2019t necessary with Row-Based-Replication.

          Note

          Using a safe-slave-backup option stops the SQL replica thread before copying the InnoDB files.

          "},{"location":"xtrabackup-option-reference.html#-safe-slave-backup-timeoutseconds","title":"\u2013safe-slave-backup-timeout(=SECONDS)","text":"

          How many seconds --safe-slave-backup should wait for Slave_open_temp_tables to become zero. Defaults to 300 seconds.

          "},{"location":"xtrabackup-option-reference.html#-secure-auth","title":"\u2013secure-auth()","text":"

          Refuse client connecting to server if it uses old (pre-4.1.1) protocol. (Enabled by default; use \u2013skip-secure-auth to disable.)

          "},{"location":"xtrabackup-option-reference.html#-server-id","title":"\u2013server-id(=#)","text":"

          The server instance being backed up.

          "},{"location":"xtrabackup-option-reference.html#-server-public-key-path","title":"\u2013server-public-key-path()","text":"

          The file path to the server public RSA key in the PEM format.

          "},{"location":"xtrabackup-option-reference.html#-skip-tables-compatibility-check","title":"\u2013skip-tables-compatibility-check()","text":"

          See --tables-compatibility-check.

          "},{"location":"xtrabackup-option-reference.html#-slave-info","title":"\u2013slave-info()","text":"

          This option is useful when backing up a replication replica server. It prints the binary log position of the source server. It also writes the binary log coordinates to the xtrabackup_slave_info file as a CHANGE MASTER command. A new replica for this source can be set up by starting a replica server on this backup and issuing a CHANGE MASTER command with the binary log position saved in the xtrabackup_slave_info file.

          "},{"location":"xtrabackup-option-reference.html#-socket","title":"\u2013socket()","text":"

          This option accepts a string argument that specifies the socket to use when connecting to the local database server with a UNIX domain socket. It is passed to the mysql child process without alteration. See mysql \u2013help for details.

          "},{"location":"xtrabackup-option-reference.html#-ssl","title":"\u2013ssl()","text":"

          Enable secure connection.

          "},{"location":"xtrabackup-option-reference.html#-ssl-ca","title":"\u2013ssl-ca()","text":"

          Path of the file which contains list of trusted SSL CAs.

          "},{"location":"xtrabackup-option-reference.html#-ssl-capath","title":"\u2013ssl-capath()","text":"

          Directory path that contains trusted SSL CA certificates in PEM format.

          "},{"location":"xtrabackup-option-reference.html#-ssl-cert","title":"\u2013ssl-cert()","text":"

          Path of the file which contains X509 certificate in PEM format.

          "},{"location":"xtrabackup-option-reference.html#-ssl-cipher","title":"\u2013ssl-cipher()","text":"

          List of permitted ciphers to use for connection encryption.

          "},{"location":"xtrabackup-option-reference.html#-ssl-crl","title":"\u2013ssl-crl()","text":"

          Path of the file that contains certificate revocation lists. MySQL server documentation.

          "},{"location":"xtrabackup-option-reference.html#-ssl-crlpath","title":"\u2013ssl-crlpath()","text":"

          Path of directory that contains certificate revocation list files.

          "},{"location":"xtrabackup-option-reference.html#-ssl-fips-mode","title":"\u2013ssl-fips-mode()","text":"

          SSL FIPS mode (applies only for OpenSSL); permitted values are: OFF, ON, STRICT.

          "},{"location":"xtrabackup-option-reference.html#-ssl-key","title":"\u2013ssl-key()","text":"

          Path of file that contains X509 key in PEM format.

          "},{"location":"xtrabackup-option-reference.html#-ssl-mode","title":"\u2013ssl-mode()","text":"

          Security state of connection to server.

          "},{"location":"xtrabackup-option-reference.html#-ssl-verify-server-cert","title":"\u2013ssl-verify-server-cert()","text":"

          Verify server certificate Common Name value against host name used when connecting to server.

          "},{"location":"xtrabackup-option-reference.html#-streamformat","title":"\u2013stream(=FORMAT)","text":"

          Stream all backup files to the standard output in the specified format. Currently, this option only supports the xbstream format.

          "},{"location":"xtrabackup-option-reference.html#-strict","title":"\u2013strict()","text":"

          If this option is specified, xtrabackup fails with an error when invalid parameters are passed.

          "},{"location":"xtrabackup-option-reference.html#-tablesname","title":"\u2013tables(=name)","text":"

          A regular expression against which the full tablename, in databasename.tablename format, is matched. If the name matches, the table is backed up. See partial backups.

          "},{"location":"xtrabackup-option-reference.html#-tables-compatibility-check","title":"\u2013tables-compatibility-check()","text":"

          Enables the engine compatibility warning. The default value is ON. To disable the engine compatibility warning use --skip-tables-compatibility-check.

          "},{"location":"xtrabackup-option-reference.html#-tables-excludename","title":"\u2013tables-exclude(=name)","text":"

          Filtering by regexp for table names. Operates the same way as --tables, but matched names are excluded from backup. Note that this option has a higher priority than --tables.

          "},{"location":"xtrabackup-option-reference.html#-tables-filename","title":"\u2013tables-file(=name)","text":"

          A file containing one table name per line, in databasename.tablename format. The backup will be limited to the specified tables.

          "},{"location":"xtrabackup-option-reference.html#-target-dirdirectory","title":"\u2013target-dir(=DIRECTORY)","text":"

          This option specifies the destination directory for the backup. If the directory does not exist, xtrabackup creates it. If the directory does exist and is empty, xtrabackup will succeed. xtrabackup will not overwrite existing files, however; it will fail with operating system error 17, file exists.

          If this option is a relative path, it is interpreted as being relative to the current working directory from which xtrabackup is executed.

          In order to perform a backup, you need READ, WRITE, and EXECUTE permissions at a filesystem level for the directory that you supply as the value of --target-dir.

          "},{"location":"xtrabackup-option-reference.html#-innodb-temp-tablespaces-dirdirectory","title":"\u2013innodb-temp-tablespaces-dir(=DIRECTORY)","text":"

          Directory where temp tablespace files live, this path can be absolute.

          "},{"location":"xtrabackup-option-reference.html#-throttle","title":"\u2013throttle(=#)","text":"

          This option limits the number of chunks copied per second. The chunk size is 10 MB.

          To limit the bandwidth to 10 MB/s, set the option to 1.

          "},{"location":"xtrabackup-option-reference.html#-tls-ciphersuites","title":"\u2013tls-ciphersuites()","text":"

          TLS v1.3 cipher to use.

          "},{"location":"xtrabackup-option-reference.html#-tls-version","title":"\u2013tls-version()","text":"

          TLS version to use, permitted values are: TLSv1, TLSv1.1, TLSv1.2, TLSv1.3.

          "},{"location":"xtrabackup-option-reference.html#-tmpdirname","title":"\u2013tmpdir(=name)","text":"

          Specify the directory that will be used to store temporary files during the backup

          "},{"location":"xtrabackup-option-reference.html#-transition-keyname","title":"\u2013transition-key(=name)","text":"

          This option is used to enable processing the backup without accessing the keyring vault server. In this case, xtrabackup derives the AES encryption key from the specified passphrase and uses it to encrypt tablespace keys of tablespaces being backed up.

          If --transition-key does not have any value, xtrabackup will ask for it. The same passphrase should be specified for the --prepare command.

          "},{"location":"xtrabackup-option-reference.html#-use-free-memory-pct","title":"\u2013use-free-memory-pct()","text":"

          The --use-free-memory-pct is a tech preview option. Before using this option in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

          Implemented in Percona XtraBackup 8.0.30-23, this option lets you configure the Smart memory estimation feature. The option controls the amount of free memory that can be used to --prepare a backup. The default value is 0 (zero) which defines the option as disabled. For example, if you set --use-free-memory-pct=50, then 50% of the free memory is used to prepare a backup. The maximum allowed value is 100.

          This option works, only if --estimate-memory option is enabled. If the --estimate-memory option is disabled, the --use-free-memory-pct setting is ignored.

          An example of how to enable the Smart memory estimation feature:

          $ xtrabackup --backup --estimate-memory=ON --target-dir=/data/backups/\n
          $ xtrabackup --prepare --use-free-memory-pct=50 --target-dir=/data/backups/\n
          "},{"location":"xtrabackup-option-reference.html#-use-memory","title":"\u2013use-memory()","text":"

          This option affects how much memory is allocated and is similar to innodb_buffer_pool_size. This option is only relevant in the --prepare phase. The default value is 100MB. The recommended value is between 1GB to 2GB. Multiple values are supported if you provide the unit (for example, 1MB, 1M, 1GB, 1G).

          "},{"location":"xtrabackup-option-reference.html#-userusername","title":"\u2013user(=USERNAME)","text":"

          This option specifies the MySQL username used when connecting to the server, if that\u2019s not the current user. The option accepts a string argument. See mysql \u2013help for details.

          "},{"location":"xtrabackup-option-reference.html#-v","title":"-v()","text":"

          See --version

          "},{"location":"xtrabackup-option-reference.html#-version","title":"\u2013version()","text":"

          This option prints xtrabackup version and exits.

          "},{"location":"xtrabackup-option-reference.html#-xtrabackup-plugin-dirdirname","title":"\u2013xtrabackup-plugin-dir(=DIRNAME)","text":"

          The absolute path to the directory that contains the keyring plugin.

          "},{"location":"xtrabackup-option-reference.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"xtrabackup-version-numbers.html","title":"Understand version numbers","text":"

          A version number identifies the innovtion product release. The product contains the latest features, improvements, and bug fixes at the time of that release.

          8.1.0 -1 Base version Minor build version

          Percona uses semantic version numbering, which follows the pattern of base version and build version. Percona assigns unique, non-negative integers in increasing order for each version release. The version number combines the base MySQL 8.1 version number and the minor build version.

          The version numbers for Percona XtraBackup 8.1.0-1 define the following information:

          • Base version - the leftmost numbers indicate the MySQL 8.1 version used as a base. An increase in base version resets the minor build version to 0.

          • Minor build version - an internal number that denotes the version of the software. A build version increases by one each time the Percona XtraBackup is released.

          "},{"location":"xtrabackup-version-numbers.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"yum-download-rpm.html","title":"Install Percona XtraBackup 8.1 using downloaded RPM packages","text":"

          Download RPM packages of the desired series for your architecture from the download page.

          The following example downloads Percona XtraBackup 8.1.0-1 release package for CentOS 7:

          $ wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-8.1.0-1/binary/redhat/7/x86_64/percona-xtrabackup-81-8.1.0-1.el7.x86_64.rpm\n

          Install Percona XtraBackup by running:

          $ yum localinstall percona-xtrabackup-81-8.1.0-1.el7.x86_64.rpm\n

          When installing packages manually like this, you\u2019ll need to make sure to resolve all the dependencies and install missing packages yourself.

          "},{"location":"yum-download-rpm.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"yum-files.html","title":"Files in the RPM package built for Percona XtraBackup 8.1","text":"

          The following tables show what data each RPM package contains:

          Package Contains percona-xtrabackup-81 The latest Percona XtraBackup GA binaries and associated files percona-xtrabackup-81-debuginfo The debug symbols for binaries in percona-xtrabackup-81 percona-xtrabackup-test-81 The test suite for Percona XtraBackup percona-xtrabackup The older version of the Percona XtraBackup"},{"location":"yum-files.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"yum-repo.html","title":"Use a YUM repository to install Percona XtraBackup","text":"

          Ready-to-use packages are available from the Percona XtraBackup software repositories and the download page. The Percona yum repository supports popular RPM-based operating systems, including the Amazon Linux AMI.

          The easiest way to install the Percona Yum repository is to install an RPM that configures yum and installs the Percona GPG key.

          Specific information on the supported platforms, products, and versions is described in Percona Software and Platform Lifecycle.

          "},{"location":"yum-repo.html#install-percona-xtrabackup-from-percona-yum-repository","title":"Install Percona XtraBackup from Percona yum repository","text":"

          To install Percona XtraBackup from Percona yum repository, do the following steps:

          1. Install the Percona yum repository by running the following command as the root user or with sudo: 

            $ sudo yum install \\\nhttps://repo.percona.com/yum/percona-release-latest.\\\nnoarch.rpm\n

          2. Enable the repository: 

            $ sudo percona-release enable-only tools release\n

            If Percona XtraBackup is intended to be used in combination with the upstream MySQL Server, you only need to enable the `tools repository: 

            $ sudo percona-release enable-only tools\n

          3. Install Percona XtraBackup by running:

            $ sudo yum install percona-xtrabackup-81\n

            Warning

            Make sure that you have the libev package installed before installing Percona XtraBackup on CentOS 6. For this operating system, the libev package is available from the EPEL repositories.

          4. To decompress backups made using LZ4 or ZSTD compression algorithm, install the corresponding package:

            Install the lz4 packageInstall the zstd package 
            $ sudo yum install lz4\n
            $ sudo yum install zstd\n

          See also

          To install Percona XtraBackup using downloaded rpm packages, see Install with package manager.

          To uninstall Percona XtraBackup, see Uninstall Percona XtraBackup

          "},{"location":"yum-repo.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"yum-uninstall-xtrabackup.html","title":"Uninstall Percona XtraBackup 8.1 on Red Hat Enterprise Linux and CentOS","text":"

          To completely uninstall Percona XtraBackup, remove all the installed packages:

          $ yum remove percona-xtrabackup\n
          "},{"location":"yum-uninstall-xtrabackup.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"release-notes/8.1.0-1.html","title":"Percona XtraBackup 8.1.0-1 (2023-11-02)","text":"

          Get started with Quickstart Guide for Percona XtraBackup.

          Percona XtraBackup (PXB) is a 100% open source backup solution for all versions of Percona Server for MySQL and MySQL\u00ae that performs online non-blocking, tightly compressed, highly secure full backups on transactional systems. Maintain fully available applications during planned maintenance windows with Percona XtraBackup.

          This is an Innovation release. This type of release is only supported for a short time and is designed to be used in an environment with fast upgrade cycles. Developers and DBAs are exposed to the latest features and improvements.

          "},{"location":"release-notes/8.1.0-1.html#release-highlights","title":"Release highlights","text":"

          Percona XtraBackup 8.1.0-1 is based on MySQL 8.1 and fully supports the Percona Server for MySQL 8.1 Innovation series and the MySQL 8.1 Innovation series. This release allows taking backups of Percona Server 8.1.0-1 and MySQL 8.1.

          Use the Percona XtraBackup 8.0 series to take backups of Percona Server for MySQL 8.0.x or MySQL 8.0.x. Percona XtraBackup 8.1.0-1 does not take a backup of the Percona Server for MySQL 8.0 or the MySQL 8.0.x series.

          "},{"location":"release-notes/8.1.0-1.html#improvement","title":"Improvement","text":"
          • PXB-3155 : In the MySQL 8.0 series, the Vault keyring was implemented as a server plugin. In the Innovation series, MySQL is transitioning to a component infrastructure. This transition does not change the keyring\u2019s functionality. PXB 8.1.0-1 supports the component keyring_vault. The keyring_vault plugin is no longer supported. Only the keyring_file plugin is supported. This plugin is deprecated and may be removed in a later version.
          "},{"location":"release-notes/8.1.0-1.html#deprecated-or-removed","title":"Deprecated or removed","text":"

          The --stats mode of operation for the xtrabackup binary has been removed.

          Only the keyring_vault component, the KMIP component, and the AWS KMS component versions are supported.

          For the keyring_file both the plugin and component are supported. The keyring_file plugin is deprecated and may be removed in the future.

          "},{"location":"release-notes/8.1.0-1.html#useful-links","title":"Useful links","text":"

          Install Percona XtraBackup 8.1

          The Percona XtraBackup GitHub location

          Download product binaries, packages, and tarballs at Percona Product Downloads

          "},{"location":"release-notes/8.1.0-1.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "},{"location":"release-notes/release-notes.html","title":"Percona XtraBackup 8.1 release notes index","text":"
          • Percona XtraBackup 8.1.0-1 (2023-11-02)
          "},{"location":"release-notes/release-notes.html#get-expert-help","title":"Get expert help","text":"

          If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

          Community Forum Get a Percona Expert

          "}]} \ No newline at end of file diff --git a/innovation-release/sitemap.xml b/innovation-release/sitemap.xml index 2fb18acfb..a15d5197f 100644 --- a/innovation-release/sitemap.xml +++ b/innovation-release/sitemap.xml @@ -2,432 +2,432 @@ https://docs.percona.com/percona-xtrabackup/innovation-release/index.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/404.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/about-xtrabackup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/accelerate-backup-process.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/apt-download-deb.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/apt-files.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/apt-pinning.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/apt-repo.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/apt-uninstall-xtrabackup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/binaries-overview.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/binary-tarball-names.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/binary-tarball.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/compile-xtrabackup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/configure-xtrabackup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/copyright-and-licensing-information.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/create-compressed-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/create-full-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/create-gtid-replica.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/create-incremental-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/create-individual-partition-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/create-partial-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/dictionary-cache.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/docker.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/encrypt-backups.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/encrypted-innodb-tablespace-backups.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/faq.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/features.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/flush-tables-with-read-lock.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/glossary.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/how-xtrabackup-works.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/installation.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/limitations.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/lock-options.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/log-enhancements.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/lru-dump-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/make-backup-in-replication-env.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/page-tracking.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/permissions.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/point-in-time-recovery.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/prepare-compressed-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/prepare-full-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/prepare-incremental-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/prepare-individual-partitions-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/prepare-partial-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/privileges.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/quickstart-overview.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/restore-a-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/restore-individual-partitions.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/restore-individual-tables.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/restore-partial-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/server-backup-version-comparison.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/set-up-replication.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/smart-memory-estimation.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/store-backup-history.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/take-streaming-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/throttling-backups.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/toolkit-version-check.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/trademark-policy.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/update-curl-utility.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/varify-backup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/work-with-apparmor.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/work-with-selinux.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/working-with-binary-logs.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-azure.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-binary-fifo-datasink.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-binary-overview.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-exbackoff.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-gcs.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-iam-profile.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-minio.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-s3.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcloud-swift.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbcrypt-binary-overview.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xbstream-binary-overview.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xtrabackup-binary-overview.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xtrabackup-exit-codes.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xtrabackup-files.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xtrabackup-implementation-details.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xtrabackup-option-reference.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/xtrabackup-version-numbers.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/yum-download-rpm.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/yum-files.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/yum-repo.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/yum-uninstall-xtrabackup.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/release-notes/8.1.0-1.html - 2023-11-02 + 2023-11-03 daily https://docs.percona.com/percona-xtrabackup/innovation-release/release-notes/release-notes.html - 2023-11-02 + 2023-11-03 daily \ No newline at end of file diff --git a/innovation-release/sitemap.xml.gz b/innovation-release/sitemap.xml.gz index 9219184bc3cd74c5145177c598ca364c3c3b3a66..623632bb9d5f1969abb67a50fa35e54676d477b9 100644 GIT binary patch delta 704 zcmV;x0zdum2k-|6ABzYG58y)z1erAgkRIG9npiA~1Qwc3+e>?YBwzJNVRzC7MDX?cgLnH$%*ab+-U)6h<2tg` z1?xy}G&!jx1+2LS7fT^oW(_gM%aet=#C+mC+SU{g|B%O_$2u4B?vVDKtC%E!3 zfl5uwz4P_~5`8f`OUtg^sidUW?mWWsQ4g%&6H-MIprDjg%G6a32Ihh3$AsoY(L$~= z7EnpMYow8;C5*8<%)jb_G_oZ*B7#bg%oWlE$oI%%-8(sZNjhU;o?a+W6;V9}OT2M> zlW`9|=9{9ShyoV9mk(+kej~PjN8W)(ZGn2J8MJSk69h;^9doSJJ!p7~c#YAWus*bI z1m;P4dTD{Q;iOU|q2vAiOv%o>V5nv_RdiT+^PFiO?QarvpskxMYg;;zcR;g{5S#)e zZqt=LO;hL@Xx)_Q;=o&~uSetJWR?DV${>Pd?U^3?s7U8@!EllUCL+fkaA$&Rvx$8# mf@Ux>!~nH#r84)}bCSVjnQeKmC19k|PS&K40MgRcp*;3H} delta 704 zcmV;x0zdum2k-|6ABzYGXtG0*2Oa@QktVZ$x_kYs_ItH^C6l%6$;^y84A5uzI9QGD z_SQX>EY0qIw);OI__F)Ly8ZeXr}|{W8^##A%esPEj8w|eKq{gOj!_6x@8>3s&uf*p@C3n-ccC(nEhx z>L*e=IXFW_+x!I@gjxa*2DpFB|U&53v=SWia=H#OJT19DPpL9ZJ1T` zK&7VT-g)}~iM|+}rDfOdR8rDwcOGH+s0Y^X38^9pP*6%LW$G#i1M|T2V?uMHXdzb_ z3#g>sHPT4a62{ma=3jL|8rhN@5kVzL<_c*7yB%QG^PcIaxim0A~CEhr` z$+!m}^G(rEL;;K5%Lla%zY*JiBkw??wm`kq4BEHN2?8Xdjycxq9yGi~yvFEGSRYz9 z0`nw2y|h5ua8fCf(DD9$rex<`FjTXeDmtvZdCoMC_BRPS(AG_swJn{