Note: This document provides guidelines for writing an automated backup plan for Docker EE. It is intended to be used as an example for writing a customized backup script. Be sure to customize and verify with your environment before using.
Criticality of Backups
Backups are critical to a highly-available infrastructure and should be regularly created for UCP and DTR. Backups store state for the Docker infrastructure such as cluster settings, access control policies, repository metadata, and certificates. This guide describes a tool and process that can be used as an example to automate backups and to ensure that they are created consistently and correctly.
Docker EE Backup Best Practices
The best practice for running backups is to only run backups on Highly Available (HA) clusters to prevent outages from occurring. Both UCP and DTR backup commands bring down their respective applications to perform a full backup of the state of the node being backed up. Thus, if one of the available UCP managers or DTR replicas were to be backed up in a HA cluster, the other remaining UCP managers or replicas would be able to continue to service requests while the application is being backed up.
UCP and DTR each have its own backup files. Saving the file system of UCP or DTR nodes does not constitute an appropriate backup. A specific tool is used to create a backup of both, and that process is described at the following links:
In each process, a
.tar file is created that has the contents of the backup. The links above describe the type of data that is contained in each backup. The
.tar file backup is a restore point that can be used to restore a UCP or DTR instance back to that point.
What do UCP and DTR backups not contain?
The UCP backup
.tar file does not contain restore data about the applications running on UCP. Thus, it does not include manifests of the running applications, any application data, networks, or running container images. It is highly recommended to encrypt the UCP backup using a password with the
The DTR backup
.tar file does not contain any image data itself. The image data is stored in the DTR storage backend which should have its own backup procedure dependent of the storage type that is used.
When should backups be taken?
The frequency of backups is dependent on the rate of change within UCP or DTR. If new repositories are added or changed in DTR daily, then daily backups are appropriate and will make sure to capture at least the last day's changes. Backups should also be taken before any major system change such as an engine, UCP, or DTR upgrade. Backups taken daily or weekly are typical.
Automating UCP and DTR Backups
UCP and DTR backups are complementary. Because DTR runs on top of UCP, both backups are necessary to fully restore a DTR instance. Because of this, it makes sense to take UCP and DTR backups together and store the backup
.tar files as pairs.
The following script can help automate the backup procedure. These steps describe how to build the script into a container so that it can be scheduled to run at regular intervals. Use this script directly or modify it to meet the needs of your environment.
This script does the following things:
- Infers UCP IDs, DTR replica IDs, and versions from the URLs provided to it
- Checks the health of DTR and UCP before starting the backup
- Sequentially takes DTR and UCP backups together
- Saves the backup
.tarfiles with the date and version
$ ls -rw-r--r-- 1 church staff 622B Nov 12 16:22 Dockerfile -rwxr-xr-x 1 church staff 3.4K Nov 12 16:22 docker-backup* # docker build -t registry_host/account/docker-backup:version . ... Successfully built 8b4802e9ed4f Successfully tagged registry_host/account/docker-backup:version
Your backup container is now built. It can be pushed to your central DTR node and pulled onto the UCP controller node for use by the organization.
$ docker push registry_host/account/docker-backup:version The push refers to repository [registry_host/account/docker-backup] b48494a33fc7: Pushed 29ececc0fa8e: Pushed 2024a3594dd0: Pushed 392b673c0f87: Pushed 76734473c594: Pushed db88a4fd1709: Pushed 6b2edeabe17e: Pushed 717b092b8c86: Pushed latest: digest: sha256:dab86254b1ae330e47e8798dec1e0131b2cc03834f015f6cbd0771f073a43a6b size: 1990
docker-ee-backup runs it will generate a single backup
.tar file for UCP and DTR. It must be run on a UCP controller node to function. It expects to have a
conf.env file present in its directory to run. The password (i.e. the
PASSWORD variable) for UCP can be passed in through an environment variable so that it is not stored on disk. In addition, the backup password can also be passed in through and environment variable as well using the
USERNAME=admin # For the following, it's best to use the host IPs to be guaranteed which machines # the backups are being run against. (i.e. don't use a load balancer IP or hostname) UCP_HOST=UCP_MGR_HOST DTR_HOST=DTR_REPLICA_HOST # Enter your UCP front end FQDN here - it should match whatever hostname or SANs you # have configured for UCP UCP_URL=UCP_FRONT_END_FQDN
Run the following command on your UCP controller to initiate the backup. The backup files will be stored in the
docker-backup volume on the UCP controller host.
export PASSWORD=<enter your UCP admin password> export BACKUP_PASSWORD=<enter a backup password to encrypt your backup> docker run --rm -it \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /var/lib/docker/volumes:/var/lib/docker/volumes \ --env-file conf.env \ --env PASSWORD=$PASSWORD \ --env BACKUP_PASSWORD=$BACKUP_PASSWORD \ registry_host/account/docker-backup