0 0 Share PDF

Automating Docker EE Backups

Article ID: KB000845

Note: This document provides guidelines for writing an automated backup plan for Docker EE. It is intended to be used as an example for writing a customized backup script. Be sure to customize and verify with your environment before using.

Criticality of Backups

Backups are critical to a highly-available infrastructure and should be regularly created for UCP and DTR. Backups store state for the Docker infrastructure such as cluster settings, access control policies, repository metadata, and certificates. This guide describes a tool and process that can be used as an example to automate backups and to ensure that they are created consistently and correctly.

Docker EE Backup Best Practices

The best practice for running backups is to only run backups on Highly Available (HA) clusters to prevent outages from occurring. Both UCP and DTR backup commands bring down their respective applications to perform a full backup of the state of the node being backed up. Thus, if one of the available UCP managers or DTR replicas were to be backed up in a HA cluster, the other remaining UCP managers or replicas would be able to continue to service requests while the application is being backed up.

UCP and DTR each have its own backup files. Saving the file system of UCP or DTR nodes does not constitute an appropriate backup. A specific tool is used to create a backup of both, and that process is described at the following links:

In each process, a .tar file is created that has the contents of the backup. The links above describe the type of data that is contained in each backup. The .tar file backup is a restore point that can be used to restore a UCP or DTR instance back to that point.

What do UCP and DTR backups not contain?

The UCP backup .tar file does not contain restore data about the applications running on UCP. Thus, it does not include manifests of the running applications, any application data, networks, or running container images. It is highly recommended to encrypt the UCP backup using a password with the -passphrase variable.

The DTR backup .tar file does not contain any image data itself. The image data is stored in the DTR storage backend which should have its own backup procedure dependent of the storage type that is used.

When should backups be taken?

The frequency of backups is dependent on the rate of change within UCP or DTR. If new repositories are added or changed in DTR daily, then daily backups are appropriate and will make sure to capture at least the last day's changes. Backups should also be taken before any major system change such as an engine, UCP, or DTR upgrade. Backups taken daily or weekly are typical.

Automating UCP and DTR Backups

UCP and DTR backups are complementary. Because DTR runs on top of UCP, both backups are necessary to fully restore a DTR instance. Because of this, it makes sense to take UCP and DTR backups together and store the backup .tar files as pairs.

The following script can help automate the backup procedure. These steps describe how to build the script into a container so that it can be scheduled to run at regular intervals. Use this script directly or modify it to meet the needs of your environment.

Build

This script does the following things:

  • Infers UCP IDs, DTR replica IDs, and versions from the URLs provided to it
  • Checks the health of DTR and UCP before starting the backup
  • Sequentially takes DTR and UCP backups together
  • Saves the backup .tar files with the date and version

The docker-ee-backup script is designed to run inside a container to make it more compatible. This Dockerfile builds a container that can run this script.

To build the backup container put the docker-ee-backup and the Dockerfile files into the current directory and run the following command (ensure that docker-backup.sh has executable permissions set):

$ ls
-rw-r--r--   1 church  staff   622B Nov 12 16:22 Dockerfile
-rwxr-xr-x   1 church  staff   3.4K Nov 12 16:22 docker-backup*

# docker build -t registry_host/account/docker-backup:version .
...
Successfully built 8b4802e9ed4f
Successfully tagged registry_host/account/docker-backup:version

Your backup container is now built. It can be pushed to your central DTR node and pulled onto the UCP controller node for use by the organization.

$ docker push registry_host/account/docker-backup:version
The push refers to repository [registry_host/account/docker-backup]
b48494a33fc7: Pushed
29ececc0fa8e: Pushed
2024a3594dd0: Pushed
392b673c0f87: Pushed
76734473c594: Pushed
db88a4fd1709: Pushed
6b2edeabe17e: Pushed
717b092b8c86: Pushed
latest: digest: sha256:dab86254b1ae330e47e8798dec1e0131b2cc03834f015f6cbd0771f073a43a6b size: 1990

Running Backups

Each time docker-ee-backup runs it will generate a single backup .tar file for UCP and DTR. It must be run on a UCP controller node to function. It expects to have a conf.env file present in its directory to run. The password (i.e. the PASSWORD variable) for UCP can be passed in through an environment variable so that it is not stored on disk. In addition, the backup password can also be passed in through and environment variable as well using the BACKUP_PASSWORD variable.

conf.env

USERNAME=admin

# For the following, it's best to use the host IPs to be guaranteed which machines
# the backups are being run against.  (i.e. don't use a load balancer IP or hostname)
UCP_HOST=UCP_MGR_HOST
DTR_HOST=DTR_REPLICA_HOST

# Enter your UCP front end FQDN here - it should match whatever hostname or SANs you
# have configured for UCP
UCP_URL=UCP_FRONT_END_FQDN

Run the following command on your UCP controller to initiate the backup. The backup files will be stored in the docker-backup volume on the UCP controller host.

export PASSWORD=<enter your UCP admin password>
export BACKUP_PASSWORD=<enter a backup password to encrypt your backup>
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/volumes:/var/lib/docker/volumes \
--env-file conf.env \
--env PASSWORD=$PASSWORD \
--env BACKUP_PASSWORD=$BACKUP_PASSWORD \
registry_host/account/docker-backup