Hi all and welcome back.  This is the second part of the previous blog post of backing up and restore ETCD in an OpenShift cluster using Kasten K10.  We already explained what is ETCD and why in some scenarios could be good to have an ETCD backup.  So, in this post we will describe the process to backup ETCD using Kasten.

When using Kasten we can automate the ETCD backup process, and sending the backup to a specific Location Profile (repository) defined in Kasten.

What we need:

• We need the etcd endpoint of at least one of the master nodes. It is possible to get it using the following command:

• The etcd pod labels.  In the case of OCP, it is likely that etcd pods have labels app=etcd,etcd=true
• The namespace in which the etcd pods are running.   In the case of OCP, it is likely that etcd pods are running in the namespace openshift-etcd

Creating a Secret

Before taking a backup of ETCD with Kasten, we need to create a Secret in a temporary new or existing namespace.  So first we are going to create a temporary namespace:

Next, we create the Secret with the data collected previously:

To make sure we are taking a backup of just the Secret, and not of any other workload in the namespace, we can label the Secret we have just created with the following command:

Create a Kanister Blueprint

In order to backup ETCD, Kasten will leverage a Kanister blueprint to do so.  Therefore, the next step is creating the Blueprint that will be used by Kasten when taking an ETCD backup.   Kasten provides a Kanister blueprint you can use to take an ETCD backup from an OpenShift cluster.  You can download and create the blueprint in the Kasten namespace using the following command:

If you check the blueprint, for the backup process you can see two important phases:

• takeSnapshot: Where the blueprint will run a Pod using the Kanister image.  This pod, using the data provided in the Secret created previously, will get the ETCD pods and will take a snapshot of ETCD.  The Snapshot will be saved in /tmp/etcd-backup.db
• uploadSnapshot: This phase will be used to upload the ETCD snapshot to the Location Profile (repository) selected when the Policy is created (described in the next section)

Once the blueprint is created we need to create an annotation for the Secret created in the previous step, to instruct Kasten to use this Secret alongside the Kanister Blueprint to take the ETCD backup.

Create a Backup Policy

The last step to taking an ETCD backup with Kasten is to create a Backup Policy for the temporary namespace:

• We create a new Policy, provide the name, frequency and retention settings according to your needs.

• We select the namespace to backup (etcd-backup in this example).
• Then we filter resources to include just resources with the label “include:true” as mentioned before.  In this way, just the secret will be backed up by this policy.  Then it will be Kanister the one responsible of taking the ETCD backup using the Blueprint previously created.

• Finally, we set the Location Profile (repository) for the ETCD backup created by the Kanister Blueprint.

Once the policy is created, we can review the policy in Kasten web UI, and we can click in “Run Once” to test the policy.

Finally, we can see the policy has completed successfully, and now we have a fully automated ETCD backup for a RedHat OpenShift cluster.

To continue with the restore process, please check the next post in this series: https://patriciocerda.com/?p=1487