Openshift

This page documents Openshift specific details such as backing up etcd and static pod manifests.

etcd backup

Openshift recommends running the script “cluster-backup.sh” regularly and backing up the etcd snapshot and other files created by it.

Note that CloudCasa backs up the manifests of all the selected resources so in most scenarios, you don’t need to restore from etcd backup explicitly. However, there are some Openshift DR scenarios where etcd data needs to be restored from its snapshot file.

In order to run the “cluster-backup.sh” script using CloudCasa, you can use App Hooks mechanism by running a pod from which this script can be run. To make the process of creating this pod simpler, CloudCasa provides a shell script called deploy-cloudcasa-openshift-etcd-backup-runner.sh.

The rest of this section will provide more details about this script.

What This Script Does

  1. Creates a dedicated namespace called “cloudcasa-openshift-etcd-backup”. All other resources mentioned below are created in this namespace.

  2. Creates a ConfigMap containing the embedded etcd backup script.

  3. Creates a PersistentVolumeClaim (PVC) to store the resulting snapshot .db and tarball files.

  4. Deploys a privileged Deployment that:

    • Mounts the host filesystem (hostPath) at /host

    • Mounts the PVC at /pvc-backup

    • Executes the embedded script to take and store an etcd snapshot

  5. Optionally assigns the privileged SCC to the service account used by the Deployment.

  6. Wait for the Deployment to become ready.

Pre-requisites

Before running the script, ensure the following:

  • You have the oc CLI installed on your system.

  • You are logged in to an OpenShift cluster using oc login.

  • You have permissions to assign SCCs (Security Context Constraints) in the target namespace.

  • The OpenShift cluster has a working storage class to be used for the PVC that is backed up by CloudCasa. By default, the script uses the storage class ocs-storagecluster-cephfs and provisions a PVC with size 5 GB. To change these values, use the command line parameters --pvc-sc and pvc-size-mb.

  • You are authorized to create namespaces, deployments, PVCs, and configmaps in the cluster.

Running the Script

To deploy the pod that can run “cluster-backup.sh”, make sure that pre-requisites are satisfied and follow these steps:

  1. Download the script.

    curl -o deploy-cloudcasa-openshift-etcd-backup-runner.sh \
    https://raw.githubusercontent.com/catalogicsoftware/cloudcasa-artifacts/refs/heads/master/scripts/deploy-cloudcasa-openshift-etcd-backup-runner.sh

  2. Make the script executable, if it isn’t already:

    chmod +x deploy-cloudcasa-openshift-etcd-backup-runner.sh

  3. Run the script:

    ./deploy-cloudcasa-openshift-etcd-backup-runner.sh

  4. Follow the on-screen prompts:

Why the Script Requires privileged SCC

This script deploys a container that needs access to:

  • The control plane host filesystem via hostPath

  • Host PID/IPC/Network namespaces

  • Run as root (runAsUser: 0)

  • Execute privileged commands (privileged: true)

These are not permitted by OpenShift’s default restricted SCC. Hence, the script prompts the user to assign the built-in privileged SCC to the default service account in the created namespace. This is an intentional security measure to inform the user and allow manual opt-in.

Important

If the privileged SCC is not assigned, the pod will fail to run due to SCC validation errors.

Next Steps After Script Execution

Once the deployment is created and the pod is ready:

  1. Log in to CloudCasa and create a new App Hook, as follows (only needed before version 1.5.1):

    • Go to Configuration => App Hooks

    • Create an app hook by clicking on Add App Hook and using the following values:

      • Name: openshift etcd backup

      • Type pre-backup

      • Pod selector: app=cloudcasa-etcd-backup-runner

      • Container: cloudcasa-etcd-backup-runner

      • Hook command:

        /bin/sh -c ". /cloudcasa-etcd-backup.sh"

  2. Create a new backup job.

    • Include the namespace cloudcasa-openshift-etcd-backup in the backup scope.

    • Add a Pre-Backup Hook to trigger the snapshot script, with the following values:

      • Namespace: cloudcasa-openshift-etcd-backup

      • App Hooks If using 1.5.1 or later, enable the option “App Hooks from templates”. select openshift etcd backup from the dropdown.

When this backup is run, the pre-backup hook will run the backup script and copy the files to the PVC, which will then be included in the backup. It is recommended that you create a dedicated job for this purpose so that you can associate it with a specific schedule suitable for creating etcd snapshots.

Cleanup

To delete all the resources created by the script, delete the namespace “cloudcasa-openshift-etcd-backup”.

Restoring to a previous cluster state

To restore the cluster to a previous state, restore the etcd snapshot and static pod manifest files from a backup and then follow the steps described here.

The cluster state files need to be restored to a control plane host that acts as “recovery host” and here are some ways that can be done.

File restore to a PVC

Requires CloudCasa 1.5.0 or later.

Using CloudCasa’s file restore feature, you can browse the files in the backup PVC (in namespace “cloudcasa-openshift-etcd-backup”) and restore them to any target PVC (in any cluster). After the files are restored to a PVC, you can copy them over to the recovery host using kubectl cp or other methods.

PVC restore

Instead of just restoring files, you can restore entire PVC containing etcd backup files. This was the PVC mounted to the pod deployed using the script deploy-cloudcasa-openshift-etcd-backup-runner.sh. After restoring the PVC, you may need to mount it to a pod so that you can copy the required files (using kubectl cp for example). Note that restoring only PVC requires CloudCasa 1.5.0 or later.

A slight variation of this method is to restore entire namespace “cloudcasa-openshift-etcd-backup” to an alternate namespace. This way, you automatically get a pod and the PVC so that you don’t need to run a pod separately. Once the files are copied over, the namespace can be deleted.

Direct file restore

It is possible to directly restore etcd and other files on to the node that is acting as the “recovery host”, using casactl CLI. However, this method is more complicated compared to the above two methods. Please contact CloudCasa support if you want to use this mechanism.