Catalog backup/restore

CloudCasa stores most of its catalog data in MongoDB. In addition to CloudCasa resources such as clusters, backups, cloud accounts, policies, and recovery points, CloudCasa also maintains some inventory information about each client cluster, including the following (the list is not exhaustive):

  • Storage classes

  • PVCs

  • Namespaces

The size of the data kept in the database is relatively small, and for most customers the total DB storage should not be more than few GB.

In addition to data in MongoDB, CloudCasa also stores some data in the form of Kubernetes resources. At present, there is only one such resource of the type TZCronJob. There is one such resource for each policy schedule. So if there are 5 policies, each with one schedule, there will be 5 TZCronJob resources in the cloudcasa-server namespace.

Catalog Backup

It is important to back up CloudCasa catalog data so that, in case of any issues with the CloudCasa server cluster, CloudCasa can be installed on a new cluster and the catalog data restored. CloudCasa provides an automated way of performing MongoDB backups to any S3 object storage or to Azure blob storage.

Note

Catalog backup is only supported for MongoDB instances installed by CloudCasa. If you are using your own MongoDB instance, please make sure that you have configured backups and that you can restore from them.

Configuring backup to S3

Create a secret containing S3 credentials and a password to be used to encrypt the backup data. In the following command that creates the secret, replace these placeholders with appropriate values:

  • BACKUP-REPO-PASSWORD - The password that will be used to encrypt the backup data

  • S3-ACCESS-KEY - Access key for S3 or compatible object storage

  • S3-SECRET-KEY - Secret key for S3 or compatible object storage

kubectl -n cloudcasa-server create secret generic catalogbackup-creds \
 --from-literal=CC_CATALOG_BACKUP_REPO_CRED='<BACKUP-REPO-PASSWORD>' \
 --from-literal=AWS_ACCESS_KEY_ID='<S3-ACCESS-KEY>' \
 --from-literal=AWS_SECRET_ACCESS_KEY='<S3-SECRET-KEY>'

After creating the secret, run helm upgrade with the following parameters:

mongo.backupLocation.s3.enabled

Set to “true” to enable the backups to S3. Default is “false”.

mongo.backup.schedule

Schedule of the backups in crontab format. Optional. Default is “0 0 * * *” which runs backup once a day at midnight.

mongo.backupLocation.s3.bucketName

Name of the S3 bucket. Required.

mongo.backupLocation.region

Region of the S3 bucket. Required for AWS and optional for other S3 providers.

mongo.backupLocation.s3.endpoint

The S3 storage endpoint. Optional for AWS but required for any other S3 providers. Note: Do not include “https:” in the endpoint. E.g. “nyc3.digitaloceanspaces.com”.

mongo.backupLocation.prefix

Path in the bucket where catalog data is stored. Optional. By default, “cc_catalog_backup” is used as prefix.

Here is a sample Helm upgrade command to configure catalog backups to S3:

helm upgrade cloudcasa-server <CLOUDCASA-SERVER-HELMCHART-TAR> \
 -n cloudcasa-server --reuse-values --wait \
 --set mongo.backupLocation.s3.enabled=true \
 --set mongo.backupLocation.s3.bucketName=<BUCKET-NAME> \
 --set mongo.backupLocation.region=<REGION>

Configuring backup to Azure blob storage

Create a secret containing Azure credentials and a password to be used to encrypt the backup data. In the following example command that creates the secret, replace these placeholders with appropriate values:

  • BACKUP-REPO-PASSWORD - The password that will be used to encrypt the backup data

  • CLIENT-ID - The Azure client ID

  • CLIENT-SECRET - The Azure client secret

  • TENANT-ID - The Azure tenant ID

kubectl -n cloudcasa-server create secret generic catalogbackup-creds \
 --from-literal=CC_CATALOG_BACKUP_REPO_CRED='<BACKUP-REPO-PASSWORD>' \
 --from-literal=AZURE_CLIENT_ID='<CLIENT-ID>' \
 --from-literal=AZURE_CLIENT_SECRET='<CLIENT-SECRET>' \
 --from-literal=AZURE_TENANT_ID='<TENANT-ID>'

After creating the secret, run helm upgrade with the following parameters:

mongo.backupLocation.azure.enabled

Set to “true” to enable backups to Azure Blob Storage.

mongo.backupLocation.region

Azure region of the storage account. Required.

mongo.backup.schedule

Schedule of the backups in crontab format. Optional. Default is “0 0 * * *” which runs backup once a day at midnight.

mongo.backupLocation.prefix

Path in the bucket where catalog data is stored. Optional. By default, “cc_catalog_backup” is used as prefix.

mongo.backupLocation.azure.resourceGroupName

Name of Azure resource group. Required.

mongo.backupLocation.azure.storageAccountName

Name of Azure storage account. Required.

mongo.backupLocation.azure.isGovCloud

Set to “true” if the storage account is in Azure gov cloud.

Here is a sample Helm upgrade command to configure catalog backups to go to Azure storage:

helm upgrade cloudcasa-server <CLOUDCASA-SERVER-HELMCHART-TAR> \
 -n cloudcasa-server --reuse-values --wait \
 --set mongo.backupLocation.azure.enabled=true \
 --set mongo.backupLocation.region=<REGION> \
 --set mongo.backupLocation.azure.resourceGroupName=<RESOURCE-GROUP-NAME> \
 --set mongo.backupLocation.azure.storageAccountName=<STORAGE-ACCOUNT-NAME>

Backup notes

  • Please keep the repo password in a safe location, such as in a secure password manager. Without it, catalog backups cannot be restored.

  • By default, the last 7 backups are kept and older ones are deleted. To use a different retention count, set the Helm parameter mongo.backup.retentionCount.

  • Details regarding catalog backups can be seen in the CloudCasa UI under Site Admin => Catalog Backups. Note that this page is only visible for site admin users.

  • In order to run ad-hoc catalog backup, navigate to the Catalog Backups page in the CloudCasa UI and click “Run Catalog Backup”.

  • An ad-hoc catalog backup can also be run with the following command:

    kubectl -n cloudcasa-server create job --from=cronjob/catalogbackup <JOB-NAME>

Catalog Restore

A catalog restore may need to be performed if you need to re-install the CloudCasa server for some reason. Follow the steps in this section after performing the re-installation.

The restore is performed using casactl CLI, so please make sure that your environment is set up properly to run the command.

Finally, don’t forget to configure catalog backups again after your restore is complete!

Note

This method of catalog restore is only supported for the MongoDB instance installed by CloudCasa. If you are using your own Mongo instance, please make sure that you have configured backups and that you can restore from them.

Restoring from S3

You can list available backups with the following command:

casactl catalog list-from-s3 --repo-password <BACKUP-REPO-PASSWORD> --bucket <BUCKET> \
 --access-key <ACCESS_KEY>  --secret-key <SECRET-KEY>

If using a self-signed certificate, be sure to add --insecure after casactl. You may need to provide some other options such as --repo-prefix and --endpoint depending on your use case. Please run casactl catalog list-from-s3 --help for more details.

After listing the backups, pick the backup ID that you want to restore from and run the following catalog restore command:

casactl catalog restore-from-s3 --repo-password <BACKUP-REPO-PASSWORD> --bucket <BUCKET> \
 --access-key <ACCESS_KEY>  --secret-key <SECRET-KEY> <BACKUP-ID>

Note that the “restore” command works exactly the same way as “list” except that it takes backup ID as an additional argument.

Restoring from Azure

You can list available backups with the following command:

casactl catalog list-from-azure --repo-password <BACKUP-REPO-PASSWORD> --client-id <CLIENT-ID> \
 --client-secret <CLIENT-SECRET>  --tenant-id <TENANT-ID> --region <REGION> \
--resgroup <RESOURCE-GROUP> --storage-account <STORAGE-ACCOUNT>

You may need to provide some other options such as --repo-prefix depending on your use case. Please run casactl catalog list-from-azure --help for more details.

If the backups are hosted in Azure Government cloud, you must use the --gov option.

After listing the backups, pick the backup ID that you want to restore from and run the following catalog restore command:

casactl catalog restore-from-azure --repo-password <BACKUP-REPO-PASSWORD> --client-id <CLIENT-ID> \
 --client-secret <CLIENT-SECRET>  --tenant-id <TENANT-ID> --region <REGION> \
--resgroup <RESOURCE-GROUP> --storage-account <STORAGE-ACCOUNT> <BACKUP-ID>

Note that the “restore” command works exactly the same way as “list” except that it takes backup ID as an additional argument.

Catalog post-restore steps

There are a few other things that will need to be done after the database is restored.

First, “TZCronJob” resources need to be re-created for policy schedules. Secondly, cluster resources need to be moved to PENDING state. They will move to ACTIVE state once the agent connections are established.

Both of these steps are automatically done for MongoDB instances installed by CloudCasa. If you are using your own MongoDB instance, run the following command to perform the post-restore steps (after restoring Mongo data):

casactl catalog post-restore-steps

Note

If the DNS name of the CloudCasa server has changed, cluster agents will need to be re-installed so that they can establish connections to the new CloudCasa server. This can be done by deleting the cloudcasa-io namespace on each cluster and running the kubectl install command as shown in the UI.