Installation

Prerequisites

This checklist provides all the requirements that must be met before self-hosted CloudCasa can be installed. Please check and make sure that you meet all the requirements before proceeding with the installation.

  • [ ] Kubernetes cluster (required)

    • Version 1.23 or later.

    • A dedicated cluster is highly recommended, though not mandatory.

    Note

    Do not try to backup the cluster, where CloudCasa server is being installed, with CloudCasa itself. CloudCasa catalog can be protected using catalog backups (see Catalog backup/restore)

  • [ ] CLI tools (required)

    • Helm 3 installed.

    • kubectl access to the cluster.

  • [ ] Storage class (required)

    • Decide on a storage class for CloudCasa PVCs.

    • Confirm it can provision PVs successfully.

  • [ ] Network configuration (required)

    • Decide how to expose thee CloudCasa service and install any prerequisites required (such as an ingress controller).

    • See Networking for more details. In particular, check the Networking requirements.

  • [ ] Authentication provider (required)

    • Decide on which provider to use: LDAP, Azure, Google, any OIDC provider, or local (not recommended for production).

    • Confirm that the CloudCasa server can communicate with the selected authentication provider.

    • Decide which users can log in (CloudCasa does not allow everyone in the configured auth provider to login automatically). You also need to decide which users are going to have site-admin permissions. See Authentication for more details.

    Note

    For POCs, we recommend using “local” auth provider (see Local provider). But we can also provide a dev LDAP deployment with a pre-configured set of users and groups if needed.

  • [ ] Certificates (required)

    • Decide whether to use cert-manager (recommended) or manually create TLS secrets.

    • If using cert-manager, install it with the command below.

    • See Certificate Configuration for more details.

    To install “cert-manager”:

    helm repo add jetstack https://charts.jetstack.io
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace \
 --version v1.12.4 --set installCRDs=true

  • [ ] License & Credentials (required)

    • Confirm you have access to CloudCasa support portal at https://support.cloudcasa.io. Login with your email to receive a temporary access code.

    • Download a valid license file. The license is tied to the cluster where CloudCasa is installed. You will need to provide the UID of kube-system namespace on the cluster so that a permanent license file can be generated. (Note: the UID is not needed for POCs.)

    • Download Helm repo credentials.

    • Download Azure container registry (ACR) credentials.

  • [ ] Database (MongoDB) setup (optional)

    • Decide whether to use MongoDB installed as part of the server deployment (default) or an external MongoDB instance.

    • If using an external MongoDB:

      • Ensure it is not in the same namespace as the CloudCasa server.

      • Have the URI and authentication information (username and password) available.

      • Confirm that the CloudCasa server can communicate with the external MongoDB instance.

    • See MongoDB Setup for details.

  • [ ] Email server (optional, but recommended)

    • Prepare SMTP details for an email server that can be accessed using SMTP.

    • Make sure CloudCasa server can communicate with the SMTP server.

    • This is needed by CloudCasa to send notifications (e.g. to report job errors) and to send user invitations to join organizations.

  • [ ] Cloud account information (optional)

    • CloudCasa supports adding cloud accounts from AWS, Azure, and GCP, though this support requires several additional steps.

    • See Cloud Accounts for details.

  • [ ] Air-gapped environment (if applicable)

    • Download the Helm Chart .tgz file from support.cloudcasa.io.

    • Mirror all container images required by CloudCasa server and CloudCasa agent into your internal registry, maintaining image paths.

    • Ensure that the CloudCasa server cluster and client clusters can pull images from the selected internal registry.

  • [ ] Object storage for backups

    • Prepare S3 or Azure Storage Account details for object storage that will be used to store backups. If you want to use NFS for backups, prepare the NFS shares accordingly.

    • Ensure client clusters can communicate with the storage.

Installing the CloudCasa Server

  • Add CloudCasa helm repo:

    helm repo add cloudcasa https://helm.cloudcasa.io --username <USERNAME> --password <PASSWORD>

  • Create the “cloudcasa-server” namespace (kubectl create ns cloudcasa-server). If your organization requires any labels or policies to be set on a new namespace, you can do so at this time.

  • (Optional) Create the “cloudcasa-mongo” namespace (kubectl create ns cloudcasa-mongo). You may want to create the namespace explicitly if you need to set some labels or policies on it before proceeding with the installation. If you do create the namespace, make sure you set the helm parameter mongo.createNamespace to false.

  • Set up a certificate issuer or create your own TLS secrets (See Certificate Configuration). Here is one way of creating a “self signed” issuer with cert-manager:

    kubectl -n cloudcasa-server apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}

EOF

    Note: If you are using self-signed certificate, see Notes about cluster agent installation.

  • Create an image pull secret called “myregistrykey” in the cloudcasa-server namespace (using cloudcasa ACR credentials). Here is one way of doing it:

    kubectl create secret docker-registry myregistrykey \
 --docker-server=https://cloudcasa.azurecr.io \
 --docker-username=<username> \
 --docker-password=<password> \
 --namespace=cloudcasa-server

  • Install the CloudCasa server by running “helm install” command.:

    $ helm install cloudcasa-server cloudcasa/cloudcasa-server --create-namespace \
    --namespace cloudcasa-server --wait --timeout 15m0s

    In case of air-gapped installation, you need to use CloudCasa helm chart file directly, like so:

    $ helm install cloudcasa-server <CLOUDCASA-CHART-TGZ> --create-namespace \
    --namespace cloudcasa-server --wait --timeout 15m0s

    In addition to the above options, you will need to set the following Helm values.

    • storageClassName (required) - Set it to the storage class name that should be used to provision PVs.

    • licenseKey (required) - A valid license is required to use CloudCasa. To set the license key from the license key file: --set licenseKey=$(cat <licenseKeyFilePath>).

    • serviceType (Optional) - By default, the CloudCasa service that needs to be exposed outside of the cluster is created with type “NodePort” so it will not be accessible externally. Set this parameter to any other allowed type, such as “LoadBalancer”. For more details, see Networking.

    • Certificate parameters (required) - Default settings use self-signed issuer (created above) with cert-manager. To change issuer or supply your own TLS secrets, see Certificate Configuration.

    • Auth parameters (required) - Especially auth.allowedUsers, auth.allowedUserGroups, auth.siteadminUsers, auth.siteadminUserGroups. For details, see Authentication.

    • SMTP parameters (optional) - These are required for CloudCasa to send notifications, user invite emails, etc. For details, see Email Configuration. Without this config no emails can be sent.

    • MongoDB connection parameters (optional) - For details, see MongoDB Setup.

    • On Openshift 4.14+ Set “scc.enabled” to “true” to create “securityContextConstraint” resources. You can also set “scc.priority”. It is set to 15 by default.

    • Miscellaneous

      • Set “ui.enable_cloudaccounts” to “false” to hide cloud accounts functionality (including “Databases” tab).

      • “CloudCasa for Velero” functionality is hidden by default. To enable it, set “ui.enable_velero” to “true”.

    Note

    We highly recommend creating a Helm values file to override default values instead of using --set or directly editing the packaged values.yaml. This makes the update process straightforward and allows administrators to easily roll out changes.

    For example, create a yaml file with ONLY values that you wish to modify. Undeclared values will use defaults:

    storageClassName: my-custom-storageclass
licenseKey: <CONTENTS OF LICENSE FILE>

    Then specify your file in the helm install/upgrade command:

    $ helm install --values my_values_file.yaml

  • Once all CloudCasa deployments are running, you need to make sure that the service “amds-envoy-grpcapi” can be accessed from outside the cluster, either by IP or by DNS name. Once that is done, set the value “amdsEnvoyUrl” to the external address and run helm upgrade, for example:

    echo "amdsEnvoyUrl: <CLOUDCASA-ENVOY-EXTERNAL-IP>" >> my_values_file.yaml

helm upgrade cloudcasa-server cloudcasa/cloudcasa-server --wait --create-namespace \
    --namespace cloudcasa-server \
    --values my_values_file.yaml

  • If you are using cert-manager: You need to refresh TLS secrets to update them with DNS names. This can be done by deleting the TLS secret and restarting the amds-envoy deployment, like so:

    kubectl -n cloudcasa-server delete secret amds-envoy.tls && \
kubectl -n cloudcasa-server rollout restart deployments/amds-envoy

    When the cert-manager sees that the secret is gone, it will generate a new secret from the Issuer and the amds-envoy container should start. To check the status of the certificate, run:

    kubectl -n cloudcasa-server describe cert amds-envoy

That’s it! You can now access the web UI using the IP or DNS name of the “amds-envoy-grpcapi” service. The URL will be https://<CLOUDCASA-ENVOY-EXTERNAL-IP>. Note that if you used the self-signed issuer above, you will see certificate warnings twice.

Please see Post-installation tasks for details on adding clusters and defining backups.

Upgrading from an Earlier Version

Updating Values

Releases may add or change Helm values, which is why we recommend using a custom values file instead of editing values.yaml. If a new value is added, the default can be read from the packaged values.yaml

If you don’t already have a custom values file and want to copy your existing install’s values, you can use helm get values to display them:

helm -n cloudcasa-server get values cloudcasa-server

# USER-SUPPLIED VALUES:
# storageClassName: my-custom-storageclass
# ...

Updating Containers

Cloudcasa-server also uses Helm values for container image tags. The values.yaml file for each release includes the latest image tags - so as long as you are not overriding the images.<component>Tag values, containers will be updated automatically during helm upgrade.

If you are using a custom container registry you will need to download the latest images for the release and add them to your registry. The list of images for every release is always uploaded to the CloudCasa support portal. For more details see Custom Container Registry Setup.

Updating the Helm Chart

New Helm chart releases are available on the CloudCasa support portal or via the Helm repo at helm.cloudcasa.io. To get the latest helm chart from the repo:

helm repo update cloudcasa

You can also check the latest release by listing versions:

helm search repo cloudcasa --versions

After updating the repo and optionally updating your values file, run the helm upgrade command to apply the changes:

helm upgrade cloudcasa-server cloudcasa/cloudcasa-server --wait \
      --namespace cloudcasa-server \
      --values my_values_file.yaml

If all goes well, you should see the new version displayed under the logo in CloudCasa Dashboard.

Starting from 1.3.4 release, “tzcronjob” resources in “cloudcasa-server” namespace are automatically updated to use the latest “amds-jobrunner” tag. But please verify this. These resources will only be present if you defined any policies.

Also, if the new version includes any agent changes, agents on clusters that haven’t disabled automatic updates will be updated during next update cycle (by default, every 24 hours). If you need to update the agent right away, go to cluster’s “Edit” page and click on “Re-deploy agent” button.

Certificate Configuration

The CloudCasa server requires TLS for internal and external services. There are a few ways to set up TLS:

  • Using cert-manager (recommended)

  • Using an Issuer resource with a private CA

  • Manually creating certificates and TLS secrets

Using cert-manager

Install cert-manager and create an Issuer. Set the following Helm parameters accordingly:

  • certificates.issuerName - Name of the existing Issuer resource. Default value: “selfsigned-issuer”.

  • certificates.issuerType - Issuer resource type. Possible values: “Issuer” or “ClusterIssuer”. Default value: “Issuer”.

Using a private CA

Create an issuer resource for your private CA and set the following Helm parameters accordingly:

  • certificates.issuerName - Name of the issuer resource

  • certificates.issuerType - Issuer resource type. Possible values: “Issuer” or “ClusterIssuer”. Default value: “Issuer”.

  • certificates.issuerGroup - issuer group, ex “cert-manager.io”

Manually creating TLS Secrets

Custom certificates can be used without an Issuer or cert-manager by creating TLS secrets in the cloudcasa-server namespace. For information about creating TLS secrets, see Kubernetes docs: TLS Secrets.

After the secrets have been created in cloudcasa-server, set the following Helm parameters in order to use them. Note that all secret name parameters are required.

  • certificates.createTlsSecrets - Set to false to use your own TLS secrets

  • certificates.envoyTlsSecretName - Secret name for amds-envoy deployment. This is the only external-facing service, so the certificate’s dnsName or caName should match amdsEnvoyUrl

  • certificates.frontendTlsSecretName - Secret name for amds-frontend deployment. Internal-facing service.

  • certificates.dexTlsSecretName - Secret name for amds-dex deployment. Internal-facing service.

Example: Cert-manager with a custom domain

In this example, we want to host CloudCasa at a custom domain using AWS route53 and using “letsencrypt” to verify ownership of the domain and automatically create TLS secrets.

First create a “letsencrypt” Issuer. We will automatically validate domain ownership using cert-manager’s dns01 solver to authenticate with route53:

kubectl -n cloudcasa-server apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email: example@example.com
    preferredChain: ""
    privateKeySecretRef:
      name: letsencrypt-prod
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
    - dns01:
        route53:
          region: us-east-1
          accessKeyIDSecretRef:
            name: cert-manager-keys
            key: access-key-id
          secretAccessKeySecretRef:
            name: cert-manager-keys
            key: secret-access-key
EOF

Now follow the basic setup instructions for the CloudCasa server. When it’s time to run helm install…, add the certificate options below. Set amdsEnvoyUrl to the desired domain name. In this case the AWS role we supplied in the Issuer has permissions to manage this domain in AWS. In this example we will also set serviceType` to provision a load balancer.

--set certificates.issuerName=letsencrypt-prod \
--set certificates.issuerType=Issuer \
--set certificates.issuerGroup=cert-manager.io \
--set amdsEnvoyUrl=cloudcasa.example.com \
--set serviceType=LoadBalancer

helm install will create the cert resources and cert-manager will attempt to validate domain ownership. Once that is done, the TLS secrets will be created and the amds-envoy deployment will start.

To complete the process, we need to create an “A” record in route53 for this domain. To do this, get the load balancer IP by checking the external IP of the service:

kubectl -n cloudcasa-server get svc amds-envoy-grpcapi

Create a DNS “A” record for cloudcasa.example.com pointing to the external IP. After DNS propagation, you should see the CloudCasa UI at this address with a valid certificate.

MongoDB Setup

By default, CloudCasa installs MongoDB in the namespace “cloudcasa-mongo” and uses it to store catalog data. If this is acceptable, no other database configuration is required.

If you want to use your own MongoDB instance, please use the following Helm parameters. If you are planning to install MongoDB yourself, make sure it is not installed in the cloudcasa-server namespace.

Helm Parameters:

  • mongo.createStatefulset - Set to false to use your own Mongo instance. Default value is “true”.

  • mongo.createNamespace - Set to false to manually manage the mongo namespace without Helm. Default “true”.

  • mongo.statefulsetNamespace - Namespace to create/manage for mongo statefulset. Default “cloudcasa-mongo”

  • mongo.pv.size - Size of Database PVC to create for mongo statefulset

  • mongo.url - URL used by the CloudCasa application to connect to MongoDB. See MongoDB Connection Strings for more details.

  • mongo.authSecretName - Name of secret in cloudcasa-server which contains username and password data fields for connecting to the provided MongoDB URL. This must be created manually if using an external MongoDB instance. Not required for default setup. Default “mongo-secrets”

  • mongo.tls.enabled - Enable to use TLS when connecting to MongoDB. Requires certificateKeyFile to be set. Not required for default setup.

  • (optional) mongo.tls.certificateKeyFile - Path to the client certificate file to present to the server for TLS. See MongoDB TLS Configuration.

Default Installation

By default, Helm will create a MongoDB StatefulSet and Persistent Volume in its own namespace: cloudcasa-mongo. Database admin credentials are generated during the install, and can be found in the mongo-credentials secret in this namespace. The container has no external port so it can only be reached from within the cluster. It is highly recommended to limit access to the namespace cloudcasa-mongo` using Kubernetes RBAC.

Changing admin credentials is not recommended, but can be done by editing the mongo-credentials secret, running helm upgrade, and restarting the mongo pod. Note that Helm does not save the password value so removing this secret will trigger a password reset on the next Helm upgrade.

Using External MongoDB (optional)

It is also possible to use an external MongoDB instance as long as CloudCasa is able to reach the instance. This includes cloud mongo services (e.g. Atlas) or other instances reachable on your network. To do this you will need a connection URL and username/password pair to authenticate.

  • Set mongo.url to the base connection string for your mongo instance, excluding credentials or TLS options. Ex: “mongodb://localhost:27017”

  • Create a secret in cloudcasa-server with data entries for the mongo username and password. User should have permissions to create and manage a new database. Example:

    kubectl -n cloudcasa-server create secret generic mongo-secrets --from-literal=username=mongo_user \
    --from-literal=password=mongo_user_password

  • Ensure mongo.authSecretName is set to the secret from the previous step.

  • If TLS is required, set mongo.tls.enabled. Download the key bundle from the provider and set mongo.tls.certificateKeyFile to its filepath.

RBAC

This section provides configuration options to setup role-based access control (RBAC) for users. Roles can be created and assigned to users to control user permissions. User groups can be created to assign roles to multiple users. See CloudCasa documentation for Roles and User Groups for details.

CloudCasa server can obtain groups for a user from the configured authentication provider. To assign a role to an external group, a corresponding user group needs to be created in CloudCasa with the same name. Follow these steps to associate an external group to a CloudCasa user group:

  1. From the CloudCasa UI, navigate to Configuration -> Roles and click Add Role. Add permissions for the role and click Create.

  2. Navigate to Configuration -> User Groups and click Add user group. The name should be the same as the name of the external group. Assign role(s) to the user group and click Save.

Note that selecting users for the CloudCasa user group is not required as the users will automatically be assigned the roles from the user group once they log in.

Email Configuration

CloudCasa needs to send emails in some cases, such as when a user is invited to join an organization or to notify users of job failures. CloudCasa supports two methods of sending email: SMTP and Azure Communication Service.

For the SMTP method, configure the following parameters:

  • emails.enabled. Set it to true.

  • emails.method. Set it to smtp.

  • emails.smtp.startTLS. Controls whether TLS is used for SMTP connection. By default, it is set to true.

  • emails.smtp.password

  • emails.smtp.sender_email

  • emails.smtp.host

  • emails.smtp.port. E.g. “587”

For the Azure Communication Service method, configure the following parameters:

  • emails.enabled. Set it to true.

  • emails.method. Set it to azureacs.

  • emails.azureacs.username

  • emails.azureacs.password

  • emails.azureacs.sender_email

Container Registries

CloudCasa Images

By default, cloudcasa-server uses two container registries:

images:
  registry: cloudcasa.azurecr.io/catalogicsoftware
  agentRegistry: cloudcasaAgent.azurecr.io/catalogicsoftware

  • images.registry: cloudcasa.azurecr.io/catalogicsoftware

    Registry for CloudCasa server images. Authentication required.

    To create a pull secret for use with the default registry:

    kubectl create secret docker-registry myregistrykey \
 --docker-server=https://cloudcasa.azurecr.io \
 --docker-username=<username> \
 --docker-password=<password> \
 --namespace=cloudcasa-server

  • images.agentRegistry: cloudcasaAgent.azurecr.io/catalogicsoftware

    Registry for CloudCasa Agent images. This repository allows anonymous pulls so that all images required for agent installation are publicly available. This means that as long as a cluster is connected to the internet (and CloudCasa), it can install the CloudCasa client.

Other Images

CloudCasa also uses some publicly available images from Dockerhub and GCR:

docker.io/mongo
docker.io/redis
docker.io/envoyproxy/envoy-alpine
docker.io/minio/minio
docker.io/fluent/fluentd-kubernetes-daemonset
gcr.io/kubebuilder/kube-rbac-proxy

Air-gapped installation

Accessing the Helm Chart

If installing from a machine that does not have access to the helm repository (helm.cloudcasa.io), you must instead download the helm chart from the CloudCasa Support Portal at https://support.cloudcasa.io.

To download the helm chart:

  1. Log into the CloudCasa support portal using your email (you will receive a temporary access code)

  2. Navigate to the relevant release

  3. Find “Files” section (under “Change log” and “Images”)

  4. Click on cloudcasa-server-<version>.tgz to download the chart

Now use the .tgz file directly in helm install/upgrade commands, replacing cloudcasa/cloudcasa-server with the path to the downloaded .tgz file.

helm install cloudcasa-server cloudcasa-server-<version>.tgz --namespace cloudcasa-server ...

Custom Container Registry Setup

In order to install CloudCasa in an air-gapped environment, all the images used by CloudCasa need to be copied to a container registry of your choice (the full list of images can be found in CloudCasa support portal). Note that wherever the images are copied, the path in the image name must be maintained. For example, “catalogicsoftware/amds-apiserver-selfhosted” must be accessible at “myregistry.io/mynamespace/catalogicsoftware/amds-apiserver-selfhosted”.

Here are a couple of additional examples:

  • copy “docker.io/fluent/fluentd-kubernetes-daemonset” to “myregistry.io/mynamespace/fluent/fluentd-kubernetes-daemonset”

  • copy “gcr.io/kubebuilder/kube-rbac-proxy” to “myregistry.io/mynamespace/kubebuilder/kube-rbac-proxy”

If the images have no prefix (e.g. “redis”), copy them to the target at the top level:

  • copy “docker.io/redis” to “myregistry.io/mynamespace/redis”

Custom Container Registry Usage

To use a custom registry some values must be set/overridden:

  • images.userContainerRegistry: The URL of the container registry, including the root directory. Ex: “myregistry.io/mynamespace”

  • images.registry: Registry path used for all cloudcasa-server images. To use a custom registry, set this to “catalogicsoftware”

  • images.agentRegistry: Optional registry path used for CloudCasa agent images (publicly available to all clusters) To use a custom registry set images.agentRegistry to null, otherwise the default value will pull from the public CloudCasa repository

Altogether the values for a custom registry should look like this:

images:
  userContainerRegistry: myregistry.io/mynamespace
  registry: catalogicsoftware
  agentRegistry:
  imagePullPolicy: IfNotPresent

Notes

  • Normally, the cluster agent is installed by running a kubectl command such as the following:

    $ kubectl apply -f <AGENT-YAML>

    The correct command is displayed in the UI when a cluster resource is created. If you used a self-signed certificate issuer while installing cloudcasa server, you need to use slightly different command:

    $ curl -k <AGENT-YAML> | kubectl apply -f -

  • CloudCasa supports backup and restore of AWS RDS databases. (See Databases for details). However, if you are not planning to use this functionality, “Databases” tab can be hidden from the menu by setting the helm parameter ui.enable_databases to false.