Hi everyone!  In a previous post I’ve described how to enable the Kasten K10 Multi-Cluster Manager using the k10multicluster tool.  In this post, I’ll describe the process of enabling the Multi-Cluster Manager using Helm, which is available since the last release (6.5.8).

 

According to Kasten documentation: In a multi-cluster setup, one cluster is designated as primary, while all others are designated as secondaries.

• Primary: The cluster from which the K10 Multi-Cluster Manager will be accessed is designated as primary.  The primary cluster defines policies and other configuration centrally. Centrally defined policies and configuration can then be distributed to designated clusters to be enacted.  This provides a single pane of glass through which all clusters in the system are managed.
• Secondary: The secondary clusters receive policies and other configuration from the primary cluster. Once policies are distributed to a secondary, the local K10 installation enacts the policy. This ensures that the policy will continue to be enforced, even if disconnected from the primary.

 

In this blog I’ll be joining 2 Google GKE clusters to the Multi-Cluster Manager.

IMPORTANT: When using this method (Helm) to enable Multi-Cluster Manager, by default from the Multi-Cluster Manager Dashboard you will get some visibility about the secondary clusters and their status, with some basic information like the number of applications, policies and so on.  But by default, from Multi-Cluster Manager Dashboard you won’t get access to the Secondary Clusters.   This is due to some changes in the Multi-Cluster Manager architecture.  In case you need to get access to Secondary Clusters from Multi-Cluster Manager dashboard, I’ll include a workaround at the end of this post using the k10multicluster tool.

 

Pre-requisites

So, what do we need to enable Multi-Cluster Manager?  Very simple:

• We need 1 Kubernetes cluster with Kasten already installed, which we will set as the Primery cluster in the Multi-Cluster Manager.   If Kasten isn’t installed yet, during the installation the Multi-Cluster Manager can be enabled immediately using Helm.
• 1 or more Kubernetes clusters with Kasten already installed, which we will set as Secondary clusters in the Multi-Cluster Manager.
• The Kasten’s ingress (it could be via Ingress, Load Balancer or an OpenShit Route) in the Primary Cluster must be accessible by all Secondary clusters.
• [Optional – Required when using k10multicluster tool] The Kasten’s ingress (it could be also via Load Balancer or an OpenShit Route) in the Secondary Clusters must be accessible by the Primary clusters.
• [Optional – Required when using k10multicluster tool] The API Server in the Secondary Clusters must be accessible by the Primary clusters.

 

 

Enabling the Multi-Cluster Manager in the Primary Cluster

Remember, the Primary cluster  is the cluster used to connect the the K10 Multi-Cluster Manager  dashboard.  To enable the Multi-Cluster Manager in the Primary Cluster, we need to use the following Helm command:

 

 

The required parameters are:

 

Once this process is completed, which usually takes just a couple of seconds, you will see that a new namespace called kasten-io-mc is crearted in the Primary Cluster.

If you connect to the K10 Dashboard in the Primary Cluster, you will see that the Multi-Cluster Manager is already enabled, with just one cluster for now, the Primary Cluster:

 

Adding a Secondary Cluster to the Multi-Cluster Manager

Once the Primary Cluster is already configured to use the Multi-Cluster Manager, we can add Secondary Clusters.  In order to do this, we need to complete the following steps:

Create a Join Token in the Primary Cluster

In order to add a Secondary Cluster to the Multi-Cluster Manager, first you need to create a join token secret in the Primary Cluster.  This secret will generate a join token that later will be used by the Secondary Cluster to connect with the Primary Cluster.  The join token secret can be created with the following YAML manifest:

This will create a secret with a random name, as we can see in the following image:

Once the secret is created, a token and an endpoint will be created automatically and they will be available in the secret itself.

• The endpoint is the primary cluster’s endpoint for multi-cluster communication.
• The token is used for authenticating the joining request from a secondary cluster.

 

 

Create a Join Secret in the Secondary Cluster

The next step will be creating a mc-join secret in the secondary cluster we want to add to the Multi-Cluster Manager, with the token and endpoint we got from the previous step.  For the secondary cluster we can optionally specify a cluster-name, basically an alias for the Secondary Cluster in the Multi-Cluster Manager.  If this cluster-name isn’t specified, then a name will be generated automatically.  The Join Secret can be created with the following YAML manifest:

 

 

This will trigger a join from the Secondary to the Primary Cluster. Then the Primary Cluster will validate the token and admit the cluster.   In order to check if the whole process has been completed successfully, you can check if the mc-join-status secret has been created in the kasten-io namespace.

Then, you can edit the secret and check the status, to make sure the status is “Joined” (base64 coded), as you can see in the following image:

 

And of course, you can now see the  secondary cluster added to the Multi-Cluster Manager.   As explained before, you can “see” the secondary clusters, but you can’t access to them using this method, at least in current version (6.5.8).

• From the drop-down menu, you can see the Secondary Cluster, but you can’t select it as it is disabled.
• In the Clusters list, you can see the secondary cluster and some information about applications, policies and actions, but you can’t click on any of them.  You can observe that the Secondary Cluster is in a dashed-line box, which means you can’t actually access the cluster from Multi-Manager Cluster.  If you need access please refer the workaround in the next section.

 

 

Workaround – Adding a Secondary Cluster

IMPORTANT: Only use the k10multicluster tool if ingress access to Secondary Clusters is required or when you are using and older version of Kasten (6.0.12 or earlier).   For all other use-cases, please use the method described before with Helm and Join Tokens/Secrets, or using Kubectl as described in the official documentation.

In case you need to get access to Secondary Cluster from Multi-Cluster Manager, use the following instructions:

Enable the Multi-Cluster Manager in the Primary Cluster using Helm as described before.  Then we can use the k10multicluster tool to bootstrap the secondary cluster using the following command:

 

When using this tool we need to provide the following parameters in the previous command:

• primary-context: Context name (from kubeconfig) of the Primary Cluster
• primary-name: Name (alias) of the Primary Cluster in the Multi-Cluster Manager dashboard.
• secondary-context: Context name (from kubeconfig) of the Secondary Cluster we are adding.
• secondary-name: Name (alias) of the Secondary Cluster to be shown in the Multi-Cluster Manager dashboard.
• secondary-cluster-ingress: Kasten K10 Ingress URL of the Secondary Cluster.

 

Once this command is completed, you can go to the dashboard and you will see the Secondary Cluster with a message highlighting than you don’t have permissions to manage the Secondary Cluster from the dashboard.   Click in “Grant Permissions” to grant the proper privileges.

The Multi-Cluster Manager dashboard will pre-fill the Form to provide access to Secondary Cluster with the current user.  Make the changes you need, or just click save to grant the proper privileges.

 

Now you access both clusters (Primary and Secondary) from Multi-Cluster Manager dashboard.