Verrazzano Enterprise Container Platform – Provision Additional Clusters

Docs: Cluster API

Mon, 01 Jan 0001 00:00:00 +0000

The Cluster API (CAPI) project was developed as a sub-project of Kubernetes and strives to standardize a set of Kubernetes-style APIs for cluster management. External organizations can then build upon these standard APIs to develop custom cluster management solutions.

Learn more about CAPI at Kubernetes Cluster API Documentation.

Verrazzano incorporates CAPI functionality through the clusterAPI component, which provides the ability to quickly design and deploy clusters and then continue managing your clusters throughout their life cycle, all from within Verrazzano.

NOTE

The terminology around clusters differs between CAPI and Verrazzano though the underlying concepts are the same. What CAPI calls Management and Workload clusters are equivalent to admin and managed clusters, respectively, in Verrazzano.

CAPI splits cluster management responsibilities across three main components, which it calls providers:

Infrastructure providers standardize the host environment by provisioning any infrastructure or computational resources required by the cluster or machine.
Bootstrap providers streamline the node creation process by converting servers into Kubernetes nodes.
Control plane providers work with the Kubernetes API to regulate your clusters, ensuring that they always strive toward a desired state.

Currently, Verrazzano supports the CAPI provider for Oracle Cloud Native Environment (CAPOCNE) which bundles a bootstrap and a control plane provider together and works with the CAPOCI infrastructure provider offered by Oracle Cloud Infrastructure (OCI).

During the setup process, the bootstrap provider converts a cluster into an admin cluster - a Kubernetes cluster that controls any other, subordinate or managed clusters. It generates certificates, starts and manages the creation of additional nodes, and handles the addition of control plane and worker nodes to the cluster.

Next, a CAPI infrastructure provider will provision the first instance on the cloud provider and generate a provider ID, a unique identifier that any future nodes and clusters will use to associate with the instance. It will also create a kubeconfig file. The first control plane node is ready after these are created.

After the admin cluster is up and running, you can use the clusterAPI component to create additional managed clusters.

Docs: Create OCNE Clusters running on OCI

Mon, 01 Jan 0001 00:00:00 +0000

Before you begin

You’ll need to:

Set up an Oracle Cloud Infrastructure (OCI) account with a compartment
Generate an SSH key pair to use for cluster authentication

Create a new OCNE cluster on OCI

To provision new Oracle Cloud Native Environment (OCNE) managed clusters on OCI, complete the following steps:

Log in to the console. To find the console URL for your cluster, refer to Get console URLs and use the rancherURL value.
Open the navigation menu and select Cluster Management.
From the left menu, select Cloud Credentials, and then click Create. Cloud credentials store the credentials for your cloud infrastructure provider.
Choose Oracle.
Provide a name for the cloud credential and then fill in the fields. You can find the required information in your OCI configuration file. Click Create.
From the left menu, select Clusters, and then click Create.
Select Oracle OCNE on OCI and provide a name for the cluster.
Expand Member Roles to add any users that you want grant access to this cluster and their permissions.
Expand Labels and Annotations to configure Kubernetes labels and annotations for the cluster.
Select the cloud credentials that you created. Ensure that the appropriate Region and Compartment are selected from their drop-down lists.
Click Next.
Set up your network. Choose Quick Create to create a new virtual cloud network (VCN) configured to the specifications required for an OCNE cluster or Existing Infrastructure to use a VCN that’s already configured in your OCI account.
- If you choose the Existing Infrastructure option, then select the compartment where your VCN is located from the VCN Compartment drop-down list, then the VCN itself from the Virtual Cloud Network drop-down list. Next, select subnets within the VCN for each of the Cloud Plane Subnet, Load Balancer Subnet, and Worker Node Subnet drop-down lists. See Configure a VCN for OCNE for requirements.
- The VCN compartment does not need to match the compartment specified in the cloud credential.
Click Next.
Choose a Node Image from the drop-down list.

You can choose to override the default Node Image. Expand OCNE Image Configuration and provide a custom image OCID. If your custom image includes OCNE binaries, then you can select Skip OCNE Dependency Installation to avoid duplicates.
Copy or upload an SSH public key to manage authentication of the cluster. Your SSH public key is installed on the cluster nodes, enabling SSH after the cluster is created.
Configure the cluster control plane. Select the OCNE Version first because it determines which Kubernetes versions are available, then choose a Kubernetes Version and a Control Plane Shape. You can leave the rest of the options at their default setting or modify them as needed.

Under Advanced, you can choose to edit image tags for ETCD, CoreDNS, and Calico, or whether to install OCI CCM/CSI and Calico.
Add node pools to your cluster. Clusters without node pools will schedule pods on control plane nodes.
(Optional) Install Verrazzano on the cluster. Choose a Verrazzano version from the drop-down list. You can also expand Advanced to make changes to the Verrazzano Resource YAML. By default, Verrazzano is installed using the managed-cluster profile which enables a limited set of components on the cluster.
Expand Advanced Settings to make additional modifications to the default settings of your new cluster.
- YAML Manifests: Supply additional YAML manifests that are automatically installed after cluster creation. The total size of all additional YAML manifests may not exceed 500 KB.
- Cluster Networking: Configure cluster IP ranges and proxy settings.
- Container Registry: Specify a private registry for your container.
Click Create. It can take up to 30 minutes to provision all of the resources for your cluster, particularly for multi-node clusters.

When your cluster finishes provisioning, you can access it from the main Cluster Management page.

For help troubleshooting cluster creation, see OCNE Cluster Creation Issues.

Docs: Customize clusterAPI

Mon, 01 Jan 0001 00:00:00 +0000

The clusterAPI component allows you to quickly create managed clusters and manage them in the Verrazzano console. See Cluster API for more information.

You can customize the clusterAPI component using component overrides in the Verrazzano custom resource. Refer to the clusterAPI reference to see which overrides are available.

Upgrade providers

You can upgrade the individual providers that make up the clusterAPI component. This allows you to take advantage of new features in the providers without upgrading your entire Verrazzano installation.

This example customizes the clusterAPI component as follows:

Sets the version of both the OCNE bootstrap provider and the OCNE control plane provider to 1.6.1

NOTE: Because the OCNE bootstrap and OCNE control plane providers are bundled together, make sure to set their overrides to the same version or it may lead to unexpected behavior.
Sets the version of the OCI infrastructure provider to 0.9.0

apiVersion: install.verrazzano.io/v1beta1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  components:
   clusterAPI:
       enabled: true
       overrides:
       - values:
           defaultProviders:
               ocneBootstrap:
                   version: 1.6.1
               ocneControlPlane:
                   version: 1.6.1
               oci:
                   version: v0.9.0

Use a private registry

If you want to upgrade the clusterAPI providers but your Verrazzano instance is installed in a disconnected environment, you can configure the clusterAPI component to retrieve the provider assets from another location, instead of the public repository.

Place the provider assets in a location that is accessible by your disconnected Verrazzano environment.
For each provider that you want to upgrade, add a url override and then enter the path to the provider assets in the private registry for your environment.

For example:

apiVersion: install.verrazzano.io/v1beta1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  components:
   clusterAPI:
       enabled: true
       overrides:
       - values:
           defaultProviders:
               ocneBootstrap:
                   url: https://my.private.network/cluster-api-provider-ocne/releases/tag/v1.6.1
               ocneControlPlane:
                   url: https://my.private.network/cluster-api-provider-ocne/releases/tag/v1.6.1
               oci:
                   url: https://my.private.network/cluster-api-provider-oci/releases/tag/v0.9.0