Verrazzano Enterprise Container Platform – Deploy and Manage Applications

Docs: About Applications in Verrazzano

Mon, 01 Jan 0001 00:00:00 +0000

Verrazzano uses the Open Application Model (OAM) specification to provide a layered approach to describing and deploying applications. OAM is a specification developed within the Cloud Native Computing Foundation (CNCF). Verrazzano is compliant with the OAM specification version 0.2.1.

An ApplicationConfiguration is a composition of Components. Components encapsulate application implementation details. Application deployers apply Traits and Scopes to customize the Components for the environment.

The OAM specification supports extensibility. The behavior of the platform can be extended by adding OAM compliant definitions and controllers. Specifically, new workload, Trait, and Scope definitions can be added. These definitions can be referenced by Components and application configurations and are processed by custom controllers.

Application configurations

An ApplicationConfiguration is a collection of references to Components. A set of Traits and Scopes can be applied to each Component reference. The platform uses these Components, Traits, and Scopes to generate the final application resources during deployment.

The following sample shows the high level structure of an ApplicationConfiguration.

apiVersion: core.oam.dev/v1alpha2
kind: ApplicationConfiguration
...
spec:
  components:
    - componentName: example-component-1
      traits:
        ...
      scopes:
        ...
    - componentName: example-component-2
        ...

Components

A Component wraps the content of a workload. The platform extracts the workload during deployment and creates new resources that result from the application of Traits and Scopes. Verrazzano and the OAM specification provide several workloads, for example VerrazzanoHelidonWorkload and ContainerizedWorkload. The workloads also can be any Kubernetes resource. For some Kubernetes resources, the oam-kubernetes-runtime operator may need to be granted additional permission.

A Component can also be parameterized; this allows the workload content to be customized when referenced within an ApplicationConfiguration. See the OAM specification for details.

The following sample shows the high level structure of a Component.

apiVersion: core.oam.dev/v1alpha2
kind: Component
...
spec:
  workload:
    ...
  parameters:
    ...

For information about Traits, see Verrazzano Application Traits.
For information about Workloads, see Verrazzano Application Workloads.

Deployment

An application deployment occurs in Verrazzano through a number of Kubernetes controllers, reading and writing various resources. Each controller processes application resources, and generates or updates other related resources. Different types of controllers process different levels of application resources.

The ApplicationConfiguration controller processes ApplicationConfiguration and Component resources. This controller extracts and stores a workload for each Component referenced within ApplicationConfiguration resources. Verrazzano implements the ApplicationConfiguration controller within the oam-kubernetes-runtime operator. Similarly, the ApplicationConfiguration controller extracts and stores Trait resources associated with Component resources in the ApplicationConfiguration.

The workload controllers process workload resources created by the ApplicationConfiguration controller, for example ContainerizedWorkload or VerrazzanoWebLogicWorkload. This controller processes these workload resources and generates more specific runtime resources. For example, the ContainerizedWorkload controller processes a ContainerizedWorkload resource and generates a Deployment resource. The VerrazzanoWebLogicWorkload controller processes a VerrazzanoWebLogicWorkload resource and generates a Domain resource. These controllers may take into account Traits and Scopes that are applied to the workload’s Component references within the ApplicationConfiguration. Verrazzano implements these workload controllers in two operators. Verrazzano specific workloads, for example VerrazzanoHelidonWorkload, are processed by a controller within the verrazzano-application-operator. Workloads defined by OAM, for example ContainerizedWorkload, are processed by a controller with the oam-kubernetes-runtime operator.

The Trait controllers process Trait resources created by the ApplicationConfiguration controller, for example MetricsTrait. The ApplicationConfiguration controller records the Component to which it was applied within each extracted Trait. The Trait controllers process extracted Trait resources, and generate or update other related resources. For example, the IngressTrait controller within the verrazzano-application-operator processes IngressTrait resources and generates related Gateway and VirtualService resources. The same operator contains a MetricsTrait controller which processes MetricsTrait resources and adds annotations to related resources such as Deployments.

Scope controllers read Scope resources updated by the ApplicationConfiguration controller during deployment. The ApplicationConfiguration controller updates the Scope resources with references to each Component to which the Scope is applied.

The following diagram shows the relationships between the resources and controllers described previously.

The following diagram, based on the hello-helidon example, shows the processing of resources from a Kubernetes operator perspective. Controllers within the oam-kubernetes-runtime process the ApplicationConfiguration and Component resources and generate VerrazzanoHelidonWorkload and IngressTrait. Then controllers within the verrazzano-application-operator process the VerrazzanoHelidonWorkload and IngressTrait resources to generate Deployment, VirtualService, and other resources.

Docs: Deploy Applications With the Open Application Model

Mon, 01 Jan 0001 00:00:00 +0000

Developing and deploying applications in Verrazzano consists of:

Packaging the application as a Docker image.
Publishing the application’s Docker image to a container registry.
Applying the application’s Verrazzano components to the cluster.
Applying the application’s Verrazzano applications to the cluster.

This section does not provide the full details for the first two steps. An existing example application Docker image has been packaged and published for use.

Verrazzano supports application definition using the Open Application Model (OAM). Verrazzano applications are composed of components and application configurations. This section demonstrates creating OAM resources that define an application as well as the steps required to deploy those resources.

Docs: Deploy Applications Using Kubernetes Objects

Mon, 01 Jan 0001 00:00:00 +0000

Verrazzano and OAM provide workloads and Traits to define and customize applications. However, some situations may require resources beyond those provided. In those cases, you can use other existing Kubernetes resources. The todo-list example takes advantage of this capability in several Components to support unique Service and ConfigMap requirements.

Most Kubernetes resources can be embedded as a workload within a Component. The following sample shows how a Deployment can be embedded as a workload within a Component. The oam-kubernetes-runtime operator will process the Component and extract the Deployment to a separate resource during deployment.

apiVersion: core.oam.dev/v1alpha2
kind: Component
...
spec:
  workload:
    kind: Deployment
    apiVersion: apps/v1
    name: ...
    spec:
      selector:
        ...
      template:
        ...

Most Kubernetes resources can also be embedded as a Trait within an ApplicationConfiguration. The following sample shows how an Ingress can be embedded as a trait within an ApplicationConfiguration. The oam-kubernetes-runtime operator will process the ApplicationConfiguration and extract the Ingress to a separate resource during deployment. In the following sample, note that the Ingress is the Kubernetes Ingress, not the IngressTrait provided by Verrazzano.

apiVersion: core.oam.dev/v1alpha2
kind: ApplicationConfiguration
...
spec:
  components:
    - componentName: ...
      traits:
        - trait:
            apiVersion: networking.k8s.io/v1beta1
            kind: Ingress
            ...
            spec:
              rules:
                ...

The oam-kubernetes-runtime operator has the following limited set of cluster role privileges, by default.

API Groups	Resources	Verbs
	`configmaps`, `events`, `services`	`create`, `delete`, `deletecollection`, `get`, `list`, `patch`, `update`, `watch`
	`persistentvolumeclaims`	`create`, `delete`, `deletecollection`, `get`, `list`, `patch`, `update`
`apps`	`deployments`, `controllerrevisions`	`create`, `delete`, `deletecollection`, `get`, `list`, `patch`, `update`, `watch`
`core.oam.dev`	`*`	`create`, `delete`, `deletecollection`, `get`, `list`, `patch`, `update`, `watch`
`oam.verrazzano.io`	`*`	`create`, `delete`, `deletecollection`, `get`, `list`, `patch`, `update`, `watch`

Your cluster administrator may need to grant the oam-kubernetes-runtime operator additional privileges to enable the use of some Kubernetes resources as workloads or traits. Create additional roles and role bindings for the specific resources to be embedded as workloads or traits. The following examples of ClusterRole and ClusterRoleBinding show how oam-kubernetes-runtime can be granted privileges to manage Ingress resources.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: oam-kubernetes-runtime-ingresses
rules:
  - apiGroups:
    - networking.k8s.io
    - extensions
    resources:
    - ingresses
    verbs:
    - create
    - delete
    - get
    - list
    - patch
    - update

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: oam-kubernetes-runtime-ingresses
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: oam-kubernetes-runtime-ingresses
subjects:
  - kind: ServiceAccount
    name: oam-kubernetes-runtime
    namespace: verrazzano-system

Docs: Deploy Applications in a Multicluster Environment

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Continuous Delivery

Mon, 01 Jan 0001 00:00:00 +0000

Argo CD is a Kubernetes deployment tool that uses Git repositories as the source of truth. It monitors running applications and compares the deployed state against the desired one in Git. Argo CD lets you visualize the differences and provides methods to automatically or manually update the live state with the desired target state. For more information, see the Argo CD documentation.

In a multicluster Verrazzano environment, Argo CD integration depends on Rancher being enabled on the admin cluster. Argo CD connects to managed clusters using the Rancher proxy, to create the required resources for Argo CD cluster registration.

Docs: Troubleshoot Application Deployment Issues

Mon, 01 Jan 0001 00:00:00 +0000

During application deployment, the oam-kubernetes-runtime and verrazzano-application-operator cooperate through the generation and update of Kubernetes resources. The oam-kubernetes-runtime processes the ApplicationConfiguration and Component resources provided by the user and generates workload and Trait resources. The verrazzano-application-operator processes Verrazzano specific workload and Trait resources. These are then used to generate additional child and related resources.

Troubleshooting application deployments should follow three general steps:

Review the status of the oam-kubernetes-runtime and verrazzano-application-operator operator pods.
Review the logs of the oam-kubernetes-runtime and verrazzano-application-operator operator pods.
Review the resources generated by the oam-kubernetes-runtime and the verrazzano-application-operator.

Review oam-kubernetes-runtime operator status

For application deployment to succeed, the oam-kubernetes-runtime pod must have a status of Running.

Use the following command to get the pod status:

$ kubectl get pods \
    -n verrazzano-system \
    -l app.kubernetes.io/name=oam-kubernetes-runtime

If the pod status is not Running, then see the instructions for reviewing the oam-kubernetes-runtime pod logs.

Review verrazzano-application-operator operator status

For application deployment to succeed, the verrazzano-application-operator pod must have a status of Running.

Use the following command to get the pod status:

$ kubectl get pods \
    -n verrazzano-system \
    -l app=verrazzano-application-operator

If the pod status is not Running, then see the instructions for reviewing the verrazzano-application-operator logs.

Review oam-kubernetes-runtime operator logs

Review the oam-kubernetes-runtime pod logs for any indication that pod startup or the generation of workloads or traits has failed.

Use the following command to get the logs:

$ kubectl logs \
    -n verrazzano-system \
    -l app.kubernetes.io/name=oam-kubernetes-runtime

Review verrazzano-application-operator logs

Review the verrazzano-application-operator logs for any indication that pod startup or resource generation has failed.

Use the following command to get the logs:

$ kubectl logs \
    -n verrazzano-system \
    -l app=verrazzano-application-operator

Review generated workload resources

The processing of a Component reference within an ApplicationConfiguration results in the generation of workloads. For example, a referenced Component might result in the generation of a VerrazzanoHelidonWorkload workload resource. In turn, the VerrazzanoHelidonWorkload workload resource will be processed and result in the generation of related Deployment and Service resources.

If the expected workload resource, for example VerrazzanoHelidonWorkload, is missing, then review the oam-kubernetes-runtime logs. If the expected related resources, for example Deployment or Service, are missing, then review the verrazzano-application-operator logs.

The following commands are examples of checking for the resources related to a VerrazzanoHelidonWorkload deployment:

$ kubectl get -n hello-helidon verrazzanohelidonworkload hello-helidon-workload
$ kubectl get -n hello-helidon deployment hello-helidon-deployment
$ kubectl get -n hello-helidon service hello-helidon-deployment

Review generated Trait resources

The processing of traits embedded with an ApplicationConfiguration results in the generation of Trait resources. For example, an IngressTrait embedded within an ApplicationConfiguration will result in the generation of an IngressTrait resource. In turn, the IngressTrait resource will be processed and result in the generation of related Certificate, Gateway, and VirtualService resources.

If the expected Trait resource, for example IngressTrait, is missing, then review the oam-kubernetes-runtime logs. If the expected related resources, for example Certificate, Gateway, and VirtualService, are missing, then review the verrazzano-application-operator logs.

The following commands are examples of checking for the resources related to an IngressTrait:

$ kubectl get -n hello-helidon ingresstrait hello-helidon-ingress
$ kubectl get -n istio-system Certificate hello-helidon-hello-helidon-appconf-cert
$ kubectl get -n hello-helidon gateway hello-helidon-hello-helidon-gw
$ kubectl get -n hello-helidon virtualservice hello-helidon-ingress-rule-0-vs

Check for RBAC privilege issues

The use of generic Kubernetes resources as workloads and traits can result in deployment failures if privileges are insufficient. In this case, the oam-kubernetes-runtime logs will contain errors containing the term forbidden.

The following command shows how to query for this type of failure message:

$ kubectl logs \
    -n verrazzano-system \
    -l app.kubernetes.io/name=oam-kubernetes-runtime | grep forbidden

Check resource owners

Kubernetes maintains the child to parent relationship within metadata fields.

The following example returns the parent of the IngressTrait, named hello-helidon-ingress, in the hello-helidon namespace:

$ kubectl get IngressTrait \
    -n hello-helidon hello-helidon-ingress \
    -o jsonpath='{range .metadata.ownerReferences[*]}{.name}{"\n"}{end}'

The results of this command can help identify the lineage of a given resource.

Some resources also record the related resources affected during their processing. For example, when processed, an IngressTrait will create related Gateway, VirtualService, and Certificate resources.

The following command is an example of how to obtain the related resources of an IngressTraits:

$ kubectl get IngressTrait \
    -n hello-helidon hello-helidon-ingress \
    -o jsonpath='{range .status.resources[*]}{.kind}: {.name}{"\n"}{end}'

The results of this command can help identify which other resources, the given resource affected.

Verrazzano Enterprise Container Platform – Deploy and Manage Applications

Docs: About Applications in Verrazzano

Application configurations

Components

Deployment

Docs: Deploy Applications With the Open Application Model

Docs: Deploy Applications Using Kubernetes Objects

Docs: Deploy Applications in a Multicluster Environment

Docs: Continuous Delivery

Docs: Troubleshoot Application Deployment Issues

Review oam-kubernetes-runtime operator status

Review verrazzano-application-operator operator status

Review oam-kubernetes-runtime operator logs

Review verrazzano-application-operator logs

Review generated workload resources

Review generated Trait resources

Check for RBAC privilege issues

Check resource owners

Check related resources