Verrazzano Enterprise Container Platform – Verrazzano Application Workloads

Docs: About Verrazzano Workloads

Mon, 01 Jan 0001 00:00:00 +0000

Components contain an embedded workload. Verrazzano and the OAM specification provide several workloads, for example VerrazzanoWebLogicWorkload and ContainerizedWorkload. Workloads can also be any Kubernetes resource.

The following sample shows a VerrazzanoHelidonWorkload workload embedded within a Component.

apiVersion: core.oam.dev/v1alpha2
kind: Component
...
spec:
  workload:
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: VerrazzanoHelidonWorkload
    spec:
      deploymentTemplate:
        podSpec:
          containers:
            - name: example-container
              image: ...
              ...

A workload can optionally have an associated WorkloadDefinition. This provides the platform with information about the schema of the workload. A WorkloadDefintion is typically provided by the platform, not an end user.

Scopes

Scopes customize Component workloads and generate related resources during deployment. An ApplicationConfiguration contains Scope references instead of the Scope’s content being embedded. The platform will update the Scopes with a reference to each applied Component. This update triggers the related operator to process the Scope.

The following sample shows a reference to a HealthScope named example-health-scope.

apiVersion: core.oam.dev/v1alpha2
kind: ApplicationConfiguration
...
spec:
  components:
    - componentName: example-component
      scopes:
        - scopeRef:
            apiVersion: core.oam.dev/v1alpha2
            kind: HealthScope
            name: example-health-scope
        ...

The following sample shows the configuration details of the referenced HealthScope.

apiVersion: core.oam.dev/v1alpha2
kind: HealthScope
metadata:
  name: example-health-scope
spec:
  probe-method: GET
  probe-endpoint: /health

Each Scope type can optionally have an associated ScopeDefinition. This provides the platform with additional information about processing the Scope:

The Scope’s schema
The workload types to which the Scope can be applied
The field within the Scope used to record related Component references

A ScopeDefintion is typically provided by the platform, not an end user.

The Verrazzano platform provides several workload definitions and implementations:

The VerrazzanoWebLogicWorkload is used for WebLogic workloads. See WebLogic Workload.
The VerrazzanoCoherenceWorkload is used for Coherence workloads. See Coherence Workload.
The VerrazzanoHelidonWorkload is used for Helidon workloads. See Helidon Workload.

OAM ContainerizedWorkload

The ContainerizedWorkload should be used for long-running container workloads which are not covered by the workload types described previously. This workload type is similar to the Deployment workload. It is provided to ensure that OAM can be used for non-Kubernetes deployment environments. See the OAM specification.

Docs: WebLogic Workload

Mon, 01 Jan 0001 00:00:00 +0000

WebLogic Server is a widely-used enterprise application server for managing Java Enterprise Edition-based applications and is certified to run on Kubernetes using the WebLogic Kubernetes Operator. The WebLogic Kubernetes Operator (the “operator”) manages the WebLogic domain life cycle in Verrazzano. The WebLogic Domain custom resource (CR) specifies the configuration of the WebLogic domain. The operator monitors the WebLogic Domain CR and reconciles the domain by creating, updating, and deleting Kubernetes resources (Pods, Services, and such), as needed. Each pod is a WebLogic Server Administration Server or Managed Server.

The operator is installed in the verrazzano-system namespace and is also part of the Istio mesh deployed by Verrazzano.

NOTE

Verrazzano installs an instance of the WebLogic Kubernetes Operator. If you have a pre-existing instance of the operator, namespaces managed by each instance must be mutually exclusive. Do not label a namespace which is managed by the pre-existing WebLogic Kubernetes Operator, to also be managed by Verrazzano.

WebLogic OAM Component

In Verrazzano, WebLogic workloads are specified as a VerrazzanoWebLogicWorkload OAM Component and one component specifies exactly one WebLogic domain. An ApplicationConfiguration can contain multiple VerrazzanoWebLogicWorkload components and therefore, multiple WebLogic domains. You can specify Traits for one or more VerrazzanoWebLogicWorkload components. All WebLogic Domain CR fields can be specified in the VerrazzanoWebLogicWorkload.

The following is an example WebLogic OAM Component.

apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:…
spec:  
  workload:    
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: VerrazzanoWebLogicWorkload
    spec:      
      template:
 metadata:
    name: todo-domain
 spec:
           domainUID: tododomain
           domainHome: /u01/domains/tododomain          …

Verrazzano application operator

The Verrazzano application operator monitors the VerrazzanoWebLogicWorkload custom resource (CR) and creates, updates, and deletes the Domain CR based on the specification provided in the VerrazzanoWebLogicWorkload CR. Also, it modifies the WebLogic Domain CR to add Fluentd sidecar injection for logging and a WebLogic Monitoring Exporter configuration for metrics, if they do not already exist. The WebLogic Kubernetes Operator creates the WebLogic domain based on the WebLogic Domain CR.

WebLogic domain life cycle

The following are the life cycle stages of a WebLogic domain.

Create a WebLogic domain.
- Application containing WebLogic component is created.
- WebLogic component added to an existing application.
Delete a WebLogic domain.
- Application containing WebLogic component is deleted.
- WebLogic component removed from an existing application.
Scale a WebLogic domain.
- Modify the replicas field in the WebLogic Domain CR within the OAM Component specification.
Update a WebLogic domain.
- Modify the other fields field in the WebLogic Domain CR within the OAM Component specification.

Scale-in and scale-out a WebLogic domain by modifying the OAM Component replicas count. Delete the WebLogic domain by deleting the OAM application or removing the component from the application.

Istio integration

Verrazzano creates all WebLogic domain pods in an Istio mesh; all WebLogic network traffic uses mutual TLS authentication (mTLS).

The Envoy proxy sidecar exists in front of workloads for each service providing security, load balancing, metrics, and such. Traffic in and out of the pod goes through the proxy.

If the namespace is labeled istio-injection=enabled, then Istio puts the WebLogic domain in the Istio mesh. You should label all the namespaces istio-injection=enabled where the WebLogic domain is to be created, or WebLogic domain creation will fail. Also, you can label the namespaces when using a VerrazzanoProject, which by default, assigns the label to all the namespaces associated with the project. In the WebLogic Domain CR, the Verrazzano application operator sets the Istio enabled field.

apiVersion: v1
items:
- apiVersion: weblogic.oracle/v8
  kind: Domain
  …
  spec:
     …
      istio:
        enabled: true

Istio mesh ingress and egress

The Verrazzano installer creates the ingress gateway service. The ingress gateway is a LoadBalancer service; TLS is terminated at the Istio ingress gateway. Ingress to the WebLogic domain is optional; you can use an IngressTrait to enable it.

Istio ingress and routing for a single WebLogic domain

The Istio Gateway resource describes a proxy providing ingress to the Kubernetes cluster and the Istio mesh. The Gateway specifies the host, port, protocol, and so on, and is bound to a gateway service (LoadBalancer/NodePort). The VirtualService specifies routes to services and load balancing.

Example of an Istio Gateway resource:

apiVersion: networking.istio.io/v1beta1
  kind: Gateway
…
 spec:
    selector:
      istio: ingressgateway
    servers:
    - hosts:
      - todo-appconf.todo-list.172.18.0.230.nip.io #Host for this gateway server
      port:
        name: https
        number: 443
        protocol: HTTPS
      tls:
        credentialName: todo-list-todo-appconf-cert-secret #Secret containing TLS certificate
        mode: SIMPLE #Terminate TLS

Example of an Istio VirtualService resource:

apiVersion: networking.istio.io/v1beta1
  kind: VirtualService
…
  spec:
    gateways:
    - todo-list-todo-appconf-gw #Gateway resource reference
    hosts:
    - todo-appconf.todo-list.172.18.0.230.nip.io #Host that this VirtualService applies to.  A Gateway resource can have multiple hosts.
    http:
    - match:
      - uri:
          prefix: /todo
      route:
      - destination:
          host: tododomain-adminserver #Back end Kubernetes service.
          port:
            number: 7001

Istio ingress and routing for multiple WebLogic domains

Multiple Gateway resources use the same Istio ingress gateway service. Verrazzano creates a single Gateway and VirtualService per IngressTrait specified on the OAM Component.

Istio authorization policy

The Istio AuthorizationPolicy resource specifies access controls for WebLogic pods, other pods in the application, the Ingress gateway, and Prometheus.

Example Istio authorization policy:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
...
spec:
  rules:
  - from:
    - source:
        principals:
        - cluster.local/ns/todo-list/sa/todo-appconf
        - cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
        - cluster.local/ns/verrazzano-system/sa/verrazzano-monitoring-operator
        - cluster.local/ns/verrazzano-system/sa/weblogic-operator-sa
  selector:
    matchLabels:
      verrazzano.io/istio: todo-appconf

WebLogic metrics

Prometheus scrapes each WebLogic pod on the metrics port periodically.

If the trait doesn’t exist, Verrazzano will inject the MetricsTrait into the ApplicationConfiguration. The WebLogic Monitoring Exporter sidecar provides the metrics endpoint. If it doesn’t already exist, Verrazzano will inject the default Monitoring Exporter configuration into the WebLogic Domain CR. The Verrazzano application operator creates Prometheus Service Monitors with WebLogic targets and Verrazzano installs Grafana dashboards to view WebLogic metrics. The WebLogic Kubernetes Operator configures the Monitoring Exporter using a REST API and labels the pods with metrics-related labels. Metrics are scraped at /metrics on port 8080.

AppConfig default injection

Review the following example MetricsTrait from the Todo List ApplicationConfiguration. If it’s missing from ApplicationConfiguration, Verrazzano will inject the default MetricsTrait.

kind: ApplicationConfiguration
metadata:
  name: todo-appconf
...
spec:
  components:
    - componentName: todo-domain
      traits:
        - trait:
            apiVersion: oam.verrazzano.io/v1alpha1
            kind: MetricsTrait
            spec:
  …

Monitoring Exporter Component

Review the following example monitoringExporter configuration in the OAM Component.

workload:
  apiVersion: oam.verrazzano.io/v1alpha1
  kind: VerrazzanoWebLogicWorkload
…
monitoringExporter:
  imagePullPolicy: IfNotPresent
  configuration:    
     metricsNameSnakeCase: true
     domainQualifier: true
     queries:      
     - key: name
       keyName: location
       prefix: wls_server_
…

Pod annotations

The following annotations can be used for enabling metrics on pods:

prometheus.io/metricsEnabled: "true" - Enables metrics scraping.
prometheus.io/metricsPath: /metrics - Specifies metrics scraping path.
prometheus.io/metricsPort: "8080" - Specifies metrics scraping port.

Example:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    prometheus.io/path: /metrics
    prometheus.io/port: "8080"
    prometheus.io/scrape: "true"

Logging

WebLogic logs are sent to OpenSearch, which is installed in the Verrazzano cluster. The Fluentd sidecar is injected into each WebLogic pod to send server logs to stdout. The Fluentd DaemonSet in the verrazzano-system namespace sends logs to OpenSearch. In OpenSearch, logs are indexed by namespace.

Lift-and-Shift WebLogic applications

Verrazzano makes it easy for you to migrate WebLogic applications from on-premises installations to the cloud. You can use WebLogic Deploy Tooling (WDT) to build the domain model and the WebLogic Image Tool (WIT) to build the WebLogic domain image.

For detailed instructions, see the Lift-and-Shift Guide.

Deploy WebLogic applications in Verrazzano

Step 1. Create a WebLogic domain image.

To deploy a WebLogic domain in Kubernetes, first you need to create a Docker image for the WebLogic domain.
To create a WebLogic domain image using WebLogic Deploy Tooling (WDT), follow the instructions at:
- For a Model in Image domain home (recommended), see Model in Image.
- For a Domain home on a persistent volume (Domain on PV), see Domain on Persistent Volume (PV).

Step 2. Create a VerrazzanoWebLogicWorkload component.

To deploy and run the WebLogic domain image in Verrazzano, create the VerrazzanoWebLogicWorkload component that specifies the definition and parameters for the WebLogic domain contained in the image.
For an example VerrazzanoWebLogicWorkload Component resource created for a sample WebLogic domain, see the ToDo List example application component YAML file.
Note that WebLogic applications require that the container registry secret be specified in the Domain resource. Create a registry secret in the application namespace and specify the secret in the imagePullSecrets field of the WebLogic Domain Spec for the application.
- For all the options supported by the WebLogic domain configuration, see Domain.md.
- Find the VerazzanoWebLogicWorkload specification here.

Step 3. Create an ApplicationConfiguration for the WebLogic application.

Next, create an ApplicationConfiguration that uses the VerrazzanoWebLogicWorkload component you created for the WebLogic domain.
For an example ApplicationConfiguration using a VerrazzanoWebLogicWorkload component, see the ToDo List example application.

Step 4. Verify the domain.

Verrazzano creates the underlying domain Kubernetes resource from the VerrazzanoWebLogicWorkload component, which is then processed by the WebLogic Kubernetes Operator to create the Administration and Managed Server pods, and deploy the applications and resources associated with the WebLogic domain.
To verify that the WebLogic domain is up and running, follow the steps found here.

NOTE

Oracle recommends that you use the WebLogic Toolkit UI (WKTUI) for deploying WebLogic applications in Verrazzano. Follow the WKTUI documentation found here.

Database connections

Typically, WebLogic applications make database connections using the connection information present in the JDBCSystemResources created in a WebLogic domain. To implement this in Verrazzano, databases are deployed as separate components and the connection information is made available to the WebLogic domain using a WDT Model.

Step 1. Deploy the database in Verrazzano.

To deploy a database, you need to create the corresponding Component and ApplicationConfiguration that will run the database in a pod and expose its connection information as a Service.
For an example, look at the tododomain-mysql descriptor.

Step 2. Create a WebLogic resource ConfigMap.

Next, create a ConfigMap that will contain the JDBCSystemResource definition with connection information for the database.
For an example, see the tododomain-configmap definition in the ToDo List example application configuration.

Step 3. Configure the WebLogic domain to use the WebLogic resource ConfigMap.

You can configure the ConfigMap, containing the resource information for the JDBCSystemResource, in the configuration section of the VerrazzanoWebLogicWorkload component of the WebLogic domain.
```
...
    configuration:
        introspectorJobActiveDeadlineSeconds: 900
        model:
            configMap: tododomain-configmap
            domainType: WLS
...
```
For more details, see the ToDo List example application configuration.

Ingresses

To access the endpoints for a Java EE application deployed as part of a VerrazzanoWebLogicWorkload component, Verrazzano lets you specify an IngressTrait for the component which is then translated to an Istio ingress gateway and VirtualService. For an example, see the ToDo List example application, where the IngressTrait is configured for the application endpoint.

...
    - trait:
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: IngressTrait
    spec:
        rules:
        - paths:
            # application todo
            - path: "/todo"
                pathType: Prefix

...

Then, you can access the endpoint using the Istio gateway, as described in Step 8. Access the ToDo List application.

$ HOST=$(kubectl get gateways.networking.istio.io -n todo-list -o jsonpath={.items[0].spec.servers[0].hosts[0]})
$ ADDRESS=$(kubectl get service -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ curl -sk https://${HOST}/todo/ --resolve ${HOST}:443:${ADDRESS}

References

Docs: Coherence Workload

Mon, 01 Jan 0001 00:00:00 +0000

A Verrazzano application can contain any number of Coherence component workloads, where each workload is a standalone Coherence cluster, independent from other Coherence clusters in the application.

Verrazzano uses the standard Coherence Operator to provision and manage clusters, as documented at Coherence Operator. The Coherence Operator uses a CRD, coherence.oracle.com (Coherence resource), to represent a Coherence cluster. When a Verrazzano application with Coherence is provisioned, Verrazzano configures the default logging and metrics for the Coherence cluster. Logs are sent to OpenSearch and metrics to Prometheus. You can view this telemetry data using the OpenSearch Dashboards and Grafana consoles.

OAM Component

The custom resource YAML file for the Coherence cluster is specified as a VerrazzanoCoherenceWorkload custom resource. In the following example, everything under the spec: section is standard Coherence resource YAML that you would typically use to provision a Coherence cluster. Including this Component reference in your ApplicationConfiguration will result in a new Coherence cluster being provisioned. You can have multiple clusters in the same application with no conflict.

apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
  name: orders
  namespace: sockshop
spec:
  workload:
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: VerrazzanoCoherenceWorkload
    spec:
      template:
        metadata:
          name: orders-coh
        spec:
          cluster: SockShop
      ...

Life cycle

With Verrazzano, you manage the life cycle of applications using Component and ApplicationConfiguration resources. Typically, you would modify the Coherence cluster resource to make changes or to do lifecycle operations, like scale in and scale out. However, in the Verrazzano environment, the cluster resource is owned by the Verrazzano application operator and will be reconciled to match the Component workload resource. Therefore, you need to manage the cluster configuration by modifying the resource, either by kubectl edit or applying a new YAML file. Verrazzano will notice that the Component resource changed and will update the Coherence resource as needed.

Provisioning

When you apply the Component YAML file shown previously, Kubernetes will create a component.oam.verrazzano.io resource, but the Coherence cluster will not be created until you create the ApplicationConfiguration resource, which references the Coherence component. When the application is created, Verrazzano creates a Coherence custom resource for each cluster, which is subsequently processed by the Coherence Operator, resulting in a new cluster. After a cluster is created, the Coherence Operator will monitor the Coherence resource to reconcile the state of the cluster. You can add a new Coherence workload to a running application, or remove an existing workload, by modifying the ApplicationConfiguration resource, and adding or removing the Coherence component.

Scaling

Scaling a Coherence cluster is done by modifying the replicas field in the Component resource. Verrazzano will modify the Coherence resource replicas field and the cluster will be scaled accordingly. The following example configuration shows the replicas field that specifies the number of pods in the cluster.

apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
  name: orders
  namespace: sockshop
spec:
  workload:
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: VerrazzanoCoherenceWorkload
    spec:
      template:
        metadata:
          name: orders-coh
        spec:
          cluster: SockShop
          replicas: 3
          ...

NOTE: A Coherence cluster provisioned with Verrazzano does not support autoscaling with a Horizontal Pod Autoscaler.

Termination

You can terminate the Coherence cluster by removing the Component from the ApplicationConfiguration or by deleting the ApplicationConfiguration resource entirely.

NOTE

Do not delete the Coherence component if the application is still using it.

Logging

When a Coherence cluster is provisioned, Verrazzano configures it to send logs to OpenSearch. This is done by injecting a Fluentd sidecar configuration into the Coherence resource. The Coherence Operator will create the pod with the Fluentd sidecar. This sidecar periodically copies the Coherence logs from /logs to stdout, enabling the Fluentd DaemonSet in the verrazzano-system namespace to send the logs to OpenSearch. Note that the Fluend sidecar running in the Coherence pod never communicates with OpenSearch or any other network endpoint.

The logs are placed in a per-namespace OpenSearch data stream named verrazzano-application-<namespace>, for example: verrazzano-application-sockshop. All logs from Coherence pods in the same namespace will go into the same data stream, even for different applications. This is standard behavior and there is no way to disable or change it.

Each log record has some Coherence and application fields, along with the log message itself. For example:

 kubernetes.labels.coherenceCluster        SockShop
 kubernetes.labels.app_oam_dev/name        sockshop-appconf
 kubernetes.labels.app_oam_dev/component   orders
 ...

Metrics

Verrazzano uses Prometheus to scrape metrics from Coherence cluster pods. Like logging, metrics scraping is also enabled during provisioning, however, the Coherence resource YAML file must have proper metrics configuration. For details, see Coherence Metrics. In summary, there are two ways to configure the Coherence metrics endpoint. Coherence has a default metrics endpoint that you can enable. If your application serves metrics from its own endpoint, such as a Helidon application, then do not use the native Coherence metrics endpoint. To see the difference, examine the socks-shop and bobs-books examples.

Bobs Books

The bobs-books example uses the default Coherence metrics endpoint, so the configuration must enable this feature, shown in the following metrics section of the roberts-coherence component in the YAML file, bobs-books-comp.yaml.

...
          coherence:
            metrics:
              enabled: true

Sock Shop

The sock-shop example, which is a Helidon application with embedded Coherence, explicitly specifies the metrics port 7001 and doesn’t enable Coherence metrics. Coherence metrics still will be scraped, but not at the default endpoint.

      ports:
        ...
        - name: metrics
          port: 7001
          serviceMonitor:
            enabled: true

Because sock-shop components are not using the default Coherence metrics port, you must add a MetricsTrait section to the ApplicationConfiguration for each component, specifying the metrics port as follows:

    - trait:
        apiVersion: oam.verrazzano.io/v1alpha1
        kind: MetricsTrait
        metadata:
          name: carts-metrics
        spec:
          port: 7001

Prometheus configuration

Prometheus is configured using the Prometheus Operator to scrape application targets. During application deployment, Verrazzano creates or updates Service Monitors based on the MetricsTrait specified in the ApplicationConfiguration. When the application is deleted, Verrazzano removes the Service Monitors so that metrics are no longer collected for it.

Here is an example of the sock-shop Prometheus Service Monitor resource for catalog-coh in the application namespace.
Notice that services with certain labels are targeted. Prometheus Operator will find the Service Monitor and generate the scrape configuration to be used by Prometheus.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  ....
  name: catalog-coh-metrics
  namespace: sockshop
  ....
spec:
  endpoints:
  - bearerTokenSecret:
      key: ""
    port: metrics
    relabelings:
    - action: labeldrop
      regex: (endpoint|instance|job|service)
  namespaceSelector: {}
  selector:
    matchLabels:
      coherenceCluster: SockShop
      coherenceComponent: coherence-service
      coherenceDeployment: catalog-coh
      coherencePort: metrics
      coherenceRole: Catalog

Here are the labels on the corresponding catalog-coh-metrics service.

kind: Service
metadata:
  labels:
    coherenceCluster: SockShop
    coherenceComponent: coherence-service
    coherenceDeployment: catalog-coh
    coherencePort: metrics
    coherenceRole: Catalog
spec:
  ports:
  - name: metrics
    port: 9612
    protocol: TCP
    targetPort: 9612
  ....

Istio integration

Verrazzano ensures that Coherence clusters are not included in an Istio mesh, even if the namespace has the istio-injection: enabled label. This is done by adding the sidecar.istio.io/inject: "false" annotation to the Coherence resource, resulting in Coherence pods being created with that label. However, other application components in the mesh using mutual TLS authentication (mTLS) may need to communicate with Coherence. To handle this case, Verrazzano automatically creates an Istio DestinationRule to disable TLS for the Coherence port. This policy disables mTLS for port 9000, which happens to be used as a Coherence extend port for Bob’s Books.

  trafficPolicy:
    portLevelSettings:
    - port:
        number: 9000
      tls: {}
   ...

Currently, port 9000 is the only port where TLS is disabled, so you need to use this as the Coherence extend port if other components in the mesh access Coherence over the extend protocol.

Docs: Helidon Workload

Mon, 01 Jan 0001 00:00:00 +0000

Helidon is a collection of Java libraries for writing microservices. Helidon provides an open source, lightweight, fast, reactive, cloud native framework for developing Java microservices. It is available as two frameworks:

Helidon SE is a compact toolkit that embraces the latest Java SE features: reactive streams, asynchronous and functional programming, and fluent-style APIs.
Helidon MP implements and supports Eclipse MicroProfile, a baseline platform definition that leverages Java EE and Jakarta EE technologies for microservices and delivers application portability across multiple runtimes.

Helidon is designed and built with container-first philosophy.

Small footprint, low memory usage and faster startup times.
All 3rd party dependencies are stored separately to enable Docker layering.
Provides readiness, liveness and customizable health information for container schedulers like Kubernetes.

Containerized Helidon applications are generally deployed as Deployment in Kubernetes.

Verrazzano integration

Verrazzano supports application definition using Open Application Model (OAM). Verrrazzano applications are composed of components and application configurations.

Helidon applications are first class citizens in Verrazzano with specialized Helidon workload support, for example, VerrazzanoHelidonWorkload. VerrazzanoHelidonWorkload is supported as part of verrazzano-application-operator in the Verrazzano installation and no additional operator setup or installation is required. VerrazzanoHelidonWorkload also supports all the Traits and Scopes defined by Verrazzano along with core ones defined by the OAM specification.

VerrazzanoHelidonWorkload is modeled after ContainerizedWorkload, for example, it is used for long-running workloads in containers. However, VerrazzanoHelidonWorkload closely resembles and directly refers to Kubernetes Deployment schema. This enables an easy lift and shift of existing containerized Helidon applications.

The complete VerrazzanoHelidonWorkload API definition and description is available at VerrazzanoHelidonWorkload.

Verrazzano Helidon application development

With Verrazzano, you manage the life cycle of applications using Component and ApplicationConfiguration resources. A Verrazzano application can contain any number of VerrazzanoHelidonWorkload components, where each workload is a standalone containerized Helidon application, independent of any other in the application.

In the following example, everything under the spec: section is the custom resource YAML file for the containerized Helidon application, as defined by the VerrazzanoHelidonWorkload custom resource. Including this Component reference in your ApplicationConfiguration will result in a new containerized Helidon application being provisioned.

apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
  name: hello-helidon-component
  namespace: hello-helidon
spec:
  workload:
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: VerrazzanoHelidonWorkload
    metadata:
      name: hello-helidon-workload
      labels:
        app: hello-helidon
    spec:
      deploymentTemplate:
        metadata:
          name: hello-helidon-deployment
        podSpec:
          containers:
            - name: hello-helidon-container
              ...
              ...

The Steps to Deploy using OAM provides end-to-end instructions for developing and deploying the Verrazzano Helidon application.

For more Verrazzano Helidon application examples, see Examples.

Customizing Helidon Workload service

By default, deploying a Helidon Workload will create a service to access the Helidon pod. That service can be customized by added a serviceTemplate to the VerrazzanoHelidonWorkload spec. This supports customizing the service metadata and serviceSpec sections to do things, like add labels and annotations, customize the service ports, modify the pod selector, and such. It is not necessary to provide complete metadata and serviceSpec sections. Just add the fields you would like to customize and Verrazzano will create the rest of the fields, based the information provided in the deploymentTemplate. When customizing service ports, refer to Protocol Selection in the Istio documentation.

apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
  name: hello-helidon-component
  namespace: hello-helidon
spec:
  workload:
    apiVersion: oam.verrazzano.io/v1alpha1
    kind: VerrazzanoHelidonWorkload
    metadata:
      name: hello-helidon-workload
      labels:
        app: hello-helidon
    spec:
      deploymentTemplate:
        metadata:
          name: hello-helidon-deployment
        podSpec:
          containers:
            - name: hello-helidon-container
              ...
              ...
      serviceTemplate:
        metadata:
          name: hello-helidon-service
        serviceSpec:
          ports:
            - name: http-hello-helidon
              ...
              ...

Provisioning

When you apply the previous Component YAML file, Kubernetes will create a component.oam.verrazzano.io resource, but the containerized Helidon application will not be created until you create the ApplicationConfiguration resource, which references the VerrazzanoHelidonWorkload component. When the application is created, Verrazzano creates a Deployment and Service resource for each containerized Helidon application.

Typically, you would modify the Deployment and Service resource to make changes or to do lifecycle operations, like scale in and scale out. However, in the Verrazzano environment, the containerized Helidon application resource is owned by the verrazzano-application-operator and will be reconciled to match the component workload resource. Therefore, you need to manage the application configuration by modifying the VerrazzanoHelidonWorkload or ApplicationConfiguration resource, either by kubectl edit or applying a new YAML file. Verrazzano will notice the Component resource change and will update the Deployment and Service resource as needed.

You can add a new VerrazzanoHelidonWorkload to a running application, or remove an existing workload, by modifying the ApplicationConfiguration resource and adding or removing the VerrazzanoHelidonWorkload component.

Scaling

The recommended way to scale containerized Helidon application replicas is to specify a ManualScalerTrait with the VerrazzanoHelidonWorkload in the ApplicationConfiguration. The following example configuration shows the replicaCount field that specifies the number of replicas for the application.

...
    spec:
      components:
      - componentName: hello-helidon-component
        traits:
        - trait:                      
            apiVersion: core.oam.dev/v1alpha2
            kind: ManualScalerTrait
            spec:
              replicaCount: 2
...

Verrazzano will modify the Deployment resource replicas field and the containerized Helidon application replicas will be scaled accordingly.

NOTE

Make sure the replicas defined on the VerrazzanoHelidonWorkload component and that the replicaCount defined on the ManualScalerTrait for that component match, or else the DeploymentController in Kubernetes and OAM runtime in verrazzano-application-operator will compete to create a different number of Pods for same containerized Helidon application. To avoid confusion, we recommend that you specify replicaCount defined on the ManualScalerTrait and leave replicas undefined on VerrazzanoHelidonWorkload (as it is optional).

Logging

When a containerized Helidon application is provisioned on Verrazzano, Verrazzano will configure the default logging and send logs to OpenSearch. You can view the logs using the OpenSearch Dashboards.

The logs are placed in a per-namespace OpenSearch data stream named verrazzano-application-<namespace>, for example: verrazzano-application-hello-helidon. All logs from containerized Helidon application pods in the same namespace will go into the same data stream, even for different applications. This is standard behavior and there is no way to disable or change it.

Metrics

Verrazzano uses Prometheus to scrape metrics from containerized Helidon application pods. Like logging, metrics scraping is also enabled during provisioning. You can view metrics using the Grafana console.

Using the MetricsTrait custom resource, you can customize configuration information needed to enable metrics for an application component.

Ingress

Using the IngressTrait custom resource, you can configure traffic routing to a containerized Helidon application for an application component.

Troubleshooting

Whenever you have a problem with your Verrazzano Helidon application, there are some basic techniques you can use to troubleshoot. Troubleshooting shows you some simple things to try, as well as how to solve common problems you may encounter.