Verrazzano Enterprise Container Platform – Setup

Docs: Prerequisites

Mon, 01 Jan 0001 00:00:00 +0000

Verrazzano requires the following:

A Kubernetes cluster and a compatible kubectl.
dev profile - Each node in the cluster should contain at least two CPUs, 8 GB RAM, and 30 GB of disk storage. The entire cluster requires at least six CPUs, 48 GB RAM, and 100 GB of disk storage. In addition, about 52 GB of storage is required for the persistent volumes.
prod profile - Each node in the cluster should contain at least two CPUs, 24 GB RAM, and 30 GB of disk storage. The entire cluster requires at least eight CPUs, 64 GB RAM, and 150 GB of disk storage. In addition, about 450 GB of storage is required for the persistent volumes.

The persistent volumes are provisioned using the default StorageClass in the cluster. In case of a local disk based PV provisioner, each node in the cluster should have a minimum of 80 GB of disk storage for both of the profiles.

NOTE

To avoid conflicts with Verrazzano system components, we recommend installing Verrazzano into an empty cluster. Also, depending on the resource requirements of the applications you deploy, the configurations previously suggested may or may not be sufficient.

Supported hardware

Verrazzano requires x86-64; other architectures are not supported.

Supported software versions

Verrazzano supports the following software versions.

Kubernetes

You can install Verrazzano on the following Kubernetes versions.

Verrazzano	Release Date	Latest Patch Release	Latest Patch Release Date	End of Error Correction*	Kubernetes Versions
1.5	2023-02-15	1.5.3	2023-05-09	2024-02-28**	1.21, 1.22, 1.23, 1.24
1.4	2022-09-30	1.4.4	2023-03-15	2023-10-31	1.21, 1.22, 1.23, 1.24
1.3	2022-05-24	1.3.8	2022-11-17	2023-05-31	1.21, 1.22, 1.23
1.2	2022-03-14	1.2.2	2022-05-10	2022-11-30	1.19, 1.20, 1.21
1.1	2021-12-16	1.1.2	2022-03-09	2022-09-30	1.19, 1.20, 1.21
1.0	2021-08-02	1.0.4	2021-12-20	2022-06-30	1.18, 1.19, 1.20

*End of error correction for Verrazzano releases.
**Projected date. Actual date will be determined when the next minor or major release is available.

For more information, see Kubernetes Release Documentation.
For platform specific details, see Verrazzano platform setup.

WebLogic Server

The supported versions of WebLogic Server are dependent on the WebLogic Kubernetes Operator version. See the WebLogic Server versions supported here.

Coherence

The supported versions of Coherence are dependent on the Coherence Operator version. See the Coherence versions supported here.

Helidon

Verrazzano supports all versions of Helidon. For more information, see Helidon and Helidon Commercial Offerings.

Installed components

Verrazzano installs a curated set of open source components. The following table lists each component, its version, and a brief description.

Component	Version	Description
Alertmanager	0.24.0	Handles alerts sent by client applications, such as the Prometheus server.
Argo CD	2.5.3	A declarative, GitOps continuous delivery tool for Kubernetes.
cert-manager	1.9.1	Automates the management and issuance of TLS certificates.
Coherence Operator	3.2.9	Assists with deploying and managing Coherence clusters.
ExternalDNS	0.12.2	Synchronizes exposed Kubernetes Services and ingresses with DNS providers.
Fluentd	1.14.5	Collects logs and sends them to OpenSearch.
Grafana	7.5.17	Tool to help you examine, analyze, and monitor metrics.
Istio	1.15.3	Service mesh that layers transparently onto existing distributed applications.
Jaeger	1.42.0	Distributed tracing system for monitoring and troubleshooting distributed systems.
Jaeger Operator	1.42.0	Provides management for Jaeger tools.
Keycloak	20.0.1	Provides single sign-on with Identity and Access Management.
Kiali	1.57.1	Management console for the Istio service mesh.
kube-state-metrics	2.6.0	Provides metrics about the state of Kubernetes API objects.
MySQL	8.0.32	Open source relational database management system used by Keycloak.
MySQL Operator	8.0.32-2.0.8	Operator for managing MySQL InnoDB Cluster setups inside a Kubernetes cluster.
NGINX Ingress Controller	1.3.1	Traffic management solution for cloud‑native applications in Kubernetes.
Node Exporter	1.3.1	Prometheus exporter for hardware and OS metrics.
OAM Kubernetes Runtime	0.3.3	Plug-in for implementing the Open Application Model (OAM) control plane with Kubernetes.
OpenSearch	2.3.0	Provides a distributed, multitenant-capable full-text search engine.
OpenSearch Dashboards	2.3.0	Provides search and data visualization capabilities for data indexed in OpenSearch.
Prometheus	2.38.0	Provides event monitoring and alerting.
Prometheus Adapter	0.10.0	Provides metrics in support of pod autoscaling.
Prometheus Operator	0.59.1	Provides management for Prometheus monitoring tools.
Prometheus Pushgateway	1.4.2	Allows ephemeral and batch jobs to expose their metrics to Prometheus.
Rancher	2.7.2	Manages multiple Kubernetes clusters.
Rancher Backup Operator	2.1.3	Manages backup and restore of Rancher configurations and data.
Velero	1.9.1	Manages backup and restore of Kubernetes configurations and data.
WebLogic Kubernetes Operator	4.0.6	Assists with deploying and managing WebLogic domains.
WebLogic Monitoring Exporter	2.1.3	Exports Prometheus-compatible metrics from WebLogic instances.

Docs: CLI Setup

Mon, 01 Jan 0001 00:00:00 +0000

The Verrazzano command-line tool, vz, is available for Linux and Mac systems. Download the binary you want from the Releases page.

NOTE

For optimal functionality, install or upgrade the CLI version to match the desired Verrazzano version.

Install the vz CLI

These instructions demonstrate installing the CLI on Linux AMD64 machines.

Download the latest release

 $ curl -LO https://github.com/verrazzano/verrazzano/releases/download/master/verrazzano-1.5.2-linux-amd64.tar.gz

Validate the binary (optional)

Download the vz checksum file.

 $ curl -LO https://github.com/verrazzano/verrazzano/releases/download/master/verrazzano-1.5.2-linux-amd64.tar.gz.sha256

Validate the vz binary against the checksum file.

$ sha256sum -c verrazzano-1.5.2-linux-amd64.tar.gz.sha256

Unpack and copy the vz binary

 $ tar xvf verrazzano-1.5.2-linux-amd64.tar.gz

The following command needs to be run as root.

 $ sudo cp verrazzano-1.5.2/bin/vz /usr/local/bin

Test to ensure that the version you installed is up-to-date

 $ vz version

The resulting output should be similar to the following.

Version: master
BuildDate: 2023-02-12T21:07:26Z
GitCommit: cb0778bbf7a2cd90e1ae8458abd242f9da27a100

Use the vz CLI

Verrazzano provides a command-line tool for managing a Verrazzano environment using the Verrazzano and Kubernetes API.

Common use cases include installing, upgrading, and uninstalling Verrazzano, as well as analyzing failures in a running Verrazzano environment.

Usage information

Use the following syntax to run vz commands from your terminal window.

vz [command] [flags]

Available commands

Command	Definition
`analyze`	Analyze cluster
`bug-report`	Collect information from the cluster to report an issue
`completion`	Generate the autocompletion script for the specified shell
`help`	Help about any command
`install`	Install Verrazzano
`status`	Status of the Verrazzano installation and access endpoints
`uninstall`	Uninstall Verrazzano
`upgrade`	Upgrade Verrazzano
`version`	Verrazzano version information

Available Flags

These flags apply to all the commands.

Flag	Definition
`--context string`	The name of the kubeconfig file context to use.
`-h`, `--help`	Help for `vz`.
`--kubeconfig string`	Path to the kubeconfig file to use.

Docs: Platform Setup

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Install

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Use a Private Registry

Mon, 01 Jan 0001 00:00:00 +0000

Installing Verrazzano using a private Docker-compliant container registry requires the following:

Loading all the required Verrazzano container images into your own registry and repository.
Installing the Verrazzano platform operator with the private registry and repository used to load the images.

You must have the following software installed:

Docker
kubectl
Helm (version 3.x+)
jq

Verrazzano distributions are available in two variations:

The Verrazzano Lite Distribution contains Kubernetes manifests to deploy Verrazzano, client binaries, and various other utilities. This distribution is provided for Linux and MacOS operating systems on AMD and ARM architectures.
The Verrazzano Full Distribution contains Kubernetes manifests to deploy Verrazzano, client binaries and various utilities for Linux and MacOS operating systems on AMD and ARM architectures. This full distribution also includes the Linux AMD container images required for Verrazzano.

Both the distributions include:

Verrazzano CLI
Installation Profiles
Helper scripts to download the images from the bill of materials (BOM) and to upload the Verrazzano images to a private registry
Helm charts for the Verrazzano platform operator
README.md which provides the layout of the respective distribution

To set up a private registry, download the desired Verrazzano distribution, then Load the images:

Lite Distribution

Download the Verrazzano Lite Distribution from the GitHub releases page.

a. In your browser, go to Verrazzano releases.

b. Download the distribution TAR file, verrazzano-<major>.<minor>.<patch>-<operating system>-<architecture>.tar.gz, and the corresponding checksum file.

c. In the downloaded directory, validate that the checksum and TAR files match. For example, if you have downloaded verrazzano-1.5.2-linux-amd64.tar.gz:
```
$ sha256sum -c verrazzano-1.5.2-linux-amd64.tar.gz.sha256
# Sample output
verrazzano-1.5.2-linux-amd64.tar.gz: OK
```
NOTE: Use the sha256sum command on Linux and shasum on MacOS.

d. Expand the TAR file to access the release artifacts.

The following example, extracts the distribution archive verrazzano-1.5.2-linux-amd64.tar.gz into the current directory.
```
$ tar xvf verrazzano-1.5.2-linux-amd64.tar.gz
```
After a successful extraction, the release artifacts will be under the verrazzano-1.5.2 directory.

e. Define an environment variable DISTRIBUTION_DIR.
```
 $ DISTRIBUTION_DIR=<path to the current directory>/verrazzano-1.5.2
```
Download the Verrazzano images defined in the BOM, ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json, using the script, ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh.
```
$ sh ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh -b ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json -f ${DISTRIBUTION_DIR}/images  
```
The previous command downloads all the images to the ${DISTRIBUTION_DIR}/images directory.

Full Distribution

Download the Verrazzano ZIP file.
- Download the Verrazzano ZIP file from the Oracle Software Delivery Cloud for major or minor releases.
  
  a. In your browser, go to the Oracle Software Delivery Cloud and log in with your credentials.
  
  b. In the drop-down menu preceding the search bar, select All Categories.
  
  c. In the search bar, enter Verrazzano Enterprise Container Platform and click Search.
  
  d. Select the REL: Verrazzano Enterprise Container Platform 1.5.0.0.0 link. This will add it to your download queue.
  
  e. At the top of the page, select the Continue link.
  
  f. Review the Download Queue, then click Continue.
  
  g. Accept the license agreement and click Continue.
  
  h. Download the file:
  - To download the ZIP file directly, select the file link in the list.
  - To download the ZIP file using Oracle Download Manager, click Download and run the Oracle Download Manager executable.
- Download the Verrazzano ZIP file from My Oracle Support for cumulative patches.
  
  a. In your browser, go to My Oracle Support and log in with your credentials.
  
  b. Select the Patches & Updates tab.
  
  c. In the Patch Search panel, select the link Product or Family (Advanced).
  
  d. In the search bar for Product is, enter Oracle Verrazzano Enterprise Container Platform.
  
  e. The previous step populates the available releases for Verrazzano in the drop-down menu Release is. Select the desired release(s) and click Search.
  
  f. A new panel with Patch Advanced Search Results will open listing all the patches for the release. Select the link for the desired patch, under the Patch Name.
  
  g. From the page providing details about the patch, click Download.
  
  h. Download the ZIP file by selecting the file link.
Prepare to do the private registry installation.

a. Extract the ZIP archive to a desired directory location. There will be two files: a compressed TAR file containing the product files and a checksum file.

b. Define an environment variable DISTRIBUTION_DIR.
```
$ DISTRIBUTION_DIR=<path to the current directory>/verrazzano-1.5.2
```
c. In the expanded archive directory, validate that the checksum and TAR files match. For example,
```
$ sha256sum -c verrazzano-1.5.2.tar.gz.sha256
# Sample output
verrazzano-1.5.2.tar.gz: OK
```
NOTE: Use the sha256sum command on Linux and shasum on MacOS.

Load the images

Load the product images into your private registry.

To log in to the Docker registry, run docker login <SERVER> with your credentials.
For use with the examples in this document, define the following variables with respect to your target registry and repository: MYREG, MYREPO, VPO_IMAGE.

These identify the target Docker registry and repository, and the Verrazzano platform operator image, as defined in the BOM file. For example, using a target registry of myreg.io and a target repository of myrepo/v8o:
```
 $ MYREG=myreg.io
 $ MYREPO=myrepo/v8o
 $ VPO_IMAGE=$(cat ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json | jq -r '.components[].subcomponents[] | select(.name == "verrazzano-platform-operator") | "\(.repository)/\(.images[].image):\(.images[].tag)"')
```
Run the ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh script to push the images to the registry:
```
$ sh ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh -t $MYREG -r $MYREPO -l ${DISTRIBUTION_DIR}/images
```
Although most images can be protected using credentials stored in an image pull secret, some images must be public. Use the following commands to get the list of public images:
- The Rancher Agent image.
```
$ cat ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json | jq -r '.components[].subcomponents[] | select(.image == "rancher-agent") | "\(.image):\(.tag)"'
```
- All the Rancher images in the rancher/additional-rancher subcomponent.
```
$ cat ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json | jq -r '.components[].subcomponents[] | select(.name == "additional-rancher") | .images[] | "\(.image):\(.tag)"'
```
- For all the Verrazzano Docker images in the private registry that are not explicitly marked public, you will need to create the secret verrazzano-container-registry in the verrazzano-install namespace, with the appropriate credentials for the registry, identified by $MYREG. For example:
```
$ kubectl create namespace verrazzano-install
$ kubectl create secret docker-registry verrazzano-container-registry -n verrazzano-install \
  --docker-server=$MYREG --docker-username=myreguser \
  --docker-password=xxxxxxxx --docker-email=me@example.com
```

Install Verrazzano

Install the Verrazzano platform operator using the image defined by $MYREG/$MYREPO/$VPO_IMAGE.
```
$ helm template --include-crds ${DISTRIBUTION_DIR}/manifests/charts/verrazzano-platform-operator \
    --set image=${MYREG}/${MYREPO}/${VPO_IMAGE} --set global.registry=${MYREG} \
    --set global.repository=${MYREPO} | kubectl apply -f -
```
Wait for the deployment of the Verrazzano platform operator.
```
$ kubectl -n verrazzano-install rollout status deployment/verrazzano-platform-operator

# Sample output
  deployment "verrazzano-platform-operator" successfully rolled out
```
Confirm that the Verrazzano platform operator pod is running.
```
$ kubectl -n verrazzano-install get pods

# Sample output
  NAME                                            READY   STATUS    RESTARTS   AGE
  verrazzano-platform-operator-74f4547555-s76r2   1/1     Running   0          114s
```

The distribution archive includes the supported installation profiles under ${DISTRIBUTION_DIR}/manifests/profiles. Verrazzano supports customizing installation configurations. See Customize Verrazzano.

To create a Verrazzano installation using the provided profiles, run the following command:

$ kubectl apply -f $DISTRIBUTION_DIR/manifests/profiles/prod.yaml

For a complete description of Verrazzano configuration options, see the Reference API.

Configuring access to an insecure private registry

A private Docker registry is called an insecure registry when it is configured for access using a self-signed certificate or over an unencrypted HTTP connection. Depending on the platform, there could be some additional configuration required for installing Verrazzano with an insecure registry.

For example, for the Oracle Cloud Native Environment platform, insecure registries must be configured in /etc/containers/registries.conf as follows on the worker nodes:

 [registries]
    [registries.insecure]
      registries = ["insecure-registry-1:1001/registry1","insecure-registry-2:1001/registry2"]

WebLogic applications

WebLogic applications require that the container registry secret be specified in the Domain resource. Create a registry secret in the application namespace and specify the secret in the imagePullSecrets field of the WebLogic Domain spec for the application.

For an example, see the ToDo List example application component YAML file.

Docs: Use a Private Registry

Mon, 01 Jan 0001 00:00:00 +0000

Installing Verrazzano using a private Docker-compliant container registry requires the following:

Loading all required Verrazzano container images into your own registry and repository.
Installing the Verrazzano platform operator with the private registry and repository used to load the images.

You must have the following software installed:

Docker
kubectl
Helm (version 3.x+)
jq

The Verrazzano distribution contains Kubernetes manifests to deploy Verrazzano, client binaries, and various other utilities. These distributions are provided for Linux and MacOS operating systems on AMD and ARM architectures.

This distribution includes:

Verrazzano CLI
Installation Profiles
Helper scripts to download the images from the bill of materials (BOM) and to upload the Verrazzano images to a private registry
Helm charts for the Verrazzano platform operator
README.md which provides the layout of the distribution

Load the images

Download the desired Verrazzano distribution from the GitHub releases page.

a. In your browser, go to Verrazzano releases.

b. Download the distribution TAR file, verrazzano-<major>.<minor>.<patch>-<operating system>-<architecture>.tar.gz, and the corresponding checksum file.

c. In the downloaded directory, validate that the checksum and TAR files match. For example, if you have downloaded verrazzano-1.5.2-linux-amd64.tar.gz:
```
$ sha256sum -c verrazzano-1.5.2-linux-amd64.tar.gz.sha256
# Sample output
verrazzano-1.5.2-linux-amd64.tar.gz: OK
```
NOTE: Use the sha256sum command on Linux and shasum on MacOS.

d. Expand the TAR file to access the release artifacts.

The following example, extracts the distribution archive verrazzano-1.5.2-linux-amd64.tar.gz into the current directory.
```
$ tar xvf verrazzano-1.5.2-linux-amd64.tar.gz
```
After a successful extraction, the release artifacts will be under the verrazzano-1.5.2 directory.

e. Define an environment variable DISTRIBUTION_DIR.
```
$ DISTRIBUTION_DIR=<path to the current directory>/verrazzano-1.5.2
```
Download the Verrazzano images defined in the BOM, ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json, using the script, ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh.
```
$ sh ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh -b ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json -f ${DISTRIBUTION_DIR}/images  
```
The previous command downloads all the images to the ${DISTRIBUTION_DIR}/images directory.
Load the product images into your private registry.

a. To log in to the Docker registry, run docker login <SERVER> with your credentials.

b. For use with the examples in this document, define the following variables with respect to your target registry and repository: MYREG, MYREPO, VPO_IMAGE.

These identify the target Docker registry and repository, and the Verrazzano platform operator image, as defined in the BOM file. For example, using a target registry of myreg.io and a target repository of myrepo/v8o:
```
$ MYREG=myreg.io
$ MYREPO=myrepo/v8o
$ VPO_IMAGE=$(cat ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json | jq -r '.components[].subcomponents[] | select(.name == "verrazzano-platform-operator") | "\(.repository)/\(.images[].image):\(.images[].tag)"')
```
c. Run the ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh script to push the images to the registry:
```
 $ sh ${DISTRIBUTION_DIR}/bin/vz-registry-image-helper.sh -t $MYREG -r $MYREPO -l ${DISTRIBUTION_DIR}/images
```
d. Although most images can be protected using credentials stored in an image pull secret, some images must be public. Use the following commands to get the list of public images:
- The Rancher Agent image.
```
$ cat ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json | jq -r '.components[].subcomponents[] | select(.image == "rancher-agent") | "\(.image):\(.tag)"'
```
- All the Rancher images in the rancher/additional-rancher subcomponent.
```
$ cat ${DISTRIBUTION_DIR}/manifests/verrazzano-bom.json | jq -r '.components[].subcomponents[] | select(.name == "additional-rancher") | .images[] | "\(.image):\(.tag)"'
```
- For all the Verrazzano Docker images in the private registry that are not explicitly marked public, you will need to create the secret verrazzano-container-registry in the verrazzano-install namespace, with the appropriate credentials for the registry, identified by $MYREG. For example:
```
$ kubectl create namespace verrazzano-install
$ kubectl create secret docker-registry verrazzano-container-registry -n verrazzano-install \
  --docker-server=$MYREG --docker-username=myreguser \
  --docker-password=xxxxxxxx --docker-email=me@example.com
```

Install Verrazzano

Install the Verrazzano platform operator using the image defined by $MYREG/$MYREPO/$VPO_IMAGE.
```
$ helm template --include-crds ${DISTRIBUTION_DIR}/manifests/charts/verrazzano-platform-operator \
  --set image=${MYREG}/${MYREPO}/${VPO_IMAGE} --set global.registry=${MYREG} \
  --set global.repository=${MYREPO} | kubectl apply -f -
```
Wait for the deployment of the Verrazzano platform operator.
```
$ kubectl -n verrazzano-install rollout status deployment/verrazzano-platform-operator

# Sample output
  deployment "verrazzano-platform-operator" successfully rolled out
```
Confirm that the Verrazzano platform operator pod is running.
```
$ kubectl -n verrazzano-install get pods

# Sample output
  NAME                                            READY   STATUS    RESTARTS   AGE
  verrazzano-platform-operator-74f4547555-s76r2   1/1     Running   0          114s
```

To create a Verrazzano installation using the provided profiles, run the following command:

$ kubectl apply -f $DISTRIBUTION_DIR/manifests/profiles/prod.yaml

For a complete description of Verrazzano configuration options, see the Reference API.

Configuring access to an insecure private registry

For example, for the Oracle Cloud Native Environment platform, insecure registries must be configured in /etc/containers/registries.conf as follows on the worker nodes:

 [registries]
    [registries.insecure]
      registries = ["insecure-registry-1:1001/registry1","insecure-registry-2:1001/registry2"]

WebLogic applications

For an example, see the ToDo List example application component YAML file.