Considerations for using a remote Kubernetes cluster
When you want to use a remote Kubernetes cluster as a location to run the supported tests or test resources, you must read the considerations first.
Before you want to run any of the supported tests or test resources on HCL DevOps Test
Hub (Test Hub) that are configured to
run on a remote Kubernetes cluster, you must have completed the following tasks:
- Verified that the Kubernetes cluster that you want to use is a supported one by referring to the OS, platform, and containers that are specified for Test Hub in System Requirements for DevOps Test Hub 2024.06 (11.0.2).
- Kept a suitable kubeconfig file with access to the remote Kubernetes cluster in a folder on the computer that you want to use to access an instance of Test Hub.
You must be familiar with the administration of a Kubernetes cluster. When you as the
cluster administrator, set up the remote Kubernetes cluster must consider the following
requirements:
- You must have a kubeconfig file with access to the remote
Kubernetes cluster in a folder on the computer that you want to use to access an
instance of Test Hub.Note: If the kubeconfig file references certificate files by using filepaths then the certificate data must be embedded in the kubeconfig file to make it usable by Test Hub. If the kubeconfig file contains multiple contexts, then only the current context is required.You can use the following kubectl command to create a suitable kubeconfig file called portable-config in the current folder. The portable-config file contains embedded certificate data and the information used by the current context:
kubectl config view --flatten --minify > portable-config
- You must ensure that the remote Kubernetes cluster is able to access the Test Hub image registry
which is the same registry that you used to install Test Hub. There is no need
to copy the Test Hub image
to the remote Kubernetes cluster but the remote cluster must be able to access
the image registry at test runtime.If the remote cluster that you want to use is a Lightweight Kubernetes (k3s) cluster, it might require a Secret to pull the Test Hub image. You must run the following command on the remote k3s cluster:
kubectl create secret docker-registry hclcr.io \ -n {remote-namespace} \ --docker-server=hclcr.io \ --docker-username={okta-email-address} \ --docker-password={harbor-cli-secret} \ --docker-email={example@abc.com}Notes:- You must replace {remote-namespace} with the value of the namespace that you have used for the remote cluster.
- You must replace {okta-email-address} with the username of the Harbor repository and replace {harbor-cli-secret} with the secret key that you copied from the Harbor repository.
- If the username contains any special characters, such as $, you must enclose it within single quotation-marks.
- You can replace {example@abc.com} with the email address of the administrator, if required.
- You need not configure separate licenses for running on the remote cluster because the licenses are consumed from the pool of available licenses that are configured for the team space on Test Hub. The licenses are returned to the pool when the test run is completed on the remote cluster.
- You must ensure that the Test Hub domain is accessible from the remote cluster so that the pod in the cluster is able to communicate with Test Hub.
- You must ensure that the appropriate PriorityClass exists on the remote cluster if you want to control the priority given to the remote execution pod by the scheduler. The same PriorityClass that is used by the Test Hub execution pod is used by the remote execution pod. If the named PriorityClass does not exist in the remote cluster, then no PriorityClass is specified by the remote execution pod.
- You must create an HTTP or an HTTPS Emissary listener when you want to run virtual services on the remote cluster. You must create both listeners when you want to run virtual services that use both HTTP and HTTPS transports. For information about creating listeners, refer to https://www.getambassador.io/docs/emissary/latest/topics/running/listener. The Emissary listener associates with the hosts created by Test Hub in the target namespace for executions that require Ingress. For example, when you want to run virtual services on the remote cluster and run tests from the remote cluster that use static agents.
- You can enter the code snippets for nodeSelectors and
tolerations in a yaml format, if you
want to use nodeSelectors and tolerations
on the remote cluster. You can enter these in the fields provided in the
Advanced configuration dialog when you register a
remote cluster with your team space on Test Hub.
For more information about nodeSelector, see nodeSelector and for tolerations, see Tolerations.
Restrictions: The following
restrictions apply when you want to run the supported tests on a remote Kubernetes
cluster:
- Using the UserLibs that are required for running certain API Suites are not supported on the remote cluster.
- Running of Compound Tests that contain performance tests, which require the NodePort access to connect to static agents, are not supported on the remote cluster.
When you plan to run any of the test assets or resources on a remote Kubernetes cluster
from a project on Test Hub, you
must complete the following tasks:
| Tasks | More information... | |
|---|---|---|
| 1 | Register a remote Kubernetes cluster with a team space on Test Hub. | Registering a remote Kubernetes cluster |
| 2 | View remote Kubernetes clusters added to a team space. | Viewing remote Kubernetes clusters |
| 3 | Add a remote Kubernetes cluster to a project in a team space. | Adding a remote Kubernetes cluster to a project |
| 4.1 | Configure a run for your test that you want to run on the remote
Kubernetes cluster by performing the following actions:
|
|
| 4.2 | Configure a run of the virtual service resources that you want to run
on the remote Kubernetes cluster by performing the following actions:
|