Install with Helm
The main Testkube Helm Charts includes both the Testkube Control Plane and Testkube Agent charts and let you set up a customized Testkube instance tailored to your environment. See the list of included components.
You can learn more about Testkube deployment options in the Installation Overview.
Prerequisites
Before you proceed with the installation, please ensure that you have the following prerequisites in place:
- A Kubernetes cluster (version 1.21+)
- Helm (version 3+)
- (RECOMMENDED) cert-manager (version 1.11+) - Used for TLS certificate management.
- (RECOMMENDED) NGINX Controller (version v1.8+) - Used for Ingress configuration.
- (OPTIONAL) Prometheus Operator (version 0.49+) - used for metrics collection
- Own a public/private domain for creating Ingress rules.
- License Key and/or License File, if offline access is required.
Installing
- Add the Testkube Helm repository:
helm repo add testkubeenterprise https://kubeshop.github.io/testkube-cloud-charts
- Create and configure your values.yaml:
helm show values testkubeenterprise/testkube-enterprise > values.yaml
Alternatively, export one of our profiles:
testkube init <profile> --export > values.yaml
Currently available profiles are: demo
.
- Install the Testkube Helm Chart:
helm upgrade --install \
--create-namespace \
--namespace testkube \
-f values.yaml \
testkube testkubeenterprise/testkube-enterprise
General Settings
License
You will have to set a license key to get started with Testkube. You can also opt to use a shared secret for your license. You can request a trial license on the Testkube website, no up-front credit card required.
global:
enterpriseLicenseKey: XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX-V3
Domain
You should enable ingress and configure your domain to access Testkube services:
global:
ingress:
enabled: true
domain: "example.com"
By default, the following services will be exposed. You can also choose to override the subdomain for each service.
Service | Default | Override |
---|---|---|
Dashboard | dashboard.$domain | global.uiSubdomain |
REST API | api.$domain | global.restApiSubdomain |
Dex | api.$domain/idp | global.restApiSubdomain |
gRPC API | agent.$domain | global.grpcApiSubdomain |
WebSockets API | websockets.$domain | global.websocketsApiSubdomain |
Storage API | storage.$domain | global.storageApiSubdomain |
TLS can either be handled through cert-manager or a manually defined secret. While custom TLS certificates are possible, we strongly recommend using cert-manager
for simplified certificate manager.
TLS should be terminated at the application-level instead of the ingress-level as it will drastically enhance the performance of the gRPC and Websocket protocols. Be mindful that NGINX defaults to a downgrade from HTTP2 to HTTP1.1 when the backend service is using an insecure port.
Auth
You will have to configure how your users can access Testkube. Testkube uses Dex which supports the most popular identity providers. You can find a OIDC example for Google below:
dex:
envVars:
- name: GOOGLE_CLIENT_ID
valueFrom:
secretKeyRef:
name: <oidc-credentials-secret-name>
key: <client-id-key>
- name: GOOGLE_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: <oidc-credentials-secret-name>
key: <client-secret-key>
configTemplate:
additionalConfig: |
connectors:
- type: oidc
id: google
name: Google
config:
issuer: https://accounts.google.com
clientID: $GOOGLE_CLIENT_ID
clientSecret: $GOOGLE_CLIENT_SECRET
redirectURI: <dex endpoint>/idp/callback
Another example of using Google’s OpenID Connect provider that performs authorization based on groups membership:
testkube-cloud-api:
additionalEnv:
OAUTH_GROUPS_SCOPE: "true"
dex:
volumes:
- name: google-auth-volume
secret:
secretName: google-auth-secret #secret with the SA JSON key file
volumeMounts:
- name: google-auth-volume
readOnly: true
mountPath: /home
configTemplate:
additionalConfig: |
connectors:
- type: google
id: google
name: Google
config:
clientID: $GOOGLE_CLIENT_ID
clientSecret: $GOOGLE_CLIENT_SECRET
redirectURI: <dex endpoint>/idp/callback
serviceAccountFilePath: /home/googleAuth.json #path from volumeMount
groups:
- <group-name>
domainToAdminEmail:
"*": <email of a Google Workspace user>
Please note that it is required to create a k8s secret with a service account JSON key file and mount it to Dex pod. Follow official documentation to see full configuration.
Alternatively, you can use a local database with static users which acts as a virtual identity provider for evaluations.
Once authenticated, users will also need to be invited to org. By default, new users will automatically join the default organization. You can change this behaviour by changing the bootstrap and invitation configuration.
Prometheus Metrics
Testkube exposes Prometheus metrics on the /metrics
endpoint and uses a ServiceMonitor
resource to expose
them to Prometheus. In order for this to work, you need to have Prometheus Operator
installed in your cluster
so that the ServiceMonitor
resource can be created.
Use the following configuration to enable metrics:
testkube-cloud-api:
prometheus:
enabled: true
Shared Secrets
Testkube requires a variety of secrets to operate. Any required secret that is not provided manually will be automatically generated. You can optionally choose to specify your own secrets. For shared secrets without additional info, it’s recommended to use auto generation.
- testkube-license
- testkube-default-agent-token
- testkube-minio-credentials
Testkube License
Enter your license key which can be either an offline or online license. Replace «license» with your license.
For an online license, the format is XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX-V3
.
kubectl create secret generic testkube-license --from-literal=LICENSE_KEY=«license»
For an offline license, the format is a long string prefixed with key/
. You will also need a license file which starts with -----BEGIN LICENSE FILE-----
.
kubectl create secret generic testkube-license --from-file=LICENSE_KEY=«license key» --from-file=license.lic=license file»
This secret is referenced by the global.enterpriseLicenseRef
setting. For offline licences, you will also have to set global.enterpriseOfflineAccess: true
.
Advanced Settings
Check out the custom installation guide to learn more about our advanced settings, including organization management, custom ingress controllers, production environments, and more.