Skip to main content
New to Testkube? Unleash the power of cloud native testing in Kubernetes with Testkube. Get Started >

Test Workflows

Introduction

Test Workflows are an easy and extremely powerful way to define and run your tests in your Kubernetes clusters. Thanks to their flexibility, Test Workflows solve many of the problems that can occur with standard Tests, including:

  • Running Tests using different testing tool versions and dependencies.
  • Defining multiple steps for the execution of the tests.
  • Having more control over how your tests are executed, including resource consumption and setup/tearDown processes.
  • Being able to configure tool-specific commands and arguments.

Test Workflow Structure

Test Workflows are defined using a specific workflow language wrapped in a CRD. The high-level structure of a Test Workflow is as follows:

testworkflows-outline.yaml
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: ... # name of the Test Workflow
spec:
content: # content specifies where to find the actual test definition(s) to run
git: # checking out from git repository - see below
...
files: # defines files to create containing the actual tests - see below
...
container: # settings applied to all images used in the workflow, can be overridden
resources: # resource settings
requests: # resource requests
...
limits: # resource limits
...
workingDir: # default workingDir in the containers
env: # global env variables
...
steps: # steps that will be executed by this Test Workflow, can be nested
- name: ... # name of step
run: # action to perform for this step - see below for possible values
...

The different properties are described with examples and in more detail below.

tip

The Schema Reference for Test Workflows describes all available properties and constructs

Example - Test Workflow for Postman

Example Test Workflow for running Postman collection from Testkube repository: /test/postman/executor-tests/postman-executor-smoke-without-envs.postman_collection.json.

postman-example.yaml
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: postman-workflow-example # name of the Test Workflow
spec:
content:
git: # checking out from git repository
uri: https://github.com/kubeshop/testkube
revision: main
paths:
- test/postman/executor-tests/postman-executor-smoke-without-envs.postman_collection.json
container: # container settings
resources: # resource settings (optional)
requests: # resource requests
cpu: 256m
memory: 128Mi
workingDir: /data/repo/test/postman/executor-tests # default workingDir
steps: # steps that will be executed by this Test Workflow
- name: Run test
run:
image: postman/newman:6-alpine # image used while running specific step
args: # args passed to the container
- run
- postman-executor-smoke-without-envs.postman_collection.json

metadata

This is the standard Kubernetes Object metadata - Read More

name

metadata.name is mandatory - it's a name of the Test Workflow (for example postman-workflow-example)

labels

Labels can be set to allow easier filtering of Test Workflows.

metadata:
name: example-workflow
labels:
example: label
another: one
one: more

content

Specifies the actual testing scripts/files to use for your tests - Schema Reference

git

spec.content.git allows checking out from the Git repository:

spec:
content:
git:
uri: https://github.com/kubeshop/testkube # repository URI
revision: main # branch/revision
paths:
- test/cypress/executor-tests/cypress-13 # path to check out

path can be a directory, or a single file

files

spec.content.files allow creating specific files from strings.

spec:
content:
files:
- path: k6.js # created in working dir
content: |
import http from 'k6/http';
export const options = {
thresholds: {
http_req_failed: ['rate<0.01'],
}
};
export default function () {
http.get('https://testkube.io/');
}
- path: /example-file.js # created in root directory
content: another file contents
steps:
- name: Run test
shell: k6 run k6.js --iterations 100
container:
image: grafana/k6:latest

events

Specifies events that trigger this Workflow - Schema Reference

cronjob

spec.events.cronjob allows to run the workflow on specified schedule(s) :

spec:
events:
- cronjob:
cron: "*/20 * * * *"
labels:
key1: value1
annotations:
key2: value2
config:
myParameter: param1
- cronjob:
cron: "*/5 * * * *"
labels:
key3: value3
annotations:
key4: value4
config:
myParameter: param2
info

Testkube's schedule data format is the same that is used to define Kubernetes Cron jobs, visit Wikipedia Cron format for details.

tags

spec.execution.tags allows to add tags for Test workflow executions:

spec:
execution:
tags:
name1: value1
name2: value2

See an example of how to use tags here.

steps

Steps are the main building blocks in Test Workflows. They describe actions that should be executed in specific order - Schema Reference

spec:
...
steps:
- name: Example step 1
...
- name: Example step 2
...
- name: Example step 3
...

sub-steps

Steps can also be nested:

spec:
steps:
- name: Step 1
...
steps:
- name: Sub-step 1-1
...
- name: Sub-step 1-2
...
- name: Step 2
...
steps:
- name: Sub-step 2-1
...
- name: Sub-step 2-2
...

run

Specifies what to run within a step - Schema Reference

command

command allows to set the command, or override the default one:

spec:
steps:
- name: Run tests
run:
image: gradle:8.5.0-jdk11
command:
- gradle

args

Arguments can be passed to container using args:

  steps:
- name: Run tests
run:
image: cypress/included:13.6.4
args:
- --env
- NON_CYPRESS_ENV=NON_CYPRESS_ENV_value
- --config
- video=true

command + args

Command can be combined with args:

  - name: Run tests
run:
image: mcr.microsoft.com/playwright:v1.32.3-focal
command:
- "npx"
args:
- "--yes"
- "playwright@1.32.3"
- "test"

Args can be also set directly with command:

  - name: Run tests
run:
image: mcr.microsoft.com/playwright:v1.32.3-focal
command:
- "npx"
- "--yes"
- "playwright@1.32.3"
- "test"

ENVs

ENVs can be defined with env keyword on the step level:

steps:
- name: Run tests
run:
...
env:
- name: EXAMPLE_ENV_NAME
value: "example-env-value"

or "globally":

spec:
container:
...
env:
- name: EXAMPLE_ENV_NAME
value: "example-env-value"

shell

shell provides an ability to run a command, or multiple commands inside the shell.

steps:
- shell: command1 && command2

The same can be achieved with command and args, but shell may be more convenient.

steps:
- shell: mkdir ~/some-directory

is an equivalent of:

steps:
- run:
comand: [/bin/sh, -c]
args:
- mkdir
- ~/some-directory

container

container defines container-related settings - Schema Reference

image

container.image defines an image which will be used for executing steps.

steps:
- name: Run tests
shell: jmeter -n -t jmeter-executor-smoke.jmx -j /data/artifacts/jmeter.log -o /data/artifacts/report -l /data/artifacts/jtl-report.jtl -e
container:
image: anasoid/jmeter:5.6-plugins-21-jre

Please remember that the image should be stored in a Docker registry, which can be public or private. In the case of local images, you can use this [Docker one] (https://www.docker.com/blog/how-to-use-your-own-registry-2/) or something similar. Currently, we receive the image metadata only from the Docker registry, not the container storage or docker-daemon.

resources

Resources can be configured for a specific container - Schema Reference.

resource requests

spec:
...
container:
resources:
requests:
cpu: 2
memory: 2Gi

resource limits

spec:
...
container:
resources:
limits:
cpu: 4
memory: 4Gi

workingDir

By default, everything will be executed in the context of workingDir from specific container.

workingDir can be set globally:

spec:
...
container:
workingDir: /data/repo/test/cypress/executor-tests/cypress-13

Or, for a specific step:

steps:
- name: Saving artifacts
workingDir: /data/repo/test/cypress/executor-tests/cypress-13/cypress/videos

Or on the Step level:

artifacts

Workflows can collect any artifact produced by your executions - Read More

Files directly in workingDir

- name: Saving artifacts
workingDir: /data/artifacts
artifacts:
paths:
- '*'

Files in directories inside workingDir

- name: Saving artifacts
workingDir: /data/artifacts
artifacts:
paths:
- '**/*'

Artifacts can also be configured for project directories (inside /data/repo):

- name: Saving artifacts
workingDir: /data/repo/test/cypress/executor-tests/cypress-13/cypress/videos
artifacts:
paths:
- '**/*'

condition: always

It is common to save artifacts in case of failure. By default the artifacts have condition: always set if added directly on a step. They will be always scraped - even if the step fails:

- name: Example step with artifacts
shell: example-command
artifacts:
paths:
- '**/*'

If artifacts are saved in a separate step, or in a sub-step, they won't be scraped by default in the event of an earlier failure. In that case, setting condition: always specifically would be needed.

Separate step:

- name: Step 1
shell: example-command
- name: Step 2 - Saving artifacts
condition: always
artifacts:
paths:
- '**/*'

Sub-step:

- name: Step 1
shell: example-command
steps:
- name: Sub-step - Saving artifacts
condition: always
artifacts:
paths:
- '**/*'

Example - Cypress Project

Example Cypress project with artifacts (video recordings):

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: cypress-example-video-artifact
labels:
core-tests: workflows
spec:
content:
git:
uri: https://github.com/kubeshop/testkube
revision: main
paths:
- test/cypress/executor-tests/cypress-13
container:
resources:
requests:
cpu: 2
memory: 2Gi
workingDir: /data/repo/test/cypress/executor-tests/cypress-13
steps:
- name: Run tests
run:
image: cypress/included:13.6.4
args:
- --env
- NON_CYPRESS_ENV=NON_CYPRESS_ENV_value
- --config
- video=true
env:
- name: CYPRESS_CUSTOM_ENV
value: CYPRESS_CUSTOM_ENV_value
steps:
- name: Saving artifacts
workingDir: /data/repo/test/cypress/executor-tests/cypress-13/cypress/videos
artifacts:
paths:
- '**/*'

template - Executing from a Test Workflow Template

template allows executing from - Test Workflow Template:

  steps:
- name: Run from template
...
template:
name: example-template/cypress
config:
version: 13.5.0
params: "--env NON_CYPRESS_ENV=NON_CYPRESS_ENV_value --config '{\"screenshotsFolder\":\"/data/artifacts/screenshots\",\"videosFolder\":\"/data/artifacts/videos\"}'"

status - Getting status of latest test workflow execution

Use the Testkube CLI status command to get the status of a named Workflow:

kubectl describe testworkflow k6-workflow -n testkube
Name:         k6-workflow
Namespace: testkube
Labels: core-tests=workflows
Annotations: <none>
API Version: testworkflows.testkube.io/v1
Kind: TestWorkflow
Metadata:
Creation Timestamp: 2024-05-13T17:28:05Z
Generation: 1
Resource Version: 1077632
UID: ec1b58af-6226-4100-8b3c-d15f855f5a89
Spec:
Container:
Resources:
Requests:
Cpu: 128m
Memory: 128Mi
Working Dir: /data/repo/test/k6/executor-tests
Content:
Git:
Paths:
test/k6/executor-tests/k6-smoke-test.js
Revision: main
Uri: https://github.com/kubeshop/testkube
Steps:
Name: Run test
Run:
Args:
run
k6-smoke-test.js
-e
K6_ENV_FROM_PARAM=K6_ENV_FROM_PARAM_value
Env:
Name: K6_SYSTEM_ENV
Value: K6_SYSTEM_ENV_value
Image: grafana/k6:0.43.1
Status:
Latest Execution:
Id: 6659fd8455d94f735273fa95
Name: k6-workflow-15
Number: 15
Result:
Duration: 7.824s
Duration Ms: 7824
Finished At: 2024-05-31T16:40:44Z
Paused Ms: 0
Predicted Status: passed
Queued At: 2024-05-31T16:40:36Z
Started At: 2024-05-31T16:40:36Z
Status: passed
Total Duration: 7.824s
Total Duration Ms: 7824
Scheduled At: 2024-05-31T16:40:36Z
Status At: 2024-05-31T16:40:44Z
Workflow:
Labels:
Core - Tests: workflows
Name: k6-workflow
Namespace: testkube