Test Workflows - Test Suites
This Workflows functionality is not available when running the Testkube Agent in Standalone Mode - Read More
With Test Workflows it is possible to run downstream Test Workflows and Tests with execute
operation,
similarly to what you can do in Test Suites.
Advantages over original Test Suite
As it is regular Test Workflow, where a single step is dispatching downstream Test Workflows and Tests, the execution is very flexible. You can:
- Fetch input data before (i.e. by using
curl
/wget
to download data, or fetching Git repository) - Run setup operations (i.e. start shared instance of database, or generate API key)
- Process the results (i.e. by notifying about the status)
- Run other tests based on the previous results
- Trigger Workflows in other environments
- Nest workflows - workflow triggering workflow(s), triggering other workflow(s)
Test Workflows is our long-term solution, so keep in mind that the original Test Suites are being deprecated - Read More .
Syntax
You have to use execute
operation in the step, and provide definition of the Test Workflows to run - Schema Reference.
The Testkube Dashboard contains a visual builder for creating Composite Workflows - Read More
- YAML
- Log Output
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: example-test-suite
spec:
steps:
- execute:
workflows:
- name: example-distributed-k6
description: Run {{ index + 1 }} of {{ count }}
count: 2
config:
vus: 8
duration: 1s
workers: 2
- name: example-sharded-cypress
tests:
- name: example-test
description: Example without request
- name: example-test
description: Example with env variables
executionRequest:
variables:
SOME_VARIABLE:
type: basic
name: SOME_VARIABLE
value: some-value
Running Test Workflows
To run Test Workflow as part of the execute
step, you have to add its reference in the workflows
list.
You need to provide name
, along with optional config
values for parametrization.
Running Tests
Tests are being deprecated from Testkube - so only use this possibility if/while transitioning to Workflows
To run Tests as part of the execute
step, you have to add its reference in the tests
list.
You need to provide name
, along with optional executionRequest
values for parametrization,
that are similar to the regular Test execution request.
Controlling the concurrency level
You can use parallelism
property to control how many Test Workflows and Tests will be running at once.
In example, to run all the downstream jobs sequentially, you can use parallelism: 1
.
It affects jobs instantiated by matrix and sharding properties (like count
) too.
- YAML
- Log Output
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: example-sequential-test-suite
spec:
steps:
- execute:
parallelism: 1
workflows:
- name: example-distributed-k6
count: 2
config:
vus: 8
duration: 1s
workers: 2
- name: example-sharded-cypress
tests:
- name: example-test
count: 5
Nesting workflows
Workflows can be nested - workflow can execute workflow(s) executing other workflow(s). Parallelism can then be used to control the execution flow.
Example:
Tool-specific workflow (in this case k6 - executing multiple workflows for different cases)
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: k6-workflow-suite
spec:
steps:
- execute:
parallelism: 2
workflows:
- name: k6-workflow-smoke
- name: k6-workflow-smoke-template
- name: k6-workflow-smoke-template-without-checkout-step
- name: k6-workflow-smoke-artifacts
- name: distributed-k6-workflow-smoke
- name: distributed-k6-workflow-smoke-artifacts
That can be executed from another workflow
kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: tw-suite-full
spec:
steps:
- execute:
parallelism: 2
workflows:
- name: artillery-workflow-suite
- name: cypress-workflow-suite
- name: gradle-workflow-suite
- name: jmeter-workflow-suite
- name: k6-workflow-suite # workflow from the example
- name: maven-workflow-suite
- name: playwright-workflow-suite
- name: postman-workflow-suite
- name: soapui-workflow-suite
Passing input from files
It may happen that you will need to pass information from the file system. You can either pass the files using Test Workflow expressions (like file("./file-content.txt")
) or using a tarball
syntax.
Specific files
You can easily use Test Workflow expressions to fetch some files and send them as a configuration variable:
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: example-test-suite-with-file-input
spec:
content:
git:
uri: https://github.com/kubeshop/testkube
revision: main
paths:
- test/k6/executor-tests/k6-smoke-test-without-envs.js
steps:
- execute:
workflows:
- name: example-distributed-k6
config:
vus: 8
duration: 1s
workers: 2
script: '{{ file("/data/repo/test/k6/executor-tests/k6-smoke-test-without-envs.js") }}'
Multiple files transfer
To transfer multiple files, similarly to transfer
in Parallel Steps,
you can use a tarball
syntax that will pack selected files and return the URL to download them:
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: example-test-suite-with-file-input-packaged
spec:
content:
git:
uri: https://github.com/kubeshop/testkube
revision: main
paths:
- test/k6/executor-tests/k6-smoke-test-without-envs.js
steps:
- execute:
workflows:
- name: example-test-reading-files
tarball:
scripts:
from: /data/repo
config:
input: '{{ tarball.scripts.url }}'
You can later use i.e. content.tarball
to unpack them in destination test:
apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
name: example-test-reading-files
spec:
config:
input: {type: string}
content:
tarball:
- url: "{{ config.input }}" # extract provided tarball
path: "/data/repo" # to local /data/repo directory (or any other)
steps:
- shell: tree /data/repo
Matrix and sharding
The execute
operation supports matrix and sharding, to run multiple replicas and/or distribute the load across multiple runs.
It is supported by regular matrix/sharding properties (matrix
, shards
, count
and maxCount
) for each Test Workflow or Test reference.
You can read more about it in the general Matrix and Sharding documentation.