Webhooks
Webhooks allow you to integrate Testkube with external systems by sending HTTP POST payloads containing information about Testkube executions and their current state when specific events occur. To set up webhooks in Testkube, you'll need to have an HTTPS endpoint to receive the events and a payload template to be sent along with the data.
Please visit our Blog, Empowering Kubernetes Tests with Webhooks for a tutorial on setting up webhooks for Slack and Grafana Dashboard.
Benefits of using Webhooks in Testkube
Testkube uses webhooks to integrate with external systems, allowing you to effortlessly synchronize your testing workflows with other tools and platforms. These webhooks are designed to carry critical information regarding your tests as HTTP POST payloads. The information can include the execution and real-time status depending on how you configure it.
To leverage webhooks, you need to ensure that the platform that you want to send information to has an HTTPS endpoint to receive the events. Testkube also allows you to customize the payloads.
You can create a webhook from the dashboard, use the CLI, or create it as a custom resource. Before we show how it’s done, let’s understand a few scenarios where Webhooks in Testkube shine:
-
Incident Management & Response: Webhooks can be used to create incidents and alert on-call teams when a critical test fails. This ensures a timely response and avoids any potential disruption due to failures and bugs. With Testkube, you can configure incident management tools like PagerDuty and OpsGenie to receive alerts based on critical events for your tests.
-
Communication and Collaboration: You can configure Webhooks in Testkube to send alerts to your teams in your communication tool. This will notify your team of any critical event that needs attention and attend to it before the issue escalates. Some of the popular communications tools like Slack and MS Teams can be configured to receive alerts from Testkube.
-
Monitoring and Observability: Webhooks can also be used to send alerts and notifications to your monitoring and observability tools like Prometheus and Grafana. This provides visibility into your tests, alerts you, and ensures that timely corrective actions can be taken.
Creating a Webhook
The webhook can be created using the Dashboard, CLI, or a Custom Resource.
- Dashboard
- CLI
- Custom Resource
The Testkube Dashboard allows you to create Webhooks interactively from the Integrations / Webhooks tab - Read More:
Webhooks can be created with Testkube CLI using the create webhook
command.
testkube create webhook --name example-webhook --events start-test --events end-test-success --events end-test-failed --uri <YOUR_ENDPOINT_URL>
--name
- Your webhook name (in this case example-webhook
).
--events
- Event that will trigger a webhook. Multiple --events
can be defined (in this case --events start-test --events end-test-success --events end-test-failed
). All available trigger events can be found in the Supported Event types section.
--uri
- The HTTPS endpoint where you want to receive the webhook events.
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook
namespace: testkube
spec:
uri: <YOUR_ENDPOINT_URL>
events:
- start-test
- end-test-success
- end-test-failed
selector: ""
Where <YOUR_ENDPOINT_URL>
should be replaced with the HTTPS endpoint URL where you want to receive the webhook events.
And then apply with:
kubectl apply -f webhook.yaml
Resource Selector (labels)
In order to limit webhook triggers to a specific resource, or resources, the Resource Selector can be used. It allows you to select the specific resource by label, or labels.
- Dashboard
- CLI
- Custom Resource
In the Dashboard - Read More.
The Resource Selector can be set with --selector
.
For example, --selector test-type=postman-collection
will limit the resources to the postman tests (label: test-type=postman-collection
)
spec:
selector: test-type=postman-collection
So, the complete definition may look like this:
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook
namespace: testkube
spec:
uri: <YOUR_ENDPOINT_URL>
events:
- start-test
- end-test-success
- end-test-failed
selector: test-type=postman-collection
Webhook Payload
Webhook payload can be configured - in this example, event id
:
{"text": "event id {{ .Id }}"}
- Dashboard
- CLI
- Custom Resource
When you have selected an existing Webhook, its payload can be configured in Webhook Settings/Action tab - Read More
Create a webhook payload template file:
{
"text": "event id {{ .Id }}"
}
And set it with --payload-template template.json
.
testkube create webhook --name example-webhook --events start-test --events end-test-passed --events end-test-failed --payload-template template.json --uri <YOUR_ENDPOINT_URL>
Where <YOUR_ENDPOINT_URL>
should be replaced with the HTTPS endpoint URL where you want to receive the webhook events.
Webhook created example-webhook 🥇
Payload template can be configured with spec.payloadTemplate
.
payloadTemplate: |
{"text": "event id {{ .Id }}"}
Example:
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook
namespace: testkube
spec:
uri: <YOUR_ENDPOINT_URL>
events:
- start-test
- end-test-success
- end-test-failed
selector: ""
payloadObjectField: ""
payloadTemplate: |
{"text": "event id {{ .Id }}"}
Webhook Payload Variables
Webhook payload can contain event-specific variables - they will be replaced with actual data when the events occurs. In the above examples, only the event Id
is being sent.
However, any of these supported Event Variables can be used.
For example, the following payload:
{"text": "Event {{ .Type_ }} - Test '{{ .TestExecution.TestName }}' execution ({{ .TestExecution.Number }}) finished with '{{ .TestExecution.ExecutionResult.Status }}' status"}
will result in:
{"text": "Event end-test-success - Test 'postman-executor-smoke' execution (948) finished with 'passed' status"}
Helper methods
We also provide special helper methods to use in the webhook template:
executionstatustostring
is the method to convert a pointer to a execution status to a string type.
testsuiteexecutionstatustostring
is the method to convert a pointer to a test suite execution status to a string type.
testworkflowstatustostring
is the method to convert a pointer to a test workflow status to a string type.
Usage example:
- name: TEXT_COLOUR
value: {{ if eq (.TestWorkflowExecution.Result.Status | testworkflowstatustostring ) "passed" }}"00FF00"{{ else }}"FF0000"{{ end }}
Using API Server Environment Variables in Webhooks
In addition to event-specific variables, it's possible to use environment variables from the Testkube API server in your webhook payloads and configurations. This feature is particularly useful for including sensitive information like API keys or URIs without exposing them in your webhook definitions.
Configuring API Server Environment Variables
To set up extra environment variables for the API Server, you can use the testkube-api.extraEnvVars
field in your Helm chart configuration.
This allows you to define environment variables, including those that reference Kubernetes secrets:
testkube-api:
extraEnvVars:
- name: "SLACK_WEBHOOK_URI"
valueFrom:
secretKeyRef:
name: my-secret
key: slack-uri
Using Environment Variables in Webhook Payloads
Once configured, you can access these environment variables in your webhook payload templates using the .Envs
object:
{{ index .Envs "SLACK_WEBHOOK_URI" }}
For example:
TESTKUBE_PRO_URL: {{ index .Envs "TESTKUBE_PRO_URL" }}
Using Environment Variables in Webhook URIs
You can also use environment variables to set the webhook URI itself, allowing you to keep sensitive URIs out of your version control:
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: slack-notification
namespace: testkube
spec:
# ...
uri: '{{ "{{" }} index .Envs "SLACK_WEBHOOK_URI" {{ "}}" }}'
# ...
Make sure to double-escape brackets when specifying values that themselves specify template syntax.
URI and HTTP Headers
You can add additional HTTP headers like Authorization
or x-api-key
to have a secret token.
It's possible to use a Golang based template string as the header or URI value.
- Dashboard
- CLI
- Custom Resource
Webhook headers can be configured in Webhook Settings->Action.
Custom headers can be set using --header
- for example:
--header X-Token="12345"
spec:
headers:
X-Token: "12345"
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook
namespace: testkube
spec:
uri: <YOUR_ENDPOINT_URL>
events:
- start-test
- end-test-success
- end-test-failed
selector: ""
headers:
X-Token: "12345"
Supported Event types
Webhooks can be triggered on any of the following events:
- start-test
- end-test-success
- end-test-failed
- end-test-aborted
- end-test-timeout
- become-test-up
- become-test-down
- become-test-failed
- become-test-aborted
- become-test-timeout
- start-testsuite
- end-testsuite-success
- end-testsuite-failed
- end-testsuite-aborted
- end-testsuite-timeout
- become-testsuite-up
- become-testsuite-down
- become-testsuite-failed
- become-testsuite-aborted
- become-testsuite-timeout
- queue-testworkflow
- start-testworkflow
- end-testworkflow-success
- end-testworkflow-failed
- end-testworkflow-aborted
- become-testworkflow-up
- become-testworkflow-down
- become-testworkflow-failed
- become-testworkflow-aborted
- created
- updated
- deleted
They can be triggered by the following resources:
- test
- testsuite
- testworkflow
- executor
- trigger
- webhook
- testexecution
- testsuiteexecution
Supported Event Variables
Event-specific variables:
Id
- event ID (for example,2a20c7da-3b77-4ea9-a33d-403187d3e9e6
)Resource
ResourceId
Type_
- event Type (for example,start-test
,end-test,success
, etc. All available trigger events can be found in the Supported Event types section).TestExecution
- test execution details (example: TestExecution (Execution) section)TestSuiteExecution
- test suite execution details (example: TestSuiteExecution section)TestWorkflowExecution
- test suite execution details (example: TestWorkflowExecution section)ClusterName
- cluster nameEnvs
(API-server ENV variables) - list of Testkube API-Server ENV variables
The full Event Data Model can be found here.
TestExecution (Execution):
Test and Test Suite webhooks are being deprecated from Testkube. Read More.
Id
- Execution ID (for example,64f8cf3c712890925aea51ce
)TestName
- Test Name (for example,postman-executor-smoke
)TestSuiteName
- Test Suite name (if run as a part of a Test Suite)TestNamespace
- Execution namespace, where testkube is installed (for example,testkube
)TestType
- Test type (for example,postman/collection
)Name
- Execution name (for example, `postman-executor-smoke-937)Number
- Execution number (for example,937
)Envs
- List of ENV variables for specific Test (if defined)Command
- Command executed inside the Pod (for example,newman
)Args
- Command arguments (for example,run <runPath> -e <envFile> --reporters cli,json --reporter-json-export <reportFile>
)Variables
- List of variablesContent
- Test contentStartTime
- Test start time (for example,2023-09-06 19:23:34.543433547 +0000 UTC
)EndTime
- Time when the test execution finished (for example,2023-09-06 19:23:42.221493031 +0000 UTC
)Duration
- Test duration in seconds (for example,7.68s
)DurationMs
- Test duration in miliseconds (for example,7678
)ExecutionResult
- Execution result (https://github.com/kubeshop/testkube/blob/main/pkg/api/v1/testkube/model_event.go)Labels
Test labels (for example,[core-tests:executors executor:postman-executor test-type:postman-collection],
)RunningContext
- Running context - how the test has been triggered (for example,user-ui
)
The full Execution data model can be found here.
TestSuiteExecution:
Test and Test Suite webhooks are being deprecated from Testkube. Read More.
Id
- TestSuiteExecution ID (for example,64f8d5b2712890925aea51dc
)Name
- TestSuite execution name (for example,ts-executor-postman-smoke-tests-472
)Status
- TestSuite execution status (for example,running
orpassed
)TestSuite.Name
- TestSuite name (for example,executor-postman-smoke-tests
)TestSuite.Namespace
- TestSuite namespace (for example,testkube
)Envs
- List of ENV variablesVariables
- List of variablesStartTime
- Test suite start time (for example,2023-09-06 19:23:34.543433547 +0000 UTC
)EndTime
- Time when the test suite execution finished (for example,2023-09-06 19:23:42.221493031 +0000 UTC
)Duration
- Test suite execution duration in seconds (for example,7.68s
)DurationMs
- Test suite execution duration in milliseconds (for example,7678
)StepResults
Labels
- TestSuite labels (for example,[app:testkube testsuite:executor-postman-smoke-tests]
)RunningContext
- Running context - how the TestSuite has been triggered (for example,user-ui
)
The full TestSuiteExecution data model can be found here.
TestWorkflowExecution:
Id
- TestWorkflowExecution ID (for example,64f8d5b2712890925aea51dc
)Name
- TestWorkflow name (for example,wf-postman-smoke-tests
)Namepace
- TestWorkflowExecution namespace (for example,my-testkube
)Number
- TestWorkflowExecution sequence numberScheduledAt
- TestWorkflow scheduled time (for example,2023-09-06 19:23:34.543433547 +0000 UTC
)StatusAt
- Time when the execution result's status has changed last time (for example,2023-09-06 19:23:42.221493031 +0000 UTC
)Signature
- Structured tree of TestWorkflow stepsResult
- TestWorkflow execution resultOutput
- Additional information from the steps, like referenced executed tests or artifactsWorkflow
- TestWorkflow definitionResolvedWorkflow
- TestWorkflow definition with resolved fields
The full TestWorkflowExecution data model can be found here.
Additional Top-level Variables:
ExecutionCommand
- The CLI command to access the execution (example:kubectl testkube get execution 6679893e3b11f4e4900e17a5
).ExecutionURL
- The Dashboard URL to look at the execution (example:https://app.testkube.io/organization/tkcorg_9deb42dda2197657/environment/tkcenv_1145999f3c4d1115/dashboard/tests/git-zap-api-test/executions/6679893e3b11f4e4900e17a5
).ArtifactsCommand
- The CLI command to access the artifacts (example:kubectl testkube get artifacts 6679893e3b11f4e4900e17a5
).ArtifactsURL
- The Dashboard URL to look at the artifacts directly (example:https://app.testkube.io/organization/tkcorg_9deb42dda2197657/environment/tkcenv_1145999f3c4d1115/dashboard/tests/git-zap-api-test/executions/6679893e3b11f4e4900e17a5/artifacts
).LogsCommand
- The CLI command to access the logs (example:kubectl testkube get execution 6679893e3b11f4e4900e17a5 --logs-only
).LogsURL
- The Dashboard URL to look at the logs (example:https://app.testkube.io/organization/tkcorg_9deb42dda2197657/environment/tkcenv_1145999f3c4d1115/dashboard/tests/git-zap-api-test/executions/6679893e3b11f4e4900e17a5/log-output
).
Make sure that the value testkube-api.dashboardUri
in the helm-charts is set to app.testkube.io
, or your remote Dashboard URL, so that the variables above can be populated with the correct values.
Additional Examples
Microsoft Teams
Webhooks can also be used to send messages to Microsoft Teams channels.
First, you need to create an incoming webhook in Teams for a specific channel. You can see how to do it in the Teams Docs here. After your Teams incoming webhook is created, you can use it with Testkube webhooks - just use the URL provided (it will probably look like this: https://xxxxx.webhook.office.com/xxxxxxxxx
).
In order to send the message when test execution finishes, the following Webhook can be used:
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook-teams
namespace: testkube
spec:
events:
- end-test-success
- end-test-failed
- end-test-aborted
- end-test-timeout
uri: https://xxxxx.webhook.office.com/xxxxxxxxx
payloadTemplate: "{\"text\": \"Test '{{ .TestExecution.TestName }}' execution ({{ .TestExecution.Number }}) finished with '{{ .TestExecution.ExecutionResult.Status }}' status\"}\n"
It will result in:
{"text": "Test 'postman-executor-smoke' execution (949) finished with 'passed' status"}
and the message:
Test 'postman-executor-smoke' execution (949) finished with 'passed' status"
being displayed.
Testing Webhooks
If you are just getting started and want to test your webhook configuration, you can use public and free services that act as HTTP catch-all apps. Here are a couple of options:
- Beeceptor: Beeceptor allows you to quickly capture HTTP payloads, review them in real time, and send desired HTTP responses as you test your webhooks. You can use its local-tunnel feature when you want to test payloads against a service running on localhost.
- Webhook.Site: Webhook.Site provides an easy way to capture HTTP payloads, review the headers and body for the incoming requests, and automatically respond with a
200 OK
status code.
By using these services, you can quickly set up an HTTP endpoint to receive webhook payloads during your testing/development process.
URL Masking
In the “Organization Management” section, under the “Settings” tab, there is an option called “Webhooks URL Masking.” When this toggle is enabled, the Testkube Dashboard will no longer display any settings/configuration related to Webhook URLs.
Disabling Webhooks
Disabling webhooks can be helpful to make sure your team does not get spammed with notifications during development. Testkube lets you disable them via the CLI, the API or modifying the CRD directly. By default, webhooks are enabled on creation.
- CLI
- API
- Custom Resource
The easiest way to operate on webhooks in Testkube is to use the official CLI.
Set on Creation
To disable:
testkube create webhook --disable --name "${WEBHOOK_NAME}" ...
To enable it explicitly:
testkube create webhook --disable=false --name "${WEBHOOK_NAME}" ...
Set on Update
To disable:
testkube update webhook --disable --name "${WEBHOOK_NAME}"
To enable:
testkube update webhook --disable=false --name "${WEBHOOK_NAME}"
When automating your operations on webhooks, you can also use the API. For more details, please consult the "Core OSS OpenAPI Definion" page, respectively the "OpenAPI Specification" for Testkube Pro and Enterprise, under the Reference tab on the left. The field specifying this on the webhooks is called disabled
and is expecting a value of type boolean
.
Keep in mind, that enabling the webhook again from the API will require specifying "disabled":false
, simply deleting the field from the object is interpreted as not setting any value on it.
Using the webhook from the example in creation, you can add the field disabled
set to "true"
to the specification:
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook
namespace: testkube
spec:
uri: <YOUR_ENDPOINT_URL>
events:
- start-test
- end-test-success
- end-test-failed
selector: ""
disabled: "true"
Apply with:
kubectl apply -f webhook.yaml
To enable a webhook again, you can either set the field disabled
to "false"
explicitly, or delete it altogether from the specification.
Disabling Webhooks on Other Resources
Disabling webhooks can also be done directly on an execution of a test, a test suite and a test workflow. This way the notifications can be disabled temporarily during running the execution. The CLI offers the flag --disable-webhooks
, the API the parameter disableWebhooks
. When not set explicitly, webhooks will be enabled by default.
Disabling Webhooks on Tests
- CLI
- API
Use the flag --disable-webhooks
on the CLI:
testkube run test k6-test --disable-webhooks
The API expects a boolean value of the form:
{
"disableWebhooks": true
}
The examples show only the create
operations, however these instructions work the same for update
and run
as well.
Disabling Webhooks on Test Suites
- CLI
- API
Use the flag --disable-webhooks
on the CLI:
testkube run testsuite test-suite-1 --disable-webhooks
The API expects a boolean value of the form:
{
"disableWebhooks": true
}
The examples show only the create
operations, however these instructions work the same for update
and run
as well.
Disabling Webhooks on Test Workflows
- CLI
- API
Use the flag --disable-webhooks
on the CLI:
testkube run testworkflow test-workflow --disable-webhooks
The API expects the following object set under the spec
attribute:
{
"disableWebhooks": true
}
The examples show only the create
operations, however, these instructions work the same for update
and run
as well.
Enable Webhooks on State Changes
It is possible to get notified by webhooks only when the latest execution's outcome differs from the previous one. This way you can see instantly when one of your scheduled tests was healed or when they got broken. It could also help prevent alert fatigue by deduplicating the alerts and therefore decreasing the load on your monitoring systems. Testkube suppots special webhook event types to track changes between current and previous execution states. For Tests:
- become-test-up (from any error state to succeed one)
- become-test-down (from succeed state to any error one)
- become-test-failed (from any state to failed one)
- become-test-aborted (from any state to aborted one)
- become-test-timeout (from any state to timeout one)
For Test Suites:
- become-testsuite-up (from any error state to succeed one)
- become-testsuite-down (from succeed state to any error one)
- become-testsuite-failed (from any state to failed one)
- become-testsuite-aborted (from any state to aborted one)
- become-testsuite-timeout (from any state to timeout one)
For Test Workflows:
- become-testworkflow-up (from any error state to succeed one)
- become-testworkflow-down (from succeed state to any error one)
- become-testworkflow-failed (from any state to failed one)
- become-testworkflow-aborted (from any state to aborted one)
- CLI
- API
- Custom Resource
testkube update webhook --name example-webhook
{
"name": "example-webhook",
"namespace": "testkube",
"uri": "https://webhook.url/",
"events": ["start-test", "end-test-success", "end-test-failed"],
"disabled": false
}
apiVersion: executor.testkube.io/v1
kind: Webhook
metadata:
name: example-webhook-enabled
namespace: testkube
spec:
events:
- start-test
- end-test-success
- end-test-failed
uri: https://webhook.url/