Cypress
Features described in this document are being deprecated as part of the transition to Test Workflows - Read More.
Our dedicated Cypress executor allows running Cypress tests with Testkube - directly from your Git repository.
- Default command for this executor:
./node_modules/cypress/bin/cypress
- Default arguments for this executor command:
run
--reporter
junit
--reporter-options
mochaFile=<reportFile>,toConsole=false
--project
<projectPath>
--env
<envVars>
Parameters in <> are calculated at test execution:
<reportFile>
- path to the generated JUnit report xml file<projectPath>
- path to where the project was placed<envVars>
- list of environment variable keys and values, separated by comma
See more at "Redefining the Prebuilt Executor Command and Arguments" on the Creating Test page.
- Cypress is a framework for making end-to-end tests of your applications, mostly used for front-end testing.
- When you run your Cypress tests, a browser is spun up so you can assert against it in an automatic way.
- With Cypress you can, for example, check that you get the expected behaviour when a button in your UI is clicked. Cypress can then record the test execution and provide you with a video of what happened with the actual test.
Check out our blog post to follow tutorial steps for end-to-end testing of your Kubernetes applications with Cypress.
Example Cypress Projectโ
If you haven't created the Cypress project yet, please follow the Cypress documentation for details: https://docs.cypress.io/guides/dashboard/projects.
In this example we will use the following Cypress project: https://github.com/kubeshop/testkube/tree/main/test/cypress/executor-tests/cypress-11
smoke.cy.js
contains three it
steps:
describe("Testkube website", () => {
it("Open Testkube website", () => {
cy.visit("/");
});
it(`Validate CYPRESS_CUSTOM_ENV ENV (${Cypress.env("CUSTOM_ENV")})`, () => {
expect("CYPRESS_CUSTOM_ENV_value").to.equal(Cypress.env("CUSTOM_ENV")); //CYPRESS_CUSTOM_ENV - "cypress" prefix - auto-loaded from global ENVs
});
it(`Validate NON_CYPRESS_ENV ENV (${Cypress.env("NON_CYPRESS_ENV")})`, () => {
expect("NON_CYPRESS_ENV_value").to.equal(Cypress.env("NON_CYPRESS_ENV")); //NON_CYPRESS_ENV - need to be loaded with --env parameter
});
});
The test opens the Testkube website and validates 2 methods of loading ENV variables.
Creating and Running Testsโ
Cypress projects consist of multiple files, so the Test can only be created using a Git Directory as the test source. The Git Directory checkouts the whole Cypress project directory from your repository.
- Dashboard
- CLI
- Custom Resource
If you prefer to use the Dashboard, just go to Tests and click the Add a new test
button. Then you need to fill in the Test Name, choose the Test Type (cypress/project
), and choose the Test Source (Git Directory
). Then, you need to fill in the repository details - Git repository URI (in this case https://github.com/kubeshop/testkube.git
), branch (main
), and path to the Cypress project directory in your repository (test/cypress/executor-tests/cypress-11
). In this example, the repository is public, but in the case of private ones, you would need to additionally fill in the Git credentials.
When the test is created, you can run it. But, in this example, the test checks ENV variables, which aren't set yet. In order for this test to pass:
CYPRESS_CUSTOM_ENV
ENV needs to be set toCYPRESS_CUSTOM_ENV_value
.NON_CYPRESS_ENV
ENV needs to be set toNON_CYPRESS_ENV_value
.
Setting ENV variablesโ
In order to set ENV variables, or arguments, you need to open the test you created, and then go to Settings
, and Variables & Secrets
.
Cypress auto-loads ENV variables with CYPRESS_
prefix, which are available in the tests without this prefix. So, if you set CYPRESS_CUSTOM_ENV
ENV, it will be available as CUSTOM_ENV
. Choose Add a new variable
, leave the default type (Basic
), and fill in the variable name CYPRESS_CUSTOM_ENV
, and its value CYPRESS_CUSTOM_ENV_value
. Then, click Save.
Another way of setting ENVs in Cypress is by the --env
argument. That's something you can also do at the Variables & Secrets
settings - just go to the Arguments
section.
Creating a Testโ
If you prefer using the CLI instead, you can create the test with testkube create test
.
You need to set test:
--name
(for example,cypress-test-2
)--type
(in this casecypress/project
)--test-content-type
(git-dir
, so the whole test directory will be checked out from the Git repository)--git-uri
- Repository URI (in case of this example,https://github.com/kubeshop/testkube.git
).--git-branch
--git-path
- Path to the Cypress project directory in the repository (in this casetest/cypress/executor-tests/cypress-11
). If it's in the root path in the repository, you can omit it.
Additionally, because the test from this example require 2 ENVs to be set, we need to set them.
--variable CYPRESS_CUSTOM_ENV=CYPRESS_CUSTOM_ENV_value
(global ENV variable that will be set in the execution container)--executor-args "--env NON_CYPRESS_ENV=NON_CYPRESS_ENV_value"
(--execution-args
pass arguments to the executor binary - in this case--env
)
So, the final command will be:
testkube create test --name cypress-test-2 --type cypress/project --test-content-type git-dir --git-uri https://github.com/kubeshop/testkube.git --git-branch main --git-path test/cypress/executor-tests/cypress-11 --variable CYPRESS_CUSTOM_ENV=CYPRESS_CUSTOM_ENV_value --executor-args "--env NON_CYPRESS_ENV=NON_CYPRESS_ENV_value"
You will get the confirmation the test has been created:
Test created testkube / cypress-test-2 ๐ฅ
Running the Testโ
The test can be started using testkube run test
command.
testkube run test cypress-test-2
Output:
Type: cypress/project
Name: cypress-test-2
Execution ID: 63f625d01e00af08138c4ea6
Execution name: cypress-test-2-1
Execution number: 1
Status: running
Start time: 2023-02-22 14:25:20.821507561 +0000 UTC
End time: 0001-01-01 00:00:00 +0000 UTC
Duration:
Variables: 1
- CYPRESS_CUSTOM_ENV = CYPRESS_CUSTOM_ENV_value
Test execution started
Watch test execution until complete:
$ kubectl testkube watch execution cypress-test-2-1
Use following command to get test execution details:
$ kubectl testkube get execution cypress-test-2-1
Getting Execution Resultsโ
You can then get test result with the testkube get execution
command using the execution name or execution ID.
testkube get execution cypress-test-2-1
ID: 63f625d01e00af08138c4ea6
Name: cypress-test-2-1
Number: 1
Test name: cypress-test-2
Type: cypress/project
Status: passed
Start time: 2023-02-22 14:25:20.821 +0000 UTC
End time: 2023-02-22 14:25:43.025 +0000 UTC
Duration: 00:00:22
Variables: 1
- CYPRESS_CUSTOM_ENV = CYPRESS_CUSTOM_ENV_value
Args: --env NON_CYPRESS_ENV=NON_CYPRESS_ENV_value
Repository parameters:
Branch: main
Commit:
Path: test/cypress/executor-tests/cypress-11
Working dir:
Certificate:
running test [63f625d01e00af08138c4ea6]
๐ Initializing...
๐ Reading environment variables...
โ
Environment variables read successfully
RUNNER_ENDPOINT="testkube-minio-service-testkube:9000"
RUNNER_ACCESSKEYID="********"
RUNNER_SECRETACCESSKEY="********"
RUNNER_REGION=""
RUNNER_TOKEN=""
RUNNER_BUCKET="testkube-artifacts"
RUNNER_SSL=false
RUNNER_SCRAPPERENABLED="true"
RUNNER_GITUSERNAME=""
RUNNER_GITTOKEN=""
RUNNER_DATADIR="/data"
๐ฆ Fetching test content from git-dir...
โ
Test content fetched to path /data/repo/test/cypress/executor-tests/cypress-11
๐ Fetching uploads from object store testkube-minio-service-testkube:9000...
๐ Placing files from buckets into /data/uploads/ []
๐ Getting the contents of bucket folders [test-cypress-test-2]
๐ Setting up access to files in /data
๐ฌ Executing in directory /data:
$ chmod
โ
Execution succeeded
โ
Access to files enabled
โ
Initialization successful
0xc0020faad0
๐ Preparing test runner
๐ Reading environment variables...
โ
Environment variables read successfully
RUNNER_ENDPOINT="testkube-minio-service-testkube:9000"
RUNNER_ACCESSKEYID="********"
RUNNER_SECRETACCESSKEY="********"
RUNNER_REGION=""
RUNNER_TOKEN=""
RUNNER_BUCKET="testkube-artifacts"
RUNNER_SSL=false
RUNNER_SCRAPPERENABLED="true"
RUNNER_GITUSERNAME=""
RUNNER_GITTOKEN=""
RUNNER_DATADIR="/data"
running test [63f625d01e00af08138c4ea6]
๐ Preparing for test run
๐ฆ Checking test content from git-dir...
โ
Test content checked
๐ฌ Executing in directory /data/repo/test/cypress/executor-tests/cypress-11:
$ npm
added 165 packages, and audited 166 packages in 4s
28 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
โ
Execution succeeded
๐ฌ Executing in directory /data/repo/test/cypress/executor-tests/cypress-11:
$ ./node_modules/cypress/bin/cypress
โ
Execution succeeded
CYPRESS_CUSTOM_ENV=CYPRESS_CUSTOM_ENV_value
๐ฌ Executing in directory /data/repo/test/cypress/executor-tests/cypress-11:
$ ./node_modules/cypress/bin/cypress run --reporter junit --reporter-options mochaFile=/data/repo/test/cypress/executor-tests/cypress-11/results/junit.xml,toConsole=false --env CYPRESS_CUSTOM_ENV=CYPRESS_CUSTOM_ENV_value --env NON_CYPRESS_ENV=NON_CYPRESS_ENV_value
... # Long output skipped
====================================================================================================
(Run Starting)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Cypress: 11.2.0 โ
โ Browser: Electron 106 (headless) โ
โ Node Version: v16.16.0 (/usr/local/bin/node) โ
โ Specs: 1 found (smoke.cy.js) โ
โ Searched: cypress/e2e/**/*.cy.{js,jsx,ts,tsx} โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Running: smoke.cy.js (1 of 1)
[62:0222/142530.135873:ERROR:zygote_host_impl_linux.cc(263)] Failed to adjust OOM score of renderer with pid 424: Permission denied (13)
(Results)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Tests: 3 โ
โ Passing: 3 โ
โ Failing: 0 โ
โ Pending: 0 โ
โ Skipped: 0 โ
โ Screenshots: 0 โ
โ Video: false โ
โ Duration: 7 seconds โ
โ Spec Ran: smoke.cy.js โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ โ smoke.cy.js 00:07 3 3 - - - โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ All specs passed! 00:07 3 3 - - -
Test execution completed with success in 22.204s ๐ฅ
The third option for creating the Test is to use a Test CRD. If you already have the test created you can check the definition in the Dashboard (Definition
tab in Test Settings).
You can also get a definition while using the testkube create test
command by adding --crd-only
.
In that case, the test won't be created, but the definition will be displayed.
testkube create test --name cypress-test-3 --type cypress/project --test-content-type git-dir --git-uri https://github.com/kubeshop/testkube.git --git-branch main --git-path test/cypress/executor-tests/cypress-11 --variable CYPRESS_CUSTOM_ENV=CYPRESS_CUSTOM_ENV_value --executor-args "--env NON_CYPRESS_ENV=NON_CYPRESS_ENV_value" --crd-only
Output:
apiVersion: tests.testkube.io/v3
kind: Test
metadata:
name: cypress-test-3
namespace: testkube
spec:
type: cypress/project
content:
type: git-dir
repository:
type: git
uri: https://github.com/kubeshop/testkube.git
branch: main
path: test/cypress/executor-tests/cypress-11
executionRequest:
variables:
CYPRESS_CUSTOM_ENV:
name: CYPRESS_CUSTOM_ENV
value: "CYPRESS_CUSTOM_ENV_value"
type: basic
args:
- "--env"
- "NON_CYPRESS_ENV=NON_CYPRESS_ENV_value"
When the Test CRD is saved to the yaml file, it can then be applied directly with kubectl apply -f SOME_FILE_NAME.yaml
.
Using Non-default Cypress Imagesโ
In the Cypress world, there are instances when you want to have control over your Runtime environment. Testkube can easily handle that for you! We're building several Cypress images to handle features that different versions of Cypress can support.
To use a different executor you can use one of our pre-built ones (for Cypress 8, 9, 10 and Custom Testkube images) or build your own Docker image based on a Cypress executor.
Let's assume we need official Cypress 10 version for our test runs. To handle that issue, create a new Cypress executor:
content of cypress-v10-executor.yaml
apiVersion: executor.testkube.io/v1
kind: Executor
metadata:
name: cypress-v10-executor
namespace: testkube
spec:
image: kubeshop/testkube-cypress-executor:1.1.7-cypress10 # <-- we're buidling cypress versions
types:
- cypress:v10/test # <-- just create different test type with naming convention "framework:version/type"
Tip: Look for recent executor versions here: https://hub.docker.com/r/kubeshop/testkube-cypress-executor/tags.
And add it to your cluster:
kubectl apply -f cypress-v10-executor.yaml
Now, create a new test with a type which our new executor can handle e.g.: cypress:v10/test
# create test
testkube create test --git-uri https://github.com/kubeshop/testkube-executor-cypress.git --git-path examples --type cypress:v10/test --name cypress-v10-example-test --git-branch main
# and run it
testkube run test cypress-v10-example-test -f