Recently, we hosted a webinar featuring Ely Lucas, Developer Experience Engineer, and Shawn Harris, Sr. Technical Solutions Engineer. They shared valuable insights and best practices for constructing a test automation strategy, configuring and setting up tests, and seamlessly integrating them into your CI/CD pipeline.
If you missed the live event, don't worry! You can watch the recorded webinar here:
We received an overwhelming number of questions during the webinar, and unfortunately, we couldn't address all of them within the allocated time. Below, you will discover the main points distilled from the content, along with our team's responses to the most prominent audience questions.
Key Takeaways
- To best take advantage of parallelizing your tests in CI/CD, it is important to split your CI/CD build workflows into two distinct jobs: an install job to download the dependencies and build the app, and a worker job to run the specs. See our guide on setting up GitHub Actions for examples of how to do so.
- Once your CI/CD jobs are set up, you can parallelize your spec run using Cypress Cloud to distribute the workload across multiple workers, optimizing test execution.
- To further optimize feedback cycles when specs fail in CI/CD, take advantage of Cypress Cloud's Spec Prioritization and Auto Cancellation.
Top questions from the audience
Is there a Cypress command for Jenkins similar to Github Actions to perform common tasks in CI/CD?
Currently, no. However, Cypress works great with Jenkins out of the box. Look at our CI provider examples on our documentation to see how to get Cypress running in Jenkins.
Hi, what is your suggestion for managing browser versions on CI?
We have prepared Docker images that you can use to specify a particular browser version. See our Docker guide for the list of all available images, and visit the CI provider documentation on how to use the image.
Hi, is Cypress Studio supported in version 12.10.0?
Yes! Cypress Studio was briefly removed when Cypress 10 first launched, but was quickly added back in 10.7. See our guide on how to get started with it.
Could you please show us how to set up Cypress Cloud?
Visit the getting started guide for Cloud for a walkthrough on setting up a project to record to Cypress Cloud.
We are integrating with TestRail, a third-party service, and our goal is to transmit a spec URL and its corresponding Cypress video recording to a specific TestRail CaseId. Is there an API provided by the Cypress dashboard that enables us to access both the video and relevant spec information?
You can tie into a set of lifecycle hooks, and you might find what you need in the after:spec hook: After Spec API | Cypress Documentation.
How can we run different suites from actions?
You can specify certain tests using the —spec flag to cypress run CLI command. Additionally, you can use the cypress-grep plugin to get better control of specific test runs.
Will you be adding WebKit to the Cypress factory?
As WebKit support is still experimental in Cypress, we have not added to the Docker factory or released any browser images for it. It is under consideration to do so when WebKit support moves to general availability.
What files exactly do we need to upload as an artifact after the completion of the install step?
In the install job, you want to make sure you cache the npm dependencies and any build artifacts so they can be used in later jobs. The Cypress GitHub Action will automatically cache the npm dependencies for you, and you can use the upload-artifact and download-artifact actions to store (and restore) the build folders. Specify the build/dist folders as the path parameter to the action. See the Cypress Heroes demo app for an example on how to do this.
What is the repo link for the sample code?
Here it is: Cypress Heroes Demo App
Does the time it takes for each parallelized step to run (shown in github actions) include the time it takes to install or uncache the dependencies for that step?
Yes, the time shown in GitHub is the total time, which includes the setup of the container, installing the dependencies, building the app, and running tests. The time shown in Cypress Cloud is only the test execution time.
I have my tests running on CI in a docker container. A test fails. I have Spec Prioritization and Auto Cancellation enabled. Will Cypress Cloud send a stop event to the docker container to stop the tests on failure?
Not exactly. Cypress Cloud will notify the Cypress app running on all the containers that the test run should be canceled, and any subsequent specs will then be skipped.
In this parallel example, is it possible to stop the E2E tests if the component tests fail?
Yes, when using auto cancellation, any subsequent specs that are a part of the same CI build/run will be canceled.
Do we need to have a paid plan to be able to use the features and to use Jenkins?
You can get started recording to the Cloud and using Parallelization on our free plan. Spec prioritization and auto cancellation require a business plan at this time.
During this demo we are watching now where these tests are being run? Cypress Cloud?
No, the tests are being run in the CI provider’s runners (GitHub actions in the case). Cypress Cloud does not run the specs themselves.
How do we schedule our production runs daily?
This would be dependent on the CI provider, but for GitHub actions you can set up a workflow to run at a specified interval using the schedule trigger.
I get xvfb: command not found, No such file or directory when using Jenkins.
Sounds like you might be missing some of the prereqs for Cypress on the Linux runner. View our guide on the software requirements, and you might be able to use our Docker images, which has all the prereqs installed.
Have more questions?
Join our team and the 9,000+ developers in our Discord community to pose questions and get help if you run into an issue. We’ll see you there. 😉 Happy Testing!