The NEW CircleCI Cypress Orb

As you embark on your journey writing automated tests, it’s likely that you’ll need to run them in a CI/CD environment. Configuring these systems can be cumbersome and unfamiliar to application developers. Luckily, Cypress is compatible with all Continuous Integration (CI) providers and systems! 🎉 But how can we help make this journey smoother and less complex?

Cypress partnered with CircleCI in 2018 to provide the Cypress Orb. Since then, Cypress has grown and matured, as has the tooling around it. Now we are proud to announce we've released version 3 of the CircleCI Cypress Orb. We revamped the Cypress Orb to be simpler to use so that you can get moving faster with less overhead. The Orb provides a set of predefined jobs and commands that allow developers to easily integrate Cypress into their CircleCI workflows.

We wanted to leverage the polished Developer Experience that our users know from their everyday use of Cypress and the Cypress Cloud. Most users should also see a noticeable speed improvement as well.

Our goal was to give two clear paths:

1. An out-of-the-box job that handles everything for you
2. Alternate custom commands to support more complex needs

In this post, we’ll unpack how to streamline your CI setup using our new Orb and how it can help you run your Cypress tests with ease.

More on Orbs

First, what is a CircleCI Orb?

Orbs are shareable packages of CircleCI configuration used to simplify your builds.

An Orb’s goal is to handle some of the complex configuration required when running applications in a virtual environment. This includes tasks like dependency installation, caching, and integration with third-party tools.

The Cypress Orb allows you to run your Cypress end-to-end and component tests without spending time configuring CircleCI.

Our goals with updating the Cypress Orb

The previous version of the Cypress Orb was packed with examples and configuration to help guide users through common patterns and solutions, some of which was dictated by CircleCI’s architecture. CircleCI has since made many advances to their own Orbs which can, in turn, be consumed by other Orbs. This means Cypress users get a simpler configuration that’s closer to what they would experience when running tests in the open source application.

This update addresses the following goals:

Eliminate the need for complex options and parameters

We audited all aspects of the previous Orb to simplify everything. We removed custom configuration and stop gap-solutions, which led to a myriad of options and parameters. Version 3 of the Cypress Orb relies more heavily on available CircleCI tooling. Now we can bridge the gap between using Cypress and the configuration needed to run in CI.

Offload dependency installation and caching

Common actions like installing and caching dependencies, integrating with browsers and internal tools for linting, testing, and publishing the orb itself are now using optimized CircleCI Orbs. This means we're able to lean more on CircleCI to handle CI optimizations and focus more on the Cypress DX.

Create a simple solution and API for the majority of users

We wanted one simple solution that covers the most ground possible, while also accounting for users with more complex needs such as more robust parallelization, pre/post installation steps, or even hyper-focused node or browser version needs.

How to use the Cypress Orb

Let’s take a look at an example using the previous Orb version.

Here we’re taking some common steps:

• Install node modules and the Cypress binary
• Split Cypress specs across machines (parallel)
• Use 4 CircleCI machines to distribute the run
• Record results with Cypress Cloud
• Group the test run for identification

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
  build:
    jobs:
      - cypress/install
      - cypress/run:
          requires:
            - cypress/install
    start: npm start
          parallel: true
          parallelism: 4
    record: true
          group: 4 machines

This gets the job done. But the majority of the parameters are flags you would pass to the Cypress command when using the CLI.

Now let’s see this same thing using the new Orb:

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
  build:
    jobs:
      - cypress/run:
          start-command: 'npm run start'
          cypress-command: 'npx cypress run --parallel --record --group 4 machines'
          parallelism: 4 # use 4 CircleCI machines to distribute specs

The difference is nominal but it addresses running Cypress like a user normally would. Now, you start your development server and add your Cypress command, passing the appropriate flags given your objective. It's more flexible and there’s no need to have a separate way to run tests in your CI config versus local development.

Also, notice above that cypress/install is removed in favor of the new cypress/run job which uses CircleCI's Node Orb under the hood to handle installation and caching for us. cypress/run is a complete job that functions as the out-of-the-box solution for most use cases. However, you can pass arguments to the job to override default behaviors. See the full list of arguments.

Using the new Cypress Component Testing? Just pass the flag. You'll skip starting the development server as this is bundled with your framework choice when configuring CT.

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
    jobs:
      - cypress/run:
          cypress-command: 'npx cypress run --component'

Let’s look at something more involved…

Here we’re taking additional steps for more granular control:

• Install node modules and the Cypress binary
• Use Yarn as the package manager
• Pull in an executor with node and browsers pre-installed
• Use the Chrome browser (via custom executor)
• Split Cypress specs across machines (parallel)
• Use 4 CircleCI machines to distribute run
• Record results with Cypress Cloud
• Group the test run for identification

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
  build:
    jobs:
      - cypress/install
      - cypress/run:
          requires:
            - cypress/install
    start: npm start
    yarn: true
    executor: cypress/browsers:node16.16.0-chrome107-ff107
    browser: chrome         
    parallel: true
          parallelism: 4
    record: true
    group: 4 machines  

What’s complicated here is the executor and browser elements. These responsibilities fall on Cypress to host a near infinite number of configured Docker images to provide value for an endless amount of use cases. And worse, what if your specific combination wasn’t available? More on creating your own Docker image later.

And now in the new Orb…

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
executor: cypress/default
workflows:
  build:
    jobs:
      - cypress/run
    		package-manager: yarn
    		install-browsers: true
    		start-command: yarn start
    		cypress-command: 'yarn cypress run --parallel --record --group 4 machines --browser chrome'
    		parallelism: 4 # use 4 CircleCI machines to distribute specs

The cypress run command uses the Electron browser by default to run your tests. The install-browsers parameter is used here to install Chrome or Firefox (at run time) to run your tests with. This is only needed if you are passing the --browser flag in your Cypress command.

The default executor here extends the CircleCI Browser Tools Orb and uses Node 16.16 by default. For additional OS and browser options you can pass in your own executor and configured Docker image. See our Executors documentation for information on available Cypress Docker images and how to compose your own.

You may want further flexibility beyond what the full cypress/run job offers. In these cases you can use separate cypress/install and cypress/run-tests commands. You can also pass arguments to these commands to override any default behaviors. See the full list of arguments.

A note about parallelization and consumption

There are 2 key metrics to understand when running a CI job across multiple machines:

1. Consumption time on CircleCI
2. Actual time it takes for tests to run

Consumption time is essentially the amount of CircleCI resources that a job requires to execute. For example, you may have a job that runs on 5 machines and takes 1 minute for all to complete. In this example it would only take 1 minute of actual time to execute all the jobs but would consume 5 minutes of CircleCI resources.

The Cypress Orb was designed to be as simple and fast as possible for the majority of use cases. If you are running your tests in parallel across more than 5 machines, you may not want to use the cypress/run job directly as it will consume more CircleCI resources than are necessary.

To lower your consumption time when running in parallel on more than 5 machines, see the example below. It shows using the cypress/default executor with cypress/install and cypress/run-tests commands separately. First dependencies are installed to a workspace only once and then run tests in parallel.

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
jobs:
  install-and-persist:
    executor: cypress/default
    steps:
      - cypress/install:
      - persist_to_workspace:
          root: ~/
          paths:
            - .cache/Cypress # this caches the Cypress binary for use across multiple jobs
            - project # this caches your source code (from the `-checkout` command) for use across multiple jobs
  run-tests-in-parallel:
    executor: cypress/default
    parallelism: 10
    steps:
      - run: echo "This step assumes dependencies were installed using the cypress/install job"
      - attach_workspace:
          at: ~/ # this references the items saved as part of `persist_to_workspace`
      - cypress/run-tests:
      		start-command: 'npm run start'
            cypress-command: 'npx cypress run --parallel --record'

Conclusion

Setting up CircleCI configuration for your Cypress tests is now easier than ever.

The Cypress Orb is compatible with CircleCI version 2.1 and higher, and can be easily set up using the CircleCI Cypress Orb Registry.

You can find additional information at the Cypress Documentation site and file any issues on the official Cypress CircleCI Orb Github repo.