cy.blog()

Sharing our challenges, mistakes, hacks, successes, opinions and news

Articles written by Gleb Bahmutov

Visual Testing for Emails Sent Using 3rd Party Dynamic Templates

This blog post will teach you how to visually verify the emails sent by a 3rd party service. Such verification is necessary to ensure the emails have the expected style, and include the necessary information. The post teaches you how to deal with the difficult email markup generated from dynamic templates when doing image comparison. Note: you can find the source code for this blog post in the repository cypress-sendgrid-visual-testing-example [https://github.com/bahmutov/cypress-sendgrid-visu

Read more

Full Testing of HTML Emails using SendGrid and Ethereal Accounts

Imagine you are sending beautiful HTML emails using your own SMTP server or even a 3rd party service like SendGrid. What if the emails are formatted incorrectly? What if the template string is incorrect and the user sees Dear <<undefined>>? Can you test the emails to make sure they really work? In this blog post I will describe the full cycle email testing. We will send actual emails using 3rd party service SendGrid [https://sendgrid.com/] and we will fetch that email from a temporary email acc

Read more

Testing HTML Emails using Cypress

In this blog post, we will use a local SMTP server to receive emails sent by the app to the user. We will test the HTML emails to make sure they look and work correctly. Note: you can find the source code shown in this blog post at cypress-email-example [https://github.com/bahmutov/cypress-email-example] [https://github.com/bahmutov/cypress-email-example] and watch a video explaining the testing process here [https://youtu.be/16WTH7XeIVw]. Sending emails If you want to send an email from a No

Read more

Testing deployed GitLab pages using Cypress and GitLab CI

In this blog post we will deploy a static site to GitLab Pages, and then we will run end-to-end browser tests against the deployed site to make sure it is working as expected. The Cypress Test Runner makes it extremely simple to write full browser end-to-end (E2E) tests, while GitLab CI is a powerful platform for running such tests. The application Today's example is a static site built from a Markdown source file using VuePress [https://vuepress.vuejs.org/]. You can find the full source code

Read more

Make Cypress Videos Look Old School

Recently, we introduced a trial feature called experimentalRunEvents [https://on.cypress.io/experimental] that simplifies running Node code before or after executing a spec file. Inside your plugin file [https://on.cypress.io/writing-and-organizing-tests#Plugin-files] you can add event hooks to perform operations that need OS access. For example, you could: * initialize the database before all tests start running (on this machine) * scaffold the database depending on the spec about to execute

Read more

Generate High-Resolution Videos and Screenshots

This blog post shows how to increase the resolution of the videos and screenshots recorded during the cypress run execution. Videos with higher resolution make debugging simpler and look better when shared with other members of the team. The problem Let's look at the Wikipedia.com website. In my local Chrome browser the home page looks nice and sharp: Let's load this page from the Cypress test. it('loads the site', () => { cy.visit('/') cy.screenshot('wiki', {capture: 'runner'}) }) The

Read more

Testing periodic network requests with cy.intercept and cy.clock combination

This blog post tests an application that fetches new data every 30 seconds, but the test itself runs in milliseconds because it controls the application's clock and stubs the network responses.

Read more

When Can The Test Blink?

Let's take a page where a loading element is displayed while the application is fetching the data from the server. If the test is observing the loading element to make sure it disappears, can the loading be too fast? Can the test runner "blink" and miss the element?

Read more

Testing the HREF Anchor Links

Once upon a time Zach Bloomquist and I were discussing how to achieve an all-time hero status. We were talking about our open source heroes of course, and I wanted to send Zach a link to the testing section of the Develop, Preview, Test blog post written by Guillermo Rauch. The testing section was in the middle of the post, I scrolled, clicked on the # link, and ... screamed in horror: the anchor URL was #undefined Of course, Guillermo quickly fixed the missing element IDs. Yet even then, he mi

Read more

Retry, Rerun, Repeat

> If at first you don't succeed, then dust yourself off and try again.                                                                                          - American R&B singer Aaliyah (1979-2001) The modern Internet is built on retries. Software and hardware failures are normal and expected. Every action can fail, and thus every software system includes code to retry the action several times. During testing, the application might be slower than expected, and thus the test runner needs to

Read more

When Can the Test Submit a Form?

In this blog post we continue exploring flaky tests and how to fix them. We will look at the situation when a page submits a form and then continues to interact with the page, which causes a problem. Tip: you can find all existing similar posts under the tag Flake [https://cypress.io/blog/tag/flake/]. You can find the source code for this blog post in Cypress example recipes [https://github.com/cypress-io/cypress-example-recipes] repository. The error The original error [https://github.com/cyp

Read more

Testing an Application in Offline Network Mode

Modern web applications need to continue working when a network is unavailable, or at least show users the current network status. In this blog post, I will show how Cypress can simulate an offline network status during a test. Note: the source code for this blog post can be found in the "Offline" recipe [https://github.com/cypress-io/cypress-example-recipes#server-communication]. navigator.onLine A web application can ask the browser if there is a network connection; there is a widely support

Read more

Debug a Flaky Visual Regression Test

In our Cypress RealWorld App [https://github.com/cypress-io/cypress-realworld-app] we are using visual regression testing [https://on.cypress.io/visual-testing] to catch unintended style and layout changes. We are using the Percy.io [https://percy.io/] service, and you can see the screenshots at https://percy.io/cypress-io/cypress-realworld-app. During tests, we periodically take image snapshots using cy.percySnapshot command: it("creates a new bank account", function () { cy.getBySel("siden

Read more

When Can The Test Navigate?

In this blog post I will show a test that appears simple, yet acts in surprisingly annoying and frustrating ways. We will slightly change the test to make it reliable and flake-free.

Read more

Sorting the Table

This recipe shows how to verify that an Ag-Grid table is sorted correctly. The source code can be found in the cypress-example-recipes repository under the "Testing the DOM" list.

Read more

Do Not Get Too Detached

When the Cypress Test Runner runs through the test's steps, the application can update itself, causing problems. Often a very frustrating problem our users encounter simply says cy... failed because the element has been detached from the DOM. You can see this error below and its explanation using the link "Learn more [https://on.cypress.io/element-has-detached-from-dom]". Element detached from the DOM error messageIn this blog post I will go through a scenario showing this problem and show how

Read more

Experimental Fetch Polyfill

Let's take an application that makes an Ajax call to the browser to load a list of ... fruits. You can find this awesome and healthy application in Cypress Example Recipes [https://github.com/cypress-io/cypress-example-recipes#stubbing-and-spying] under name "Stubbing window.fetch". Application Every time you visit an application you get five new fruitsThis application presents two challenges to end-to-end tests. First, the data is random - every time the test visits the page localhost:7080 a

Read more

When Can The Test Log Out?

Recently, we released the Cypress Real World App [https://www.cypress.io/blog/2020/06/11/introducing-the-cypress-real-world-app/] - a modern web application with a full set of E2E and API tests showing the recommended best practices for writing tests. From the start, we had Linux continuous integration running tests on every commit. After the release, we added a Windows continuous integration workflow in #421 [https://github.com/cypress-io/cypress-realworld-app/pull/421]- and it started failing

Read more

Extending the Cypress Config File

In this blog post I will show how to implement extends syntax in the Cypress JSON configuration file without waiting for the Cypress team to add support.

Read more

Where Does the Test Spend Its Time?

No one likes slow tests. But before we can make a test faster, we need to understand where the test actually spends its time. In this blog post I will show how to report the total test duration and time per individual Cypress command.

Read more

How to Run Changed Specs First in a Pull Request

Imagine a large project with hundreds of Cypress spec files. If you work on just one feature and open a pull request with small code changes and corresponding updates to 1 Cypress spec file, it makes sense to try running that changed spec file first.

Read more

Streaming Test Results

A single Cypress test might be fast, but if you wait for an entire large test run to finish, you might as well watch the paint dry.

Read more

Run Cypress Tests on Netlify Using a Single Line

Netlify became a popular Continuous Delivery (CD) and hosting solution for a lot of organizations due to its simplicity and power. Pick your repository, click "Deploy site" ... and the site magically appears.

Read more

Working with Select elements and Select2 widgets in Cypress

This blog post shows how to control native <select> HTML elements from Cypress tests. We will also look at how to work with a very popular wrapper library called Select2, that supplants the native <select> elements with an additional HTML markup.

Read more

How to Test an Application that Changes a CSS Variable

Let's take an application that has an <input type="color"> element. When the user picks a new color, the application changes a CSS variable which controls the background color. In action, it looks like this...

Read more

Testing Edge Data Cases with Network Stubbing and App Actions

Your application might be a layered cake of historical data. Often old records are missing pieces because at first the web application never asked for them, or never validated them.

Read more

Working with iframes in Cypress

In this blog post I will show how to interact with DOM elements inside an iframe (even if the iframe is served from another domain), how to spy on window.fetch requests that the iframe makes, and even how to stub XHR requests from the iframe.

Read more

Testing the Browser Notification API

How do you test native browser APIs like Notification? If you don't have a real modern browser during tests, it seems like a challenge. But Cypress controls an Electron or Chrome or (soon) Firefox browser, so your tests can access the real thing.

Read more

When Can The Test Stop?

Imagine a small "application" that shows Window Confirm popup when the user clicks a button. The HTML might look like this...

Read more
Asserting network calls from cypress tests

Asserting Network Calls from Cypress Tests

The Cypress Test Runner can "see" everything happening inside the web application under test: DOM elements, cookies, local storage items, location, etc. The Test Runner can also spy on and stub network requests the application is making to its backend API or even to the 3rd party APIs. This blog post shows how easy it is to confirm the network calls from the web app as expected. What Are Network Calls? Network Calls, in the context of web applications, refer to the interactions between the ap

Read more

Test Your Web App in Dark Mode

Designing a good Dark appearance style takes work, which brings me to the main topic of this blog post - how do you test your web application using a Light or Dark appearance?

Read more

Use meaningful smoke tests

Let's say you cover your application with end-to-end tests. You achieve the elusive 100% code coverage using the Cypress code coverage plugin. Is this enough to guarantee a good user experience?

Read more

Drastically Simplify Testing on CI with Cypress GitHub Action

We are so excited to introduce cypress-io/github-action - an action we specifically coded to make running Cypress end-to-end on GitHub as easy as 🥧.

Read more

Self-testing JAM pages

JAM Stack [https://jamstack.org/] has become a popular way to build websites - you can write your pages using Markdown (the M in JAM), build using JavaScript tools that fetch data from APIs and get mostly static sites that are very fast. In this blog post I will show how each Markdown page can also include its own end-to-end tests using the Cypress Test Runner. Code: The source repo for this blog post is at bahmutov/self-testing-jam [https://github.com/bahmutov/self-testing-jam] and the deploye

Read more

Split a very long Cypress test into shorter ones using App Actions

How can we split a longer test into a series of independent short tests without losing the test's parts?

Read more

Show Code Coverage on GitLab CI

End-to-end tests can be extremely effective at covering a lot of your application's code. Even a single realistic test, because it loads and exercises the entire application, can cover 60-70% of the statements. Introduction Let's look at a concrete example in the cypress-example-todomvc-redux [https://github.com/cypress-io/cypress-example-todomvc-redux] repository. The spec cypress/integration/first-spec.js has a single test "adds todos" that covers 72.6% of the source code with just a few com

Read more

Testing Electron.js applications using Cypress - alpha release

Note: At this time, Cypress is no longer supporting testing Electron.js applications, and are no longer publishing the binaries referenced in this post. Thank you for your interest, and check out our Roadmap [https://docs.cypress.io/guides/references/roadmap.html#Upcoming-features] for information on upcoming features from Cypress. Cypress is great for testing web applications in its built-in Electron.js browser or in an external Chrome browser. Today I am happy to announce that we have started

Read more

Cypress code coverage for applications created using create-react-app v3

Creating a new fully featured React application using Create React App [https://github.com/facebook/create-react-app] v3 is as easy as two NPM commands $ npm i -g create-react-app + [email protected] $ create-react-app my-new-app Then switch to the new folder my-new-app and run npm start or yarn start - the application will be transpiled, and bundled, and will be running locally at url localhost:3000. Under the hood the start script calls the react-scripts commands to actually do the he

Read more

Testing the first built-in browser module, KV Storage, using Cypress

If you live on the bleeding edge of browser technology, you are probably excited about new standard modules that soon (might) come built-in to modern browsers.

Read more

Writing Cypress tests in TypeScript using create-react-app v3

Let’s create a new application using Create React App [https://github.com/facebook/create-react-app] v3 using TypeScript following the official instructions [https://facebook.github.io/create-react-app/docs/adding-typescript] before adding Cypress end-to-end tests. #CRA-applicationCRA application $ npm i -g create-react-app + [email protected] $ create-react-app cra-cypress --typescript ... We suggest that you begin by typing: cd cra-cypress yarn start Happy hacking! Let’s check t

Read more

How to Run Cypress in Docker With a Single Command

If you are a Node.js developer, installing Cypress as a dev dependency in your package.json file or even via direct download seems simple enough. Yet for developers working in other languages like Python or Go, using npm can be an obstacle. We often hear this question from developers: Why don’t you have a Docker image with Cypress pre-installed? Having an image with both OS dependencies and the Cypress test runner ready to go seems to be very convenient. We have listened and created a Doc

Read more

Using Events Emitted from Your Application during End-to-End Tests

Imagine a typical web application during end-to-end testing. The test runner (in this case Cypress.io [https://www.cypress.io]) is driving the app via its DOM elements; it is clicking and typing and scrolling. The test runner checks if the application behaves correctly by explicitly executing assertions against the displayed text the user sees and implicitly by interacting the elements. But what is going on under the hood? Is the web application correctly processing user actions? In this blog po

Read more

Shrink the Untestable Code With App Actions And Effects

This blog post shows how to shrink untestable code and pushes it to the edges of the applications. The core of the web application will remain easy to check from our tests, while external effects can be observed or stubbed. The Cypress.io test runner has limitations. For example, when controlling the network requests, Cypress can spy and stub XHR requests only. We will remove this limit when issue #687 [https://github.com/cypress-io/cypress/issues/687] lands, but for now if your application is

Read more
Picture of a woman typing on a keyboard at a desk

When Can The Test Click?

Recently a user opened a Cypress issue #3135 asking why the cy.click() command was behaving differently than the way click behaved when a user clicks on the button. We’ve seen many people ask similar questions involving click and wanted to address how to ensure your tests behave the same way as a user’s behavior. The problem The problem is straightforward once we walk through how Cypress works. Let’s use the web application at https://github-ylq5q4.stackblitz.io as an example. This ap

Read more
An abstract image of computer code in a code editor

Application Actions: Use Them Instead of Page Objects

Hello, reader. This article was an opinion piece about why app actions are better than page objects. The page object pattern is well known in the QA community, and widely and successfully used in the Cypress ecosystem. Writing maintainable end-to-end tests is challenging. Often testers create another layer of indirection on top of the web page called page objects to execute common actions. In this post I argue that page objects are a bad practice, and suggest dispatching actions directly to the

Read more

Element Coverage

Our users are periodically asking us to instrument application code and save code coverage information after Cypress runs end-to-end tests. In this blog I will argue that code coverage is less than useful for end-to-end tests. Then I will show element coverage that might be a better metric. > Experimental feature This document shows a very early proof of concept. The API and features shown here are likely to change. Use at your own risk. How do you know that you have written enough tests? Ho

Read more
Image of a frozen lake in the winter with ice that is starting to break into pieces

Testing Redux Store

Cypress tests usually operate via public browser APIs like DOM, network, or storage. But the tests can just as easily reach into the application and check if the internal state is updated correctly. In this blog post I will show you how to run assertions against a React application that uses a Redux store. This blog post assumes you’ve already installed Cypress, and you can find the source code in cypress-example-recipes#testing-redux-store. First test I have copied the offic

Read more
A picture of people working together at a conference table with notepads.

ReactiveConf 2018

October 2018 was a great month for Cypress - we went out of Beta with our SaaS services, did a ton of work on the test runner and released v3.1.1, and I did ReactiveConf Roadshow and Conference tour. This blog post summarizes what I have done during the conference. If you could not attend the ReactiveConf 2018, you can still catch up with Cypress by reading. Meetups We have driven around Czech Republic with Matej Stratený and conducted 3 meetups dedicated to web application testing usin

Read more

Cypress Halloween Theme

A good tool can become a great tool when users make it their own through customization. In this blog post I will explain how to customize the Cypress test runner’s appearance through user space color themes. Since it is the season, we have prepared a dark and spooky Halloween color theme that might scare all your bugs away! When Cypress executes your end-to-end tests, the application under test is placed in an iframe element. The tests themselves are placed into another iframe to isolate them f

Read more

Run Your End-to-end Tests 10 Times Faster with Automatic Test Parallelization

I have no patience waiting for a lot of Cypress end-to-end tests to finish running on CI. Just sitting and waiting … staring at the CI badge. Any run taking longer than a minute feels like an eternity. Our Cypress development team felt this pain and decided to do something about it. Today we have a solution that slashes those waiting periods - it is automatic test file load balancing across multiple CI machines using a single --parallel flag. #Load-BalancingLoad Balancing By default cypress run

Read more
An image of someone on his phone in front of a computer looking at a stock trading application

Record Test Artifacts from any Docker CI

We worked hard to make sure the Cypress Test Runner could be compatible with every continuous integration system. Jenkins, Travis, Circle, Zeit Now, Google Cloud Build, or your own homegrown CI system - they should all work with zero or minimum effort. We also worked hard to make our Dashboard Service particularly useful in collecting and showing all of your test artifacts in a single place. Test artifacts When you execute cypress run, the end-to-end tests produce artifacts for each spe

Read more
An abstract image of different shades and colors

Fast Tests, Tiny Docker Image

If the Cypress Test Runner were a person, its best friend would be a person named Docker. Really, Cypress and Docker work so well together! For example, all our CI builds are using cypress-docker-images to include all necessary dependencies in order to successfully install and run Cypress tests. Just run npm ci and cypress run and you are good to go. FROM cypress/base:10 RUN npm install # or even better: RUN npm ci RUN $(npm bin)/cypress run We use this approach to run a lot of te

Read more

JSON Schemas are your True Testing Friend

⚠️Please note: This article refers to the json-schemas package which is no longer supported. The repository has also been archived as a result. We use JSON schemas to describe the data flowing through our system, document API routes, test server code and even validate fixtures used during end-to-end testing. This long(ish) blog post describes in detail how we did this. Introduction At Cypress we have 3 large pieces of software: * The Cypress Test Runner, an Electron desktop applicati

Read more
An abstract of curved, rusted metal

Sliding Down the Testing Pyramid

A good end-to-end test runner, like Cypress, can be very effective at performing integration tests and even at running unit tests. In this blog post I will show how to write view component tests for a Hyperapp application and how to unit test individual functions. The same approach can work for components written using other frameworks like Vue, React, Cycle, Svelte and others. End-To-End tests Recently I watched a screencast video recorded by Alex Barry where he learned Cypress by wr

Read more
An abstract, closeup image of code in a code editor that is blurred

When Can The Test Start?

Can the test runner be too quick? Can the tests start before the application is ready to handle the test steps? It definitely can happen. This blog post shows a little trick one can use to detect when a web application really starts, so that the test runner never has to wait longer than necessary. A slow application I will start with a tiny “web app”. It asks the user to enter a name, and then greets the user with the entered name. Here is the body of the HTML page. <body> What's your na

Read more
An abstract image of blurred light

End-to-End Snapshot Testing

Snapshot testing has taken the JavaScript unit testing world by storm. It removes much of the manual coding and much of the boilerplate, leaving the test runner to save or compare produced values. This blog post shows how the Cypress Test Runner can bring the same power to your end-to-end tests. Experimental feature This document shows a very early proof of concept. The API and features shown here are likely to change. Use at your own risk. I will start by reviewing snapshot testing a

Read more
An abstract image of blurry colors

An Alternative to Protractor for Angular Projects

Fans of Angular CLI get Protractor end-to-end tests generated with each scaffolded project. In this blog post I will show how to add Cypress E2E tests instead with minimum effort (and TypeScript support)! Scaffolding a project Scaffolding an Angular project using ng new <name> is a huge time saver. Just 4 commands give you a complete “Hello Angular” application with all recommended practices. Let us make an app - you can find the results in bahmutov/ng-cli-hello repository. npm i

Read more
An abstract image of javascript code up close

Testing apps hosted on Codepen.io

Lots of people love to quickly throw web applications on Codepen.io. Who can refuse a quick and powerful way to start writing code immediately; sharing results is almost instant. In this blog post I will show how to write end-to-end tests for web apps embedded in a Codepen. Warning: no longer working Warning ⚠️ Codepen.io has added a CAPTCHA to prevent scrapers and 3rd party tools from accessing the hosted web apps. The testing approach shown in this blog post no longer works, but we wi

Read more

Testing Vue web applications with Vuex data store & REST backend

In this blog post we will test a typical web application that uses a central data store to manage state. The data store communicates with the backend that stores the data long term. You can find patterns in React + Redux applications, Angular + MobX and countless other combinations. This example looks at a Vue.js [https://vuejs.org/] web application with Vuex [https://vuex.vuejs.org/en/] data store. The backend server implements a simple REST API to store, modify and delete data. You can find t

Read more

Control an AngularJS Application From E2E Tests

Cypress runs close to your application. Only a thin iframe separates the production code from the testing code. Thus Cypress can take shortcuts and control your application code directly, bypassing always going through the DOM. For example imagine you are developing an AngularJS 1.x TodoMVC application such as http://todomvc.com/examples/angularjs/; how could you take advantage of this? AngularJS from DevTools You can change the data in the application’s model directly from the DevTools [https:

Read more

Add GUI to your E2E API tests

We like to pretend there are two disjointed end-to-end test types: API and browser automation tests. And that the tools used are very different; the API tests are executed by a CLI test runner, while automation tests all fire up a browser. Yet, a good E2E automation test runner, in my opinion, should be really good at working with a remote server API via HTTP. After all, this is what every useful website does. Every website exchanges data with remote servers through HTTP requests. What if the s

Read more
An image of a code editor displaying CSS / SASS code

Good error messages

If I can paraphrase a cliché: bad things happen to good programs Things go wrong, even in the most carefully coded and thoroughly tested applications. In my opinion, a sign of a good app is if it gives meaningful and useful error messages when things go wrong. At Cypress we spend a lot of time trying to make every type of error we display to the user as useful as possible. These include failed assertions during tests (after all, Cypress is an end to end test runner), and crashes insid

Read more

Immutable deploys and Cypress

It is simple to run Cypress both locally and on CI. And it is simple to point your tests to a different server. If you are using immutable deploys, Cypress can be the ultimate “health check”. If the tests pass locally, then on CI, and then at a newly deployed environment - the chances are very high that you can point production DNS record at the new deployment and it will work. Here is an example. Let us take a “stock” React TodoMVC app [https://gitlab.com/bahmutov/todomvc] and see all the mome

Read more

Why I joined the Cypress.io team

You know what is hard? Changing jobs is hard. Testing web applications is hard. Luckily for me, I have an opportunity to tackle both problems at once! I am happy to announce that I am joining the awesome Cypress.io [https://www.cypress.io/] team. This is a big personal change. I spent last three years developing web applications for financial startup, Kensho. I was happy to see Kensho grow from 10 people to 75 and sign very first contracts. > I want the software to be better and by extension m

Read more