When Can The Test Click?

Recently a user opened a Cypress issue #3135 asking why the cy.click() command was behaving differently than the way click behaved when a user clicks on the button. We’ve seen many people ask similar questions involving click and wanted to address how to ensure your tests behave the same way as a user’s behavior.

The problem

The problem is straightforward once we walk through how Cypress works. Let’s use the web application at https://github-ylq5q4.stackblitz.io as an example. This application has an input element that opens a calendar modal when you click on the input. When you click on any date in the calendar model, the modal closes and the date you picked appears as the value of the input element. Here is how it works when I use it in the browser.

Here is the Cypress end-to-end test the user has written - and it looks good to me. 👍

/// <reference types="cypress" />

describe('date component works', () => {
  beforeEach(() => {
    cy.visit('https://github-ylq5q4.stackblitz.io')
  })

  context('calendar opens on click', () => {
    beforeEach(() => {
      // open the calendar modal
      // StackBlitz takes a while to load, thus we need
      // to give "cy.get" a longer timeout for the element to appear
      cy.get('[data-context="container"]', { timeout: 20000 })
        .find('input')
        .click()
    })

    it('calendar auto closes on click', () => {
      const day = 5
      const dayRegex = new RegExp(`^\\b${day}\\b$`)

      cy.get('.owl-dt-popup')
        .should('be.visible')
        .contains('.owl-dt-calendar-cell-content', dayRegex)
        .click()

      // calendar should close on the user click
      cy.get('.owl-dt-popup').should('not.be.visible')
    })
  })
})

Source code

Find the source code for this blog post in cypress-io/when-can-a-test-click repo.

But, when we run the test, it is failing. 🙁

Why isn’t clicking on the date closing the modal?

cy.get('.owl-dt-popup')
  .should('be.visible')
  .contains('.owl-dt-calendar-cell-content', dayRegex)
  .click()

The calendar modal is visible - the assertion .should('be.visible') ensures it is. Then the test runner finds a particular day and clicks on it. Command cy.click() must find the element before clicking, so that is not the problem. We can even see the element Cypress clicks on - thanks to its “time-travel” feature when hovering or clicking on the commands in the Command Log.

Weird - we can clearly see that the cy.click() command HAS performed its action - there is a date set in the input element!

And just to finish describing the problem: if we click on the date AFTER the test has finished, it goes away. The application does behave correctly when an actual user clicks in the web application, even if it is running inside of Cypress.

What is going on? 🤔

Race condition

The red flag in this case is that clicking manually after the test has failed works. If the application fails to respond correctly to the “click” event during the Cypress test, but handles the click correctly after, the problem is likely that the application was slow to respond, while Cypress was fast to act. In this case, when the framework shows the calendar modal, it starts attaching the event listeners to the DOM elements. However before the event listeners are attached, Cypress manages to find the DOM element with the required date and click on it. The “click” goes nowhere, since we know:

If a tree falls in the forest and no one has attached a “fall” event listener, did it really fall?

In our case, there was probably an event listener already attached and ready to process the actual click on a date element (that’s why the date is changing, even if the modal stays open), but no event listener has been attached yet to close the modal. Our test runner is getting ahead of the web application in this particular case.

We can verify that our problem is due to a race condition between the app and the test runner by adding a manual cy.wait() after opening the modal.

cy.get('.owl-dt-popup')
  .should('be.visible')
  .wait(500)
  .contains('.owl-dt-calendar-cell-content', dayRegex)
  .click()

Now the test passes.

Try and try again

We discourage using hard-coded waits.

1. They slow down the tests.

2. The wait running the tests locally in the interactive mode may work fine, but when running in CI may not allow enough time. Thus sometimes tests might fail because the wait value you picked was just at the edge, where any variability in speed will cause flake. To avoid the flaky tests you end up increasing the wait interval, leading back to 1. - hard-coded waits slow down tests.

In Cypress, if you want to wait for an element to be visible, just add an assertion to a DOM querying command like

cy.get('.owl-dt-popup')
  .should('be.visible')

The test runner will automatically re-query the element again and again until the assertions following the cy.get() command pass. This built-in retry-ability works with every idempotent command like cy.get(), .find(), .contains(), etc. The command cy.click() is NOT retried - because clicking actually changes the application’s state!

In our particular case, clicking should be retried though. Because we want to keep clicking until the event listeners are all attached and the calendar modal closes. We can retry arbitrary user actions by using an ingenious 3rd party Cypress plugin called cypress-pipe. It adds a custom command .pipe() that just takes a callback function followed by the assertions to check against. In our case we want to keep clicking until the calendar modal is closed and the particular date element we are clicking on is no longer visible.

// cypress-pipe does not retry any Cypress commands
// so we need to click on the element using
// jQuery method "$el.click()" and not "cy.click()"
const click = $el => $el.click()

cy.get('.owl-dt-popup')
  .should('be.visible')
  .contains('.owl-dt-calendar-cell-content', dayRegex)
  .pipe(click)
  .should($el => {
    expect($el).to.not.be.visible
  })

// calendar should close on the user click
cy.get('.owl-dt-popup').should('not.be.visible')

We have to use our own click function, but otherwise, pipe is very easy to use and most importantly our test passes as soon as the modal disappears.

We can even count how many times our callback was retried before cypress-pipe has computed “the assertions that follow pass” and finished.

let count = 0
const click = $el => {
  count += 1
  return $el.click()
}

cy.get('.owl-dt-popup')
  .should('be.visible')
  .contains('.owl-dt-calendar-cell-content', dayRegex)
  .pipe(click)
  .should($el => {
    expect($el).to.not.be.visible
  })
  .then(() => {
    cy.log(`clicked ${count} times`)
  })

In this particular run it has clicked 19 times.

Of course, this solution to use .pipe(click) to keep clicking on the element will only work for some applications. If our application saved records into the database on each click, retrying click would not be a good option. Since Cypress has no way of knowing the intended behavior of your application, only the author of the application knows, .click() was written to not retry by default.

Related

When Can the Test Start shows a similar problem for when the application startup is taking a long time and Cypress starts the test too early. Read that blog post for its solution.