Do Not Get Too Detached

When the Cypress Test Runner runs through the test's steps, the application can update itself, causing problems. Often a very frustrating problem our users encounter simply says cy... failed because the element has been detached from the DOM. You can see this error below and its explanation using the link "Learn more".

In this blog post I will go through a scenario showing this problem and show how to fix it. The source code is available in the Select widgets recipe.

The flaky error

In the recent blog post Working with Select elements and Select2 widgets in Cypress, I have shown how the test might interact with Select2 HTML widgets. To further answer a user's question, we have expanded the recipe to cover a dynamic data fetching scenario and its tests. In this new section, we interact with a Select2 widget by typing the search string; the widget fetches options to display dynamically based on the results returned from the server. For example, let's search for "Clementine Bauch" by entering the string "clem" and selecting the user from the list of found names:

The test as I have written it originally follows the above steps. The test finds the input element, types the search query and then selects the user name from the list of options fetched by the widget.

it('selects a value by typing and selecting', () => {
  // spy on the search XHR
  cy.server()
  cy.route('https://jsonplaceholder.cypress.io/users?term=clem&_type=query&q=clem').as('user_search')

  // first open the container, which makes the initial ajax call
  cy.get('#select2-user-container').click()

  // then type into the input element to trigger search, and wait for results
  cy.get('input[aria-controls="select2-user-results"]').type('clem{enter}')

  // select a value, again by retrying command
  // https://on.cypress.io/retry-ability
  cy.contains('.select2-results__option', 'Clementine Bauch').should('be.visible').click()
  // confirm Select2 widget renders the name
  cy.get('#select2-user-container').should('have.text', 'Clementine Bauch')
})

The test looks correct, and passes locally

Yet the test occasionally fails when running on CI. Hmm, that's not good.

Luckily, when running the example recipes on CI, we record the test artifacts on the Cypress Dashboard. The recordings for this projects are public at https://dashboard.cypress.io/projects/6p53jw. This particular failed test shows a screenshot at the moment of the failure (but before the error message appears).

The screenshot looks correct: the test has typed the letters "clem" into the input element, and two users with matching names were found ... so what is wrong?

Investigation

To understand what is going wrong in this test, we need to slow it down and observe our application, while the test is interacting with the page. I will add .pause() call after opening the Select2 widget in the test.

// first open the container, which makes the initial ajax call
cy.get('#select2-user-container').click().pause()

The Test Runner pauses after the widget opens.

Notice the little jump after the "click" command? The widget fetches the initial list of values to display; that's the "XHR /users?_type=query" request we see in the Command Log. Let's block that request from the DevTools Network tab to confirm.

When the test clicks on the Select2 widget, the widget shows "Loading..." message while fetching the list of choices. Once the request returns, those choices are shown to the user. Notice another important detail: the initial list of names with ten or so choices includes the name Clementine.

Let's look at the test again.

// first open the container, which makes the initial ajax call
cy.get('#select2-user-container').click()

// then type into the input element to trigger search, 
// and wait for results
cy.get('input[aria-controls="select2-user-results"]').type('clem{enter}')

cy.contains('.select2-results__option', 
   'Clementine Bauch').should('be.visible').click()

The test does NOT wait for the initial query to return, the test does not wait for the search "clem" to return - it blindly searches for the DOM elements with "Clementine Bauch" to be found while the application fetches data and updates the page.

Detached elements

The above test might pass most of the time when running it locally, yet on CI it might fail often due to slower network calls and potentially slower browser DOM updates. Here is how the test and application can get into a race condition leading to the "detached element" error.

1. The test clicks on the widget
2. The Select2 widget fires off the search Ajax call. On CI this call might be slower than expected
3. The test types "clem" search string
4. The Select2 widget receives the response to the first search Ajax call with ten user names, one of them being "Clementine Bauch". These names are added to the DOM
5. The test then searches for the visible selection "Clementine Bauch" - and finds it in the initial list of users.
6. The test runner is then about to click the found element ... when 💥 the second search Ajax call for "term=clem" returns from the server. The Select2 widget removes the current list of choices and shows just the two found users: "Clementine Bauch" and "Clementina DuBuque".
7. Cypress throws an error because the DOM element with the text "Clementine Bauch" it was about to click is no longer linked to the HTML document; it was removed from the document by the application, while Cypress still has a reference to that element.

You can code the test to detach the element yourself to observe the problem by inserting .then between the cy.contains and .click commands:

cy.contains('.select2-results__option', 
            'Clementine Bauch').should('be.visible')
  .pause()
  .then(($clem) => {
    // remove the element from the DOM using jQuery method
    $clem.remove()
    // pass the element to the click
    cy.wrap($clem)
  })
  .click()

As we step through the test commands you can observe removal of the element followed by the ".click" throwing an error.

Avoid race conditions

Once we understand how our application and the test runner interact during the test, we can solve the underlying race condition and make the test flake-free. We want the test to always wait for the application to finish its action before proceeding. For example, the application loads the initial users asynchronously on click - let's have the test wait for it using built-in retry-ability.

cy.get('#select2-user-container').click()

// flake solution: wait for the widget to load the initial set of users
cy.get('.select2-results__option').should('have.length.gt', 3)

// then type into the input element to trigger search
// also avoid typing "enter" as it "locks" the selection
cy.get('input[aria-controls="select2-user-results"]').type('clem')

When the test types "clem" into the search box, the application fires off an Ajax call that returns a subset of users. Thus the test needs to wait for that new set to be shown - otherwise it will find "Clementine Bauch" from the initial list and run into the detached error! We know there are only two users matching "clem", thus we can confirm the number of displayed users again to wait for the application.

// then type into the input element to trigger search, and wait for results
cy.get('input[aria-controls="select2-user-results"]').type('clem')

// flake solution: wait for the search for "clem" to finish
cy.get('.select2-results__option').should('have.length', 2)

cy.contains('.select2-results__option', 'Clementine Bauch')
    .should('be.visible').click()
      
// confirm Select2 widget renders the name
cy.get('#select2-user-container')
  .should('have.text', 'Clementine Bauch')

The test now checks the number of options displayed by the Select2 widget to be two; thus it waits for the application to finish doing its part. Then the test selects the choice with text "Clementine Bauch" - the application is not doing anything, and the DOM elements will not be suddenly updated causing an error.

Slow down network

We can simulate the slower network responses by stubbing the XHR calls. For this, we can copy the response sent by the server and save it as a fixture file.

Similarly we can copy the response to the "term=clem" XHR call

Both responses should be saved as JSON files in cypress/fixtures folder.

Now we can stub the network calls using cy.route to simulate slow responses.

cy.server()
cy.route({
  url: 'https://jsonplaceholder.cypress.io/users?_type=query',
  response: 'fixture:query.json',
  delay: 1000
}).as('query')
cy.route({
  url: 'https://jsonplaceholder.cypress.io/users?term=clem&_type=query&q=clem',
  response: 'fixture:clem.json',
  delay: 1000
}).as('user_search')

The test still passes - because we forced the test runner to "wait" for the application to finish its logic in response to the test steps.

More info

You can find the flaky test and the described solution in the pull request 526.

Read the blog post When Can The Test Log Out? where I show another instance of the test runner and the application racing each other; the problem was also solved by making the test runner "wait" for the application to react to the test's action before proceeding.