Guest Post: Improve your error screenshots in Cypress

Note: This post was originally published by Filip Hric on Medium. Filip is a QA lead at Slido, a lifelong IT enthusiast with a background in psychology, and a Cypress Ambassador.

Cypress makes it incredibly easy to read their code. Even people that don’t work with Cypress here at Slido can easily understand what is going on in the test code.

That’s why I overlooked the cy.log() command. It seemed kind of arbitrary and didn’t make too much sense to add extra log into what is already pretty readable flow. Cypress pretty much nailed it when it comes to readability of code. Not only that, readability of test reports is amazing as well.

When working with headless mode, you can actually see what your test does, since Cypress records video for your headless runs. If you need to examine a failed run, you can easily look into what your test did before it failed.

Extra tip: Did you know, you may choose the option to upload video only on failed runs? Save some extra seconds by adding videoUploadOnPasses: false to your cypress.json config file.

The situation changes rapidly as your test suite grows. With over 1000 tests here at Slido, we sometimes face a situation where we have to look into failures of multiple tests. It takes slightly more time to look into video than to look at a screenshot. Although video shows the whole test run, screenshots can be more efficient if done properly. I don’t know about you, but I’d rather go through 10 screenshots than 10 videos to find out why a test failed.

But, on an error screenshot, you are limited to screenshot height so you probably see only a couple of commands. This is where I found out cy.log() might be handy. With cy.log() I can caption my screenshots, and give more context to what the test was doing before it failed. I started adding cy.log() commands to each step of my test, to describe action that is being executed.

Then, I added a few tweaks to this.

1. I rewrote the command to add a triple dash in front and at the end of the message, to make my logs pop out. Basically to guide my eyes better (yes, I know some psychology)
2. I added a counter, so that every log has a number and I can get a grasp of where the test failed

The result looks like this:

Adding this was in fact quite easy. I added this piece of code to my support/index.js file:

beforeEach( function() {
  window.logCalls = 1;
});

Cypress.Commands.overwrite('log', (originalFn, message) => {
  Cypress.log({
    displayName: `--- ${window.logCalls}. ${message} ---`,
    name: `--- ${window.logCalls}. ${message} ---`,
    message: ''
  });

  window.logCalls++;
});

While writing this blog, I realized, that these logs can be further used for creating better error logs. Each of these steps can be added to the error message, which would look something like this:

Basically we are just creating an empty array that we are going to output as a part of error message when a test fails.

beforeEach( function() {
  window.logCalls = 1;
  window.testFlow = [];
});

Cypress.Commands.overwrite('log', (originalFn, message) => {
  Cypress.log({
    displayName: `--- ${window.logCalls}. ${message} ---`,
    name: `--- ${window.logCalls}. ${message} ---`,
    message: ''
  });

  window.testFlow.push(`${window.logCalls}. ${message}`);
  window.logCalls++;
});

Cypress.on('fail', (error) => {
  throw new Error(error + '\n\nTest flow:\n' + window.testFlow.join('\n')); 
});

These error messages are written out to the Cypress Dashboard, so if you caption them right, boom! Instant error scenario! There goes a JIRA ticket 😆

Let me know what you think about this in comments.