Cypress Case Study: Quizlet

With more than 30 million active users every month, Quizlet is now the world’s largest student and teacher online learning community. It’s used by one in two high school students and one in three college students, so keeping their application up and running smoothly is a top priority. In this case study, we’ll dig into why and how Quizlet now trusts Cypress as their primary end-to-end testing framework.

The Problem

Quizlet was using Capybara, which is a Ruby layer on Selenium for E2E testing, but it was the only Ruby code in their codebase. This really didn’t fit their needs, as the Quizlet dev team preferred their testing framework to be written in a language they coded in everyday: JavaScript.

The other issues Quizlet experienced were the limitations of Selenium itself. Although many popular end-to-end testing frameworks are built on top of Selenium WebDriver, Quizlet found them to have some major drawbacks for their testing needs. Their Selenium-based tests took a long time to configure for their environment, and the tests were quite slow, as Selenium had to spin up a browser for every test.

The Solution

In order to find the integration testing framework that worked for them, Quizlet tested out Cypress, Puppeteer, and TestCafe. They evaluated them based on these key criteria: their cross-browser support, tech support, debugging & coding ease, and the runtime of a full suite of tests. After reviewing their options, it was clear to Quizlet that Cypress excelled in each of these categories.

Unlike TestCafe, Cypress’s assertion libraries, Mocha and Chai, are familiar to Quizlet developers: whose current set of of assertion libraries uses describe() and it() blocks. Debugging integration tests can be painful, but not in Cypress. Whether running tests in interactive or run mode, it’s easy to debug the code through the output in CLI (run mode) or in Chrome DevTools (interactive mode). Cypress is also built on Node, so it doesn’t depend on Selenium and is a standalone framework that supports JavaScript.

Another feature that stood out to Quizlet was Cypress’s relatively mature community; Cypress has many, easy-to-find resources and support for issues or requested additions. There are companies who are already using Cypress for integration testing, such as Slack, Crunchbase and 99designs. Quizlet also appreciated Cypress’s diligence in actively updating its API documentation and changelog with each release.

The Results

After reviewing Cypress, Puppeteer, and TestCafe, it was clear to Quizlet that Cypress was the best end-to-end integration framework to meet their needs. Cypress hit all of the requirements Quizlet needed in an integration testing framework.

Quizlet has had a very positive experience with Cypress so far, and has noted how responsive the community is to any issues that they’ve had in getting started. “I was personally really happy that the Cypress community is very active and there are people on board to help us troubleshoot our problems” says Evelyn Chan, a software engineering member of Quizlet’s web team. “I really appreciated how fast and diligent you guys respond to issues on Github, it really helped me when trying to integrate Cypress with our product.”

Quizlet uses Cypress to run a suite of smoke tests. Each commit to master is built to staging and tested on heavily trafficked user flows. They keep the test suite to under five minutes to maintain their deployment velocity. They also run a broader suite periodically against production to detect any systems integration issues. Ultimately, Cypress has made it easier for Quizlet engineers to write tests that are reliable and catch both application bugs and integration problems before they reach users.