Customers Pendo

Pendo is a product experience platform that combines product analytics, in-app guides, and feedback in a single place. Pendo helps companies understand how customers engage with their products and use those insights to improve the experience. Non-technical teams use Pendo to facilitate important customer interactions such as onboarding, highlighting new features, providing support, running surveys, and driving product-led growth.

As a market leader in understanding and improving the user experience, Pendo understands the importance of quality.

Having adequate test coverage helps us feel confident that we’re delivering a high quality product that customers enjoy using.

Challenge

In the early days of its hyper growth, Pendo faced challenges with maintaining quality due to the lack of organizational adoption of a testing solution. The highly integrated nature of their products led to shipping code that unintentionally affected other areas of the app, impacting customers.

In their drive to improve quality, Pendo transitioned from manual testing to a homegrown automated solution. However, the team soon discovered that building and maintaining the solution on their own was challenging and resource-intensive. The process of troubleshooting inaccurate failure reports delayed releases and diverted the team’s focus from more customer-focused work. Additionally, the solution was difficult to set up, taking engineers a full week to get started. “We had this framework for months, but no one was successfully continually writing tests in it yet,” Blake says.

Finding a test automation solution that was stable, efficient, and easily adoptable became a priority. Blake set out to explore their options.

Solution

After discovering Cypress on Vue’s documentation in 2019, Blake was able to quickly incorporate it into the Pendo repository. “In one day, I automated almost all tests in an area of our product. It took us two weeks to replace six months’ worth of work from the team using the homegrown solution.” Shortly after, that team dropped the homegrown solution and started using Cypress.

Cypress spread rapidly through Pendo because it was reliable, well-documented, user-friendly, and had a robust community. In a few minutes, a frontend engineer or Quality Engineer could NPM install Cypress and write an effective test. “Cypress was so intuitive, we didn’t have to do any training,” Blake says. “Even associate engineers, some of whom are college grads without Javascript experience, can pick up Cypress quickly,” says Blake. In their first year of using Cypress, the Pendo team wrote 3,500 automated tests.

The Pendo team successfully relaunched a feature that was rolled back for quality issues. “Cypress helped us catch a few bugs with that product. Seeing the code coverage actually evolve with the code was a huge win,” says Mallory.

As Pendo continues to scale rapidly, the team turned to Cypress Cloud to help them maximize their investment in product quality by:

• Saving valuable CI resources. Cloud intelligently parallelizes Cypress test specs across multiple virtual machines in CI, and allows Pendo to configure canceling runs after test failures.

• Increasing developer productivity. Cloud gives Pendo’s developers clear visibility into why tests fail or flake in their CI pipeline, reducing time spent reproducing and debugging problems in an otherwise headless environment.

• Facilitating a culture of quality. “Cypress’ Slack integration is indispensable. We post failures to team channels so they can fix bugs quickly,” says Blake.

• Providing visibility to leadership on the value of their test suite. “Leadership wants to know that we’re not sacrificing quality for other priorities,” says Mallory. “Cypress Cloud helps us prove that we are balancing speed, time, quality, and cost.

Result

Today, Cypress is used by 100% of Pendo’s quality organization and front-end developers across 12 repositories. Cypress has enabled independent, fast-moving teams to reach their product quality goals. Instead of spending time resolving customer-reported regressions in core workflows, frontend teams can ship new features with confidence.

Before Cypress, product quality was the highest complaint among NPS detractors. In six months, our NPS increased after resolving those quality issues with Cypress.

With Cypress, Pendo’s Quality team can focus on edge cases and other interesting, more impactful challenges. “This is a great quality of life improvement for me. I can focus on the bigger picture – making sure the system supports the data flow we want, versus checking that a drop down is functional,” says Mallory.

Pendo is now able to push releases with the scale and confidence unimaginable before Cypress Cloud. Today, more than 5,000 automated tests cover their quickly growing products. Testing that used to take days now completes in 20 minutes, saving more than 6 months of CI machine time in the past year. “As our test coverage expands, I’m confident when reporting to leadership that it’s time to increase our testing budget. Testing is a net-positive for Pendo and our customers, not a cost-center,” says Blake. “We used to be nervous about product launches, and had to build in buffer time for quality. Now we can push releases twice a week, and it’s business as usual.”

Just try Cypress. It’s such a low investment to configure. Don’t even think about it. Just do it. Cypress is worth the time investment and resources.

What's next

Pendo is looking forward to adding automated testing to their component library with Cypress Component Testing. “We would love to run Cypress with every PR, but trying to use our end-to-end tests to deeply check implementation details made them brittle and flaky. We can convert these to component tests instead for a faster feedback loop and reduced flake,” says Blake.

Cypress Component Testing will also allow Pendo to simplify their tech stack. “We’re looking forward to replacing an expensive test management tool that we use to track manual regression tests with another tool we already use – Cypress!” says Mallory.