Customers Relias

Relias, a prominent provider of healthcare education and training solutions, caters to over 10,000 healthcare organizations and serves approximately 4.5 million caregivers. Their platform focuses on enhancing clinical knowledge and improving patient care outcomes through personalized learning experiences.

Despite their success, Relias faced early challenges in optimizing their testing processes. Adopting Cypress in 2019, the organization operated on a limited plan with restricted runs and user access. However, as the utilization of Cypress grew, it became evident that the existing plan’s limitations were hindering testing efforts. Recognizing the need to implement widespread adoption of testing practices beyond the traditional testing team, Relias saw the imperative to upgrade to the Enterprise plan on Cypress Cloud.

Challenge

Eventually, Relias grew to a team of approximately 38 test engineers, with significant interest from additional developers and managers in accessing test results. Recognizing the need to prioritize testing across the entire department, not just among test engineers, Relias sought to empower everyone in the organization. Previously, the organization faced limitations with scheduled test runs occurring nightly for development and bi-weekly for staging.

Providing our extended team with additional tools, like Cypress Cloud, was driven by my desire to demonstrate the profound impact of robust automated testing on a company’s success. It also empowers developers interested in embracing whole team testing. In my opinion, everyone involved in testing, or even the entire team, should actively participate in testing efforts. When I joined two and a half years ago, there was a prevailing ‘throw it over the wall’ culture, where testers handled manual testing and automation was solely managed by a small team. Over the past two years, and with the help of Cypress, we’ve worked to expand this approach, and the results have been remarkable, spreading like wildfire throughout the organization.

Another significant hurdle was the time-consuming and technically demanding process of setting up Cypress projects for other teams in pipelines, so that they could access data beyond simple pass/fail responses. This manual setup process not only consumed valuable resources but also impeded the organization’s agility in deploying and testing applications efficiently. Additionally, generating visually appealing HTML reports using a Cypress extension required substantial effort from team members, diverting valuable time and energy from core testing activities.

Solution

Through the adoption of Cypress Enterprise, Relias has undergone a remarkable transformation in its testing practices, fostering a testing-centric culture that emphasizes collaboration and efficiency across the organization. One of the key benefits of Cypress Enterprise was the elimination of manual setup processes for Cypress projects in pipelines. Instead, the organization seamlessly connected the dashboard for other teams, significantly reducing setup time and complexity. Leveraging Cypress Cloud’s native capabilities, Relias no longer needed to generate HTML reports separately, further enhancing efficiency and productivity.

The most valuable analytics dashboard in Cypress Cloud for the team at Relias is “Flaky tests.” With a current focus on establishing a suite of smoke tests, the team prioritizes minimizing flakiness. Even if tests eventually pass, initial success is crucial to save performance time. A high flakiness rate results in tests being excluded from the smoke suite. The team utilizes the flakiness rate, accessible via a dropdown, to monitor test stability. Tests with a flakiness rate exceeding 50% are automatically removed from the smoke suite and must be maintained to regain inclusion.

Among the other essential analytics dashboards in Cypress Cloud, Malia and her team pay particular attention to ‘Slowest tests’ and ‘Run duration.’ Currently, focus lies on identifying and quarantining the slowest tests for performance enhancement. This prioritization aids in reducing overall test suite execution time and exposes potential architectural or design weaknesses requiring attention to enhance system performance.

The Test Replay feature emerged as a game-changer for Relias, revolutionizing their approach to addressing test failures. Prior to its implementation, encountering a failure necessitated a laborious process of manual inspection and troubleshooting. Team members would painstakingly step through the test, scrutinizing API responses and backend behaviors to identify the root cause. This not only consumed valuable time but also introduced the potential for human error.

The enhanced visibility from Test Replay has made a world of difference in our debugging efforts. We estimate that we’re saving close to 60 hours of debugging time per month. We can swiftly identify and address issues without the need for manual intervention. It’s been a real time-saver and has greatly improved our testing efficiency.

Result

By bypassing manual setup processes and leveraging Cypress Cloud’s advanced features, Relias achieved a significant time saving of approximately 20 hours per month. Moreover, Test Replay saved the team close to 60 hours of debugging time per month, and the utilization of the test parallelization feature resulted in an impressive 80% reduction in test run times, leading to substantial cost savings for their CI platform.

In the future, Relias looks forward to leveraging the value-added services provided by the Enterprise plan. This includes access to a dedicated account team and best practice sessions aimed at optimizing account usage. With these enhancements, Relias aims to further enhance their testing practices and drive continuous improvement.

The journey towards a testing-centric culture at Relias has been driven by the empowerment of teams through Cypress Enterprise. By democratizing access to advanced testing tools and capabilities, Relias has fostered a culture where testing is not just the responsibility of a select few but a shared priority for every individual in the organization. As Relias continues to evolve its testing practices, Cypress Cloud will remain instrumental in empowering teams to deliver high-quality solutions and drive positive outcomes for their clients.