Online Dating Company Significantly Speeds Up Time to Market and Delivers Digital Confidence with Advanced Automation

Client Overview

The Client is an industry leader in innovative and unique online dating services, connecting millions of people around the world. Driven by high-tech advancements and outside-the-box thinking, their brands have grown to become highly recognized. The Client has millions of members in more than 50 countries worldwide.

To remain competitive in an already highly competitive market, the Client needs to always ensure a flawless user experience through consistent and stable quality.

A Good Partner Makes All the Difference

The Client has a large, monolithic legacy application and wanted to be able to reduce time-to-production to a matter of days once new code drops. With the legacy system it would take a couple of weeks to deploy/test and this delay could have a negative effect on revenue.

The Client needed a stable, reliable, and reproducible automated suite to retire ~1500 manual tests and free-up those manual testers for other efforts.

They also had a roadmap for additional projects that spanned different functional areas, all testing the modernization and optimization of both new and legacy codebases.

By combining Qualitest’s test automation expertise and resources with Sauce Labs’ best in class platform coverage we were able to deliver an end-to-end solution. The client could have access to have access to real devices and emulators as needed, greatly simplifying their testing, and ensuring that the code base was stable against a diverse set of platforms.

But even after identifying the path forward, progress on automation was slow.  Qualitest stepped in to support the Client’s challenges:

• Automate over 1,800 tests using open-source automation tools and offshore automation engineers to keep costs down.
• Provide experience with open-source automation across diverse platforms using frameworks that create repeatable tests.
• Demonstrate proven experience at deploying and using Sauce Labs instances at scale and unattended.

Give the People What They Want – Faster

Qualitest implemented an iterative process model, with three clear, well-defined phases:

• Phase 1 – Triage the situation and establish an automation foundation
• Phase 2 – Stabilize the automation
• Phase 3 – Steady-state automation

Phase 1 – Triage the situation and establish an automation foundation

In this phase, Qualitest automation engineers and architects partnered with the Client to understand:

1. What needed to be tested.
2. What open-sourced framework was being used and how it could be leveraged.
3. Learning the application under test.
4. Validating steps to be taken with the Client’s timeline.

The Client already had a team of talented automation engineers on staff.  The goal was to combine the framework they had developed with Qualitest’s proprietary tool Qualiframe.

The combined framework would allow for existing automated tests to be incorporated into the new automated test repository without having to be re-written and with minimal re-architecture.  Tests could be created quickly, with minimal maintenance, and easily connect to the Sauce Labs environments.

Using a small set of pilot tests, Qualitest and the Client were able to construct a repeatable process that could be used by offshore resources to create, maintain, and run automated tests going forward.

Phase 2 – Stabilize the automation

Stabilizing the Client’s automation had two objectives:

1. Qualitest would assume 80% ownership of all automation after Phase 1.
2. Increase automation velocity to 100 new tests per month.

Taking an iterative, Agile-like approach in Phase 1 allowed Qualitest to quickly assume ownership of both the automated and manual testing repositories.

Most of the automation errors that happen using open-source tools are because of coding errors, framework errors, or automation engineers using their own processes instead of a shared process.  Iterating as Qualitest took on more work in Phase 1 eliminated these common missteps early on.  Qualitest taking on more work enabled the Client’s automation engineers to work on new parts of the modernized architecture.

Assuming 80% ownership of all automation also enabled Qualitest to increase automation velocity.  The scripts in Phase 1 were decomposed to common navigation paths and unique business functionality.  Doing this allowed Qualitest to create a library of functions that now made up 75% of each automation test case with the new automation only being done to cover the unique features of the new test case.

Phase 3 – Steady-state automation

Qualitest and the client finished Phase 2 in three months. In Phase 3, Qualitest accomplished the following:

1. Owned 100% of all automation.
2. Continually enhanced and optimized the joint open-source framework.
3. Provided the Client with reverse knowledge transfer on any framework changes and automation best practices.
4. Ensured that 100% of purchased Sauce Lab environment time was fully used.

Qualitest was able to work with the Client to create an automation factory that gave Qualitest 100% ownership of all automation.  We were fully responsible for the creation, maintenance, and execution of the automated tests, while The Client’s automation team still leads the overall automation effort, identifying what areas to automate next.

Additionally, Qualitest was able to achieve full, continuous, automated test execution and accelerate Sauce Labs time to value and maximize existing investment.

Why the solution worked

The solution Qualitest implemented would not have been successful if not for clear, constant communication between the Client, Sauce Labs and Qualitest.  It was this partnership where the three teams held each other accountable that facilitated the success and created a one-team mindset.

Additionally, there were other actions and benefits created through Qualitest’s automation efforts:

• A formal coding standard for all automated tests.
• Integration with the Client’s email servers to test email capabilities.
• Standardized onboarding strategies and assets used by new Qualitest and Client resources.

Key Benefits

Qualitest and SauceLabs were able to meet all the Client’s goals within the allotted time frame:

• Significantly reduced time to market with a 17.9% reduction in the targeted test cases, for new code features that were considerably delayed by the manual testing effort.
• Onboarded new automation resources with efficiency and proficiency.
• Triage tested to identify the pool of tests that could be automated and those that couldn’t.
• Helped the Client to keep costs down through right staffing, including increased flexibility if there were any delivery delays.
• Transformed the automation effort into a Managed Test Service.
• Expanded automation coverage to include more devices and configurations through a modular automation framework and a plethora of execution options through Sauce Labs.

As a result, the Client was able to speed up releases and add more modernization tasks per release that will retire the legacy architecture earlier.