Client Overview

The Client is an international software company of over 25 years’ standing, with a core product offering of business intelligence and data visualization solutions.

No Bugs Allowed

The Client had recently moved their CRM platform to Salesforce, a highly complex migration involving six Enterprise technologies with disparate functions:

  • NetSuite (Billing)
  • Marketo (Marketing Automation)
  • Auto0 (Website Security)
  • Zuora (Subscription Management)
  • Dell Bhoomi (Data Integration)
  • SpringCM (e-Signature Application).

The Client’s in-house development team and an outside QA team had handled the testing, mainly focusing on functional testing at the individual systems level and not on the end-to-end journey. That strategy had resulted in significant defect leakage into production. Business confidence was low, impacting the customer relationship and potential revenue.

The Client wanted to reduce the critical functional defects in both the User Acceptance testing and production environments to zero. They also wanted to increase functional test coverage and expand testing to cover integrations.

The Parts Work. How about the Whole? 

During handover activities with the previous teams, we quickly saw the challenges with their test approach: limitations of the functional testing performed, complex data dependencies and lack of knowledge of the technologically diverse applications outside the core solution.

Armed with this understanding, in partnership with the Client we implemented a detailed functional test strategy, covering Integration and User Acceptance utilizing end-to-end customer journeys. Our emphases included:

  • Functional system testing of each individual system
  • Integrated end-to-end testing of the Client’s data visualization tool
  • User Acceptance testing of the complete solution
  • Data and functional testing reports for customized dashboards.

Finally, to ensure a holistic approach by dedicated professionals, we identified personnel with the precise skills to perform the end-to-end testing.

The Matrix: At-a-Glance Business Risk Measurement   

Working with the Client’s business analysts, we mapped all business requirements to the applicable integrated systems and design elements. We also reviewed the defects found in production and incorporated them into our test cases.

Next we created a matrix to develop end-to-end test conditions and scripts, test data and environment requirements. All test cases—and therefore results and defects from System Testing to Integration Testing to User Acceptance testing—were measurable against our matrix, providing quantification of coverage and quality for our end-of-test status reports.

The matrix also provided end-to-end visibility of the business impact of all changes, enabling stakeholders to quantify and visualize the level of remaining risk. This informed their decisions on whether to promote code to Go-Live.

“The matrix also provided end-to-end visibility of the business impact of all changes, enabling stakeholders to quantify and visualize the level or remaining risk.”

Key Benefits

Qualitest’s comprehensive, risk-based test approach and thorough analysis of coverage and quality enabled the Client to achieve significant business improvements.

  • Reduced critical defect leakage to production to zero.
  • Enabled stakeholders end to end visibility and quantification of risk based on business impact.
  • Established end-to-end scope of all the systems in the product, leading to a better understanding of the business processes for future development.
  • Improved business users’ confidence and provided a better understanding of user needs.

“Qualitest’s comprehensive, risk-based test approach, extensive automation expertise and thorough analysis of coverage and quality enabled the Client to achieve significant business improvements.” 

Download the PDF