Global FS and Reinsurance Leader Secures 40% Cost Efficiency with Expert Quality Engineered Managed Testing Services

Client overview

Our Client is a global multi-line reinsurance and financial services company. The Client was embarking on a multi-year program of work to modernize the IT applications that support its global commercial risk products.

Quality deficiencies were hampering competitive edge

The Client had been working with the same three Tier 1 suppliers for the last seven years, who were too comfortable with the status quo, with their testing maturity levels assessed at TMMi level 1/2. The Client’s transformation initiative was being executed in Agile and was in year two of the journey, but results were faltering.

With the Client’s frustration growing, Qualitest was engaged to transform the service and to address the general lack of focus on testing activities, which was resulting in quality and agility issues.

At the project outset, we concluded that the Client lacked overall testing governance, and its processes and quality controls required improvement. There was a lack of industry best practices, training, and upskilling for the teams delivering testing at all stages of the development lifecycle. Our analysis concluded that the Client’s specific challenges were:

• Test Maturity was at Level 1/2 (TMMi) with no industry-standard practices in place.
• Production stability issues with a defect leakage rate of 10%.
• Defect leakage into UAT (13%), increasing the dependency on business teams and causing frustration.
• Automation coverage <5% and no in-sprint automation focus slowing down the testing process and extending project timelines.
• 30-35% of unproductive effort due to test environment and data issues.
• Metric trends not improving, no improvements from release to release.
• Shift Left, for both functional and non-functional testing, was non-existent.

Industry best practices establish quality method of operation

Qualitest conceptualized and deployed an overall governance framework, and established a testing services management team, with governance and management layers, within the Client’s testing and quality assurance organization.

Our test managers were deployed across the projects, collaborating, guiding, and overseeing and improving testing process and quality outcomes from blended vendor teams. We implemented new best-practice Quality Engineering initiatives and approaches, supported by our Centre of Excellence (CoE) teams, to provide training, and upskilling for the teams delivering the testing work.

We performed a diagnostic assessment of the incumbent landscape using Qualisense, our proprietary tool which offers AI-powered Quality Engineering/DevOps solutions, including failure prediction and test case optimization. With AI-generated insights on historical data enabled, we laid out a complete Test to Quality Engineering transformation roadmap, with a defined approach to deliver the change. Our AI analysis also presented detailed data on the root causes of the issues the Client was facing. This roadmap covered:

• Enhanced test automation – Laid out the automation framework and scalability for Expanded automation (intelligent), coverage and leverage, test data automation and configuration of test data.
• Test program management – product, configuration & environment.
• Performance and data gap analysis.
• Complete regression test suite gap analysis and prioritization.
• Tool integration and virtualisation strategy – covering regression suite re-build / foundation for a dynamic AI-based suite selection solution and identified non-conventional automation opportunities.

Beyond the solutions already implemented, Qualitest has also specified multiple Proof of Concept initiatives for the Client’s continuous improvement:

• Requirements Quality Engineering using Qualisense – results showing 85% requirements quality failure rate have been presented to the Client’s leadership team.
• Performance Engineering (PE) – Performance Testing by Design – to integrate PE into the scrum lifecycle, and building automated checks for developers into pipelines.
• Security Engineering (SE) – Security by Design – to integrate SE into the scrum lifecycle, and building automated checks for developers into pipelines.

Key benefits

Our detailed roadmap for this Client confidently commits Qualitest to overall cost savings of 40% over three years. Initial results include:

• 2,000 hours saved by utilization improvements.
• 100% critical scenarios automated.
• 47% regression automation achieved (increased from <5% in three months). Critical projects are already at 100% target.
• Defect rejection rate reduced by 85%.
• Defect detection rate increased by 3.6x.
• Defect removal effectiveness increased to 98%.