Global Consumer Goods Company Achieves Pan-enterprise DevOps Transformation

Client overview

Our Client is one of the world’s largest consumer goods companies with an extensive portfolio of household names.  

To maintain its competitive edge, the Client sought an enterprise-wide digital transformation to be embedded at every level across the board. The Client turned to Qualitest to deliver this coordinated digital transformation with DevOps and Agile in scope.

Disparate vendor landscape was impacting delivery quality

The Client had already commenced an Agile DevOps transformation project with their flagship ERP system, but they were encountering issues because they had no defined Quality Assurance (QA) Framework or processes in place to hold their vendors to account for the quality of code deliveries. The Client sought to standardize and roll the Agile DevOps transformation out across the entire organization, and they wanted to do it rapidly but in a structured and properly governed way. 

They had been working with many vendors who were each following their own disparate processes, developing software and testing it in their own way. There was a huge variation of quality within the client’s QA stage and, with no standardized and defined QA process, the resulting deployments to the live environment had widely varied results and had resulted in significant production issues.

Enterprise-wide Quality Assurance Governance Framework underpins a universally adopted Quality Engineering culture

Qualitest’s initial assessment made 18 recommendations. Seven of the recommendations were selected for wave one of the overall transformation strategy:

• Build a Quality Assurance Framework to codify and standardize quality definitions – include project gates for code deliveries from vendors, and method of achieving it. Support with defined core metrics that allow quality to be consistently measured and reported across the enterprise.  
• Deliver a QA and Automation policy – to define both scope of QA for the Client and articulate quality standards across the organisation and vendors. 
• Build a Master Test Strategy with approaches for QA, Automation and Test – incorporate a Shift-Left QA methodology and perform a tooling assessment, implementing automation tools that are accessible to all. Collaborate with workshops and target key systems to ensure the strategy’s integration, and to ensure adoption through buy-in of the affected teams. 
• Develop documented Quality Standards – Establish governance mechanisms (review processes, approvals, standards, expected test structures, reporting dashboards) that provide ways and means for governance across programmes. 
• Develop documented change management standards – Communicate to enable workstream teams. 
• Build a requirements definition framework including definition of quality – to give all target systems access to an improved requirements validation process along with a simple prioritisation method. 
• Perform a test case migration POC – to standardise and unify test management tools.

In building the automation strategy we selected our proprietary tool, Qualiframe. Qualiframe is an Enterprise Automation Framework designed to expedite automated test design. This proved the optimum solution for integration with open-source and commercial tools.  

Using Qualiframe, we built a test execution solution which is entirely automated, from the daily trigger to run the test suite before the working day starts, to automatic triggers when either a test case is changed, or code changes are uploaded.

For every run, the results are now automatically updated will full evidence (including detailed html report including videos and logs and DOM (Object model details for developer triage) in the test management tool. 

We implemented a Guild approach for Automation and QA to support each program with best practices into each project. This supported sprint activities and the scrum teams to address the backlog and build sustainable solutions which can be reused. Our education of the Client’s teams, and collation of collateral, has ensured this accelerated as processes were adopted, embedded and refined. 

Key benefits

• Automation has reduced the Regression window from 1 week to 2 hours (>95%) and is integrated into the CI/CD pipeline to run every day and automatically on code check-in. 
• Automated test execution cost $0.008 per minute, down from $0.064. 
• Automation scripting effort has been reduced by >30% by using automation framework and reuse of established assets. 
• QA Framework for internal and external use to standardise and govern the quality of deliveries from Vendors and reduce the burden on client’s QA function and delays in releases. 
• Requirement framework with inbuilt workflows to improve the quality and creation of requirements with hygiene dashboards to flag non-compliance from requirements to defects.