Client Overview 

Our Client, part of a large UK government department dealing with judicial process, utilises a major and complex system that integrates with different parts of the judiciary, courts, police and so on. It works to protect the public and reduce reoffending and to provide a more effective, transparent criminal justice system for victims and the public. It is developed, tested and quality assured by meeting the different user needs with reference to relevance, accuracy, timeliness and accessibility.    

Overcoming delays by improving regression pack coverage 

Qualitest inherited a challenging environment of project delays, spiralling overtime costs and under-delivery against requirements. Defect leakage into production was common and needed to be rectified. 

Our Client engaged us to help improve test practices and raise the level of quality throughout the programme.  A key requirement was to develop a test strategy that included the automation of the entire regression pack, an essential part of every sprint release on a weekly basis.  Existing automation frameworks by different vendors were designed on unit test typescript frameworks which was not feasible for UI automation across different browsers. Also, this framework did not support database connectivity because 60% of the regression pack remained without test automation. 

The objective of Qualitest was to develop a new automation framework using Serenity that supports Web UI/API and database connectivity.  

Our Client asked us to: 

  • Propose and implement a new automation framework 
  • Deliver an automation regression pack that covered all business-critical flows. 
  • Increase automation regression coverage from 40% to 90%. 
     

Identifying solutions and developing new automation frameworks for maximum success 

To raise quality in line with stakeholder expectations and restore expectations, Qualitest set out to introduce innovation and best practices. Our collaborative input into sprint planning rationalised targets, enabling the development team to deliver consistently on time and at the level of quality required by our Client. 

One of the key innovations implemented was a new automated test framework, which allowed expansion of both coverage and technical reach while saving time and money. The framework also enabled enhancements to legacy test cases with poor traceability, cutting root cause analysis effort in half and eliminating inconsistencies in execution across lower environments. These actions raised efficiencies in day-to-day operations even more. 

For transparency and client engagement, all regression packs were formally assessed and approved by stakeholders before being implemented. 

The Qualitest framework identification needed to consider: 

  • A solution for developing a robust automation framework using Serenity Java. 
  • A UI automation framework that was able to connect to UI/API and relevant databases. 
  • How to automate 95% of regression coverage using the newly developed framework. 
  • How to accommodate the migration from IAAS to PAAS and services migration to AKS.  Ultimately, the automation framework also helped in connecting both IaaS and PaaS. 

Services Provided 

Rigorous UI automation services that included the use of: 

  • Functional Regression UI Tests 
  • API endpoints tests 
  • DB Connectivity tests 

Key benefits 

  • The improvement of execution timelines from two days to five hours.  
  • A drop from frequent to zero percent of defect leaks into production.  
  • Qualitest increased regression coverage from 30 tests to 250 
  • Automation reliability was increased by more than 85% along with a substantial reduction in data generation time.   
  • The new framework that was implemented ensured that many blockers along with major/minor defects were identified in every release. 
  • Full confidence on release cycles was achieved for our Client.  
  • Automation coverage was increased to 90% and parallel testing introduced in retrospective environments to accelerate testing and reduce the testing window by over 80%. 
quality engineering free assessment Download the PDF