The current methods of Test Data Management (TDM), production golden copy and custom scripting, are proving inadequate for the speed and quality demands of today. Hence, there’s a pressing need for a paradigm shift in our test data management practices to accommodate modern data requirements. 

 This transformation is particularly crucial as the traditional golden copy-based data refresh proves to be time-consuming and less effective. It’s time to embrace a more agile and efficient approach to ensure our readiness for the future.

Common constraints in traditional TDM

  • Time wasted on querying database to identify required test data conditions. 
  • Concerns about data security when utilizing production data with respect to GDPR and other regulatory requirements, often necessitating waivers and additional safeguards in lower environments. 
  • Handling large datasets poses challenges in terms of transfer and storage capacity. 
  • Lack of support for Agile and DevOps methodologies. 
  • Difficulty in providing test data for Enterprise Commercial off-the-Shelf (COTS) products. 

Beyond these constraints, customers often hesitant to consider transformations due to their significant investments in existing TDM legacy tools, with the entire ecosystem and resources built around them, acting as a key deterrent to considering transformation of this area. 

Navigating these constraints calls for a strategic re-evaluation of our Test Data Management practices to align with contemporary needs and industry best practices. 

At Qualitest, we have developed our next-gen TDM solution to tackle these challenges, enhancing existing TDM operations for the adoption of modern practices and gradually phasing out legacy test data refresh processes.  

The core concept behind next-gen TDM is dismantling the barriers between TDM operations teams and data consumers by introducing a more self-serviceable approach to data search, creation, and provisioning. Additionally, our approach involves a shift from relying on production golden copies to leveraging more AI-infused synthetically generated data for testing purposes. 

Our next-gen TDM solution blueprint

Key aspects of modernization

  • TDM Self Service: It enables data consumer to use self-serviceable portal to independently consume TDM solutions provided by TDM operators. Its value lies in enhancing tester productivity by allowing flexible searches for test data conditions, thereby preventing conflicts with other test data usage. 
  • AI Infused data discovery and generation: Harness the full potential of AI to drive your data discovery and generation process, eliminating the need for manual efforts in executing queries and understanding the dataset. 
  • Synthetic test data generation: Move away from relying on production data for the test environment, leverage synthetic test data generation which is much cheaper and faster to provision.   
  • Containerized test data: With a Microservices architecture, it is convenient to provision ephemeral test datasets to swap it with a new dataset for every test cycle. It opens an opportunity to create separate test data sources for each data consumer when needed. 
  • CI/CD Integration: Integrate the test data solution into the continuous integration and deployment pipeline, rather than operating as a standalone activity. This contributes to enhancing fully automated data provisioning. 
  • Data Virtualization:  Use of data virtualization technique to create VDBs in minutes instead of hours of data-copying activities.  It helps in speeding up data provisioning and reducing storage spending. 

Case Study: Synthetic data for UK insurer

Qualitest has successfully demonstrated its ability to meet the test data requirements for a microservices architecture. We implemented a solution using synthetically generated data for a prominent insurance provider based in the UK.  

Specifically, we collaborated with our partner tool, GenRocket, to create test data for insurance claims. This innovative solution seamlessly integrates with the Continuous Integration (CI) pipeline, offering on-demand test data generation.  

It goes beyond mere generation by intelligently ingesting the necessary data conditions within the system, ensuring a comprehensive and tailored approach to fulfilling the specific needs of the microservices-based architecture for our client in the insurance industry. 

Case Study: Hybrid TDM for US insurer

Qualitest implemented a hybrid test data management solution for a prominent US health insurer. Established within just two weeks, this solution equips them with the necessary capabilities to perform synthetic test data generation in the X12 837P format. Additionally, it offers a generation-based masking solution to replace their conventional masking approach.  

With this solution, we successfully generated numerous sample EDI files within seconds. It enables the customer to leverage reference datasets for creating fully integrated and valid test data for consumption. Furthermore, the solution seamlessly integrates with CI toolsets, allowing the on-demand generation of test data.

quality engineering free assessment