The Client is a UK Ministry of Defense (MOD) contractor, operating under tight security for the provision of facilities management for all MOD properties across the UK.
The Client already had in motion a program that had implemented the new data center, with a virtual server configuration, new and upgraded versions of the existing applications, and was already in daily use for two of its four UK regions. The program consisted of 2 phases.
Phase 1 – the first data migration
Phase 1 was the first migration and was the result of the Client being awarded a new contract. The incumbent for the two new regions had all the data stored on their secure servers, which they did not wish to maintain for longer than contractually necessary. The Client migrated the data to their new “future-proof” servers, and built a new instance of their CRM solution over the top with all the apps that were needed to support their operations. Phase 1 was complete.
The next phase was supposed to start immediately after phase 1, which was to migrate legacy data onto the new Client servers and infrastructure, so that everything was in one place and seamless and clean. However, it was delayed for nearly 4 years. The delay meant that the migration became more challenging because in those intervening 4 years, each data set and each set of infrastructure and apps underwent independent and divergent development, so after 4 years, getting the data structure to match was more of a challenge.
Phase 2 – the transfer of legacy data
This next phase of the program required transfer of the original two larger regions (legacy) data, to the new (target) secure data center location and networks, transforming and merging the legacy data into the target database. Only the new applications/versions would be used, so assurance was needed the migrating data would be migrated fully, integrated effectively, and work seamlessly in the new applications to maintain Business As Usual (BAU).
The project was up against a hard deadline as the existing data center on physical servers was hosted in a building overdue to be decommissioned and demolished, and demolition was already underway:
Phase 2 included extensive data analysis, a considerable data clean-up exercise (redundant data objects and processes), the physical migration, data transformation to fit to the target database and business rules, implementation of all the new applications and application versions, hardware upgrades for field staff, training for new hardware, new applications and new business processes, and a managed cutover weekend directed from a central “war room” and post-migration support.
The Qualitest team worked in a blended team, managing the deliverables of other testing suppliers, and integrating with inhouse and contract resources to deliver the program to schedule.
Due to the nature of the work, all people involved had to be security cleared and on site, and only authorized personnel could remove hardware from the secure location.
Qualitest provided support across all aspects of the migration, deploying a small team led by a senior experienced data migration and data warehousing testing specialist. With no data mapping or system diagrams available, it was down to Qualitest to engineer the quality rather than simply “quality assure”.
The first task was to establish a detailed understanding of the legacy data itself, the locations and structures of that data, and method of transfer, the transformation stages, and the new applications which access that data.
Qualitest worked very closely with the program designers and developers, the Database Administrators (DBA) and the Data Owners, the Business Analysts and the Business SMEs (Subject Matter Experts) to determine the scale and scope of the migration, to document the legacy data schema with primary, secondary and foreign keys, and create the data dictionary, a detailed system architecture diagram, and diagrams documenting the migration process to identify the right validation points.
All this output was reviewed, revised, and approved with all the stakeholders.
The identification of the primary, secondary and foreign keys was critical, as there had been some duplication of primary keys between the target data and the legacy data while they were running in parallel, and many legacy tables were missing foreign keys.
This, the subsequent clean-up and testing to prove data integrity, and the continued operation of the legacy applications — with the removal of the duplicated data and processes — was all required before migration could be attempted. Qualitest’s detailed test plan addressed all aspects of these risks.
The holistic granular test strategy started with detailed data assurance and incorporated robust end-to-end testing to support the complexity introduced by the sweeping scale of the changes, and the criticality of Operations to the success of the Client organization:
Database administrators and business SMEs were brought in to review the target tables and apply hands-on expertise in identifying any missed data clean-up once the data was present in the new tables.
With the number of elements of the program being run simultaneously and the tight timelines, automated reporting of test results and quality progress was a key success factor. Qualitest developed a sophisticated automated dashboard, which provided reporting across all the projects to all the individual timelines and quality markers. This ensured that data was available daily to empower critical program decision making.
Through the meticulous validation of data fields, and data integrity checking covering more than 40,000 records, data integrity was assured with no loss of data and no error records on the day. The migration was executed well within the time parameters set, with War Room activities completed six hours ahead of schedule.
The migration was scheduled to ensure nationwide coverage of SMEs for production validation. Production validation found a total of 13 errors, all non-critical, which were resolved by the war room over the weekend and before operations recommenced on the Monday morning, assuring production stability and zero impact to operations. The migration was carried out exactly on time, with the rollback procedures completely unused – one-hit migration, right first time.
