Qualitest Enables DevOps through Test Automation for Leading News Agency
The client contacted Qualitest to help drive test process improvement and dramatically increase test automation coverage. Qualitest and the Client decided on a Results Based Testing (RBT) approach, with Key Performance Indicators (KPI) and a contractual Service Level Agreement (SLA).
The client is a multinational non-profit news agency, owned by newspapers, radio and television stations who contribute news to the client. The client’s multi-topic structure has resulted in leading web portals relying on the client as their first source for news coverage of breaking news, with constant updates of evolving stories.
Business Needs and Objectives
The goals of the client included the following:
- Establish a QA partner who understood their industry and business rationale, with the purpose of launching a multi-year engagement
- Implementation of a “shift-left approach” to their agile development cycle, with Continuous Integration and release capabilities, with a rapid transition to DevOps methodology
- Achieving an optimal level of automation for their 60+ internally developed applications
- Creating a testing solution that would decrease prices while increasing test coverage with the best tools and resources for the job
- Establishing a “Test Automation Factory” environment to rapidly perform Knowledge Transfer and ramp up the automated test efforts
Qualitest and the client decided on an Outcome-Based Testing (OBT) approach, with Key Performance Indicators (KPI) and a contractual Service Level Agreement (SLA). After performing tool comparisons, QualiTest recommended and implemented Selenium for writing the automation framework for web applications, and Microsoft Team Foundation Server and Microsoft Test Manager for source control and test management.
The Qualitest Solution
Before engagement with the client started, a large part of the partner selection process was involved with ensuring that Qualitest could fully identify the client’s key business processes and requirements, comprehend their business and infrastructure and assist in configuring an agile, fast-paced testing environment.
The client initiated a structured program, involving the mapping and identification of 60+ applications, with a champion from the client side assigned to each application. After the presentation of each of these applications, Qualitest was required to reciprocate with their own interpretation of each application, with the intention of ensuring that Qualitest possessed a deep understanding of the client’s business goals and technology.
The first three months of the engagement were involved with Knowledge Transfer; after these 3 months, the SLA went into effect.
The Knowledge Transfer was handled in a “2-in-a-box” approach, where a QualiTest employee and a client employee worked together closely to ensure understanding. A primary Qualitest resource was assigned to each application to learn from the client-side champion. The 3 months of Knowledge Transfer were structured for each application as follows:
- In Month 1, the client performed testing and execution of test scripts, while QualiTest watched and learned
- In Month 2, QualiTest performed testing and execution of test scripts, while the client supervised and guided
- In Month 3, the client’s supervision was ramped down to minimal involvement, with guidance as needed
Leveraging Qualitest’s Right-Shore Testing model, test leadership resources were located on-site at the client’s New Jersey offices, and supported by a test team at the Qualitest India site. There are many parallel working hours between a US-based Development/Test team and a test team located in India in order to maximize collaboration of the scrum team.
Some of the key functions performed by the Qualitest engineers include:
- Identification and development of an in-depth understanding about the client’s applications and system architecture for the purpose of application test automation
- Development of a scalable, maintainable and user-friendly automation framework.
- Reduction in the turnaround time of regression suite execution across dozens of applications; in many cases this reduced the execution time from weeks to days or hours.
- Creation of scripts and scenarios to produce consistent, measurable and repeatable regression tests. These tests and scenarios were designed to mirror the client’s live production environment, business patterns and throughput, including projections for future growth when applicable
- Dynamic performance testing with rapid ramp up and down through the off-shore model
- SQL migration testing with rapid ramp up and down capabilities
Some benefits that have resulted from Qualitest’s on-going Managed Testing Services include:
- Achieving cost savings of an estimated 42% after three months of engagement
- Gaps and risks are identified early in the process, reducing bugs at later stages and avoiding project delays
- The client was able to implement a “shift-left” approach to their Agile development cycle; because the testing was implemented and completed within an earlier time frame, Development and QA departments were able to interact from an earlier stage and applications were entered into “go-live” production in a more expedient manner
- The client received expedited test automation implementation at a fixed price service, resulting in lower prices