Global Elevator Manufacturer Slashes Regression Testing Time from 24 Weeks to Under 3 Days
How Qualitest’s automation expertise supported a fast-paced development cycle and new regional deployments to speed up a multinational legacy application re-engineering project.
Manual regression testing was taking six people four weeks to execute, failing a two-week software deployment cycle.
Software development was centralized but software testing was performed independently by each regional business unit.
We designed and implemented automated regression testing to fit the two-week deployment cycle and the demands of new markets.
We created a test automation framework to support new application technologies and accommodate regional differences.
Full regression testing can now be executed within 2-3 days for each new application version.
Test suites are fully supported across the Client’s US and international markets.
The Client is a global elevator manufacturer with business units on six continents and more than 50,000 employees. Achieving sales of around €8 billion in the fiscal year 2020/21, its products range from standard elevators for residential and commercial buildings to customized systems for modern skyscrapers.
The firm also makes escalators, moving walks, passenger boarding bridges, and stair and platform lifts. Its IoT predictive maintenance services are also growing, giving customers improved elevator and escalator performance and longer product lifespans.
Banishing delays and wasted testing effort
The Client was undertaking a major project to re-engineer its legacy applications onto modern platforms. This called for functional testing followed by full regression testing for each new business market. But it was proving tricky to execute.
The Client’s software development was concentrated in its Asia-Pacific business group. This operated independently and used an iterative process that delivered new versions of applications ready for testing every two weeks. But each regression test cycle took six business analysts a full four weeks to execute by hand. So, either a full regression test wasn’t possible or there would be a delay in testing and accepting new functionality. This wasted testing effort.
The Client wanted a global test automation solution that would meet its specific objectives to:
- Manage its major application re-engineering project already in progress.
- Address the needs of its development group, who delivered new application versions independent of testing group participation.
- Assure application platforms linked in an end-to-end framework.
- Speed up Agile release schedules.
- Accommodate regression testing, which currently exceeded the time available.
- Support its global user base with regional application and database differences.
- Meet a corporate directive to decrease application licensing costs.
Finally, our Client needed more stability and flexibility in its mobile app testing. It wanted an automated solution that would work with its current setup and then transition to a mobile cloud provider to scale and support new device models.
Building a fast and flexible global test automation framework
Resolving each of these issues required two elements:
- A test automation framework to support the new application technologies.
- Automated regression testing that would cut testing times to fit the two-week deployment cycle but still support new markets and different countries with no need for code changes.
So, Qualitest implemented a staged approach with three clear, well-defined phases.
Phase 1: identifying the right framework
The assessment process to develop the recommended solution was very straightforward.
First, we spoke to our Client’s subject matter experts and stakeholders to learn about their goals and challenges. They walked us through the applications in scope for testing. Next, we reviewed the manual testing process covering test design, execution, metrics and reporting. Then we reviewed the existing testing tools and any existing test automation. We also reviewed existing test cases to assess clarity, condition and data needs. Finally, we talked to the testing team to assess members’ skill sets and roles.
Initial tool environment
Because of the diverse portfolio of applications and platforms involved we employed a best-of-breed test tool selection approach. This balanced choosing the best test tool for each application, ensuring the best combination of automation features and driving down license costs.
As a result, our consultants and engineers recommended a blend of both off-the-shelf commercial tools and open-source tools for the new test automation framework.
“We designed automated regression testing to fit the two-week deployment cycle and the demands of new international markets.”
Phase 2: developing an automation framework
Major changes in the testing environment were guaranteed. So, we designed a test automation framework that would flex and scale for our Client.
We also ensured that the framework would support the test environments and data of our Client’s international business units – without the need for test script modifications.
To improve testing, we trained the manual testing team to use the existing test management tool, which they’d shied away from using in the past.
We ensured that test data was identified or created at runtime wherever possible. This accommodated dynamic data from production refreshes, extension into new markets and international testing support.
We also ensured that the required test environment, country and market were specified in one location. That way, test execution could be instantly configured for the correct test target with no change to any test script code or data.
Mobile device platforms
Sauce Labs was selected to build a private device cloud that reflected the mobile devices of current users. This choice also allowed the use of the public device cloud to validate deployment against future models.
To make it easy to deploy current and future DevOps tools, we ensured test execution could be triggered from within the testing development tool – through a command-line interface as part of a scheduled task or CI/CD pipeline or through an integrated application lifecycle management tool.
The existing decentralized testing and reporting process was a challenge. Our first improvement was a central SharePoint site for posting testing progress, results and documentation. Next, we ran training classes to prepare for moving to a new centralized test management tool with a lower licensing price tag.
Phase 3: migrating to a new test management tool
We replaced the existing test management tool with an application lifecycle management tool. Why? Because it supported open source, Micro Focus UFT and Tricentis Tosca automation tools in one unified platform.
This tool also integrated with CI/CD-pipeline tools such as Jenkins. Plus, it offered migration tools and services to maintain existing testing assets.
“Other Client-side project managers now want to implement test automation for their projects and realize the same proven benefits.”
Qualitest’s automation engineers met all the Client’s stated goals:
- Full regression testing can now be executed within 2-3 days for each new application version to support the rapid development cycle.
- Freed up 24 weeks’ worth of manual regression testing effort for each release.
- Quick turnaround response on the stability of a new application version for acceptance by the manual functional testing team.
- Able to validate multiple application versions simultaneously to support informed decisions on which version to accept into testing.
- Able to restart regression testing for a new application version with no investment or loss of manual testing hours.
- Test automation is run against multiple test environments and databases to verify new environments and prove the approach to support new regional deployments.
- Supports US-based and international regression test suites.
- Made license cost savings from migration to a new tool.
- Identified and captured defect information for intermittent issues found during manual testing that were hard to reproduce.
News of the success of the test automation framework has spread. Other Client-side project managers now want to implement test automation for their projects and realize the same proven benefits.download the pdf