Qualitest Enables World-Leading News Agency to Shift Left and Offer Top-Notch UX

The Client is one of the largest American non-profit news agencies, who has been leading press coverage and used by numerous news and media outlets around the world for over a century.

The Client employs more than 3,000 people globally and is known as the news source of the world’s leading web portals, who rely on the Client for coverage of breaking news.

The growing demand for news, especially during a period of extreme global changes, requires the Client’s to be nimble and adaptable, deliver instant – reliable – frictionless value on scale and offer a better user experience for the countless people who rely on their news cycle.

Millions of Users Demand and Deserve the Best Experience

Due to the dynamic nature of their business, the Client’s testing team had difficulty supporting the changing product needs arising from market fluctuations. Lack of the specialist testing skills and inadequate team sizes in complex multi-vendor development and globally-distributed teams resulted in delayed releases and prolonged testing cycles, putting projects at risk and impacted the revenue.

The Client’s existing automation framework did not count for the rapid pace of development in an Agile setup. This led to a huge backlog of eligible automation test cases in the regression suite over multiple releases, resulting in longer execution times.

Also, the automation framework was not generic so it could not be re-used across different applications, resulting in lower re-usable components and exhaustive maintenance efforts and enhancements.

It also lacked integration functionalities to connect with 3rd party tools, like Jenkins and MS Teams, to achieve continuous testing and get instant reporting.

The Client wanted to:

• Work with a QA partner that understood its industry and business rationale and form a multi-year partnership that would help drive the business value and enhance customer experience by improving quality.
• Implement a “shift-left approach” to its Agile development cycle by implementing continuous testing, optimizing the test automation pyramid and achieving in-sprint automation.
• Create a testing solution for its 60+ internally developed applications that would reduce the testing cycle times, increase cross platform test coverage and gradually decrease costs.
• Establish a “Test Automation Factory” and have a team of automation engineers readily available at any time to automate the backlog of regression test cases and also pick-up new work during peak demand periods.

Ideal UX is Driven by Increased Velocity and Flexibility

Qualitest’s solution was a Managed Testing Service with 85% of the team to maximize cost efficiency, while ensuring governance and alignment of the service.

The service was measured against mutually agreed KPIs (Key Performance Indicators) and with clear SLAs (Service Level Agreements). These were agreed upon after the capability assessment and during the transition period where Qualitest brought in thought leadership by providing a Test Architect and a Transition Manager to ensure a smooth transition of knowledge and services, and to establish the right governance structure.

Reporting structures, formats and frequencies were established early on, so that the service was completely transparent, giving the Client confidence in our progress.

Qualitest leveraged our Rightshoring Testing Model by providing the right blend of onshore and offshore quality engineers to effectively manage the testing activities mapping to the respective time zones, which included:

• Right location: Locating test leadership talent onsite at the Client’s New Jersey offices and creating a test support team at Qualitest’s Bangalore India Test Center. This paved the way for a sun-never-sets delivery model.
• Testing in Agile: The sun-never-sets model, along with proactive collaboration and co-ordination between globally-distributed and dynamic teams of client and vendor resources, delivered a true MTS model with increased velocity for the scrum teams.
• Business Agility: Qualitest teams were able to coach business teams on the User Stories/ Acceptance Criteria to help identify the gaps early in the sprint cycle during the daily transition. Post that, offshore teams designed test scripts, executed them, and prepared defect reports for the onshore teams to address each morning.
• Quality Enablers: Core-Flex resourcing model enabled knowledge preservation with ramp-up and ramp-down flexibility. That also enabled wide range of support with minimum cost to the Client.

Diving Into Optimization

Following transition, Qualitest started to introduce improvements. Beginning with functional testing activities, we reviewed the existing regression test suites, optimizing where applicable and creating additional test cases mapped to the user stories/features using exploratory testing techniques, with quick adaptation to the Client’s agile development process.

After quick dry-run executions and functional conformance, they were categorized into Sanity and Regression by relevancy and then handed over to the automation team to develop the automation scripts.

Test Impact/Root Cause Analysis was performed for each change introduced using Risk Based Testing Techniques and relevant test cases were executed/updated to validate the functional conformance.

This was the continuous process followed to build the automation test bed and became very effective during hotfix or rollback testing; this paved the way for in-sprint automation and uninterrupted automation execution using CI/CD tools. To date, functional testers have been able to create around 10,263 regression test cases out of which 5,800 test cases are automated and executed successfully.

To progress automation, Qualitest started with the applications using an in-house developed hybrid framework, our bespoke automation framework, which involved GUI, backend, and API. The framework follows best practice design architecture and element locator strategies to handle the application changes for each release and reduce the maintenance of automation scripts drastically. The results from this impressed the Client, who gave us the green light to further expand the automation activities across multiple platforms.

The Automation Framework was customized to address the crucial data validation challenges presented by the Client’s applications and business processes, by developing utilities. This increased the efficiency of data comparison tests between legacy and new data providers which involved testing of thousands of records. This also ensured comprehensive volume coverage within a very limited timeframe.

To perform distributed and remote test execution across multiple browsers, Selenium Grid was implemented, which contributed to increased test coverage in reduced time.

To reduce the production slippages that the Client had been experiencing, Qualitest proposed a Shift-Left approach to their agile development life cycles. By aligning with the development teams in a close-collaborative model, whether Client’s in-house resources or vendor development teams, QA was able to get involved in testing much earlier – utilizing Jenkins to execute testing earlier. The resulting quality increases allowed accelerated delivery as well as decreased production incidents.

Integrated automated tests into the CI pipelines are triggered at a regular cadence and whenever a build is deployed, the test suites are executed, and notifications sent out to all the stakeholders about the execution results/stats. This helps identify the application issues early in the software development cycle.