Insights Blog Search Engine Giant Lifts Quality, Speed and UX in 31 Countries with Expert Multilingual Solution

Case Study

Search Engine Giant Lifts Quality, Speed and UX in 31 Countries with Expert Multilingual Solution

To elevate overall search quality as well as support its search-based products and create seamless integrations between them in more than 30 different countries, the Client needed a scalable global program.

Overview
Challenges

The Client urgently needed to improve search quality and speed for speakers of 45 languages in 31 countries.

Poor quality and slow reaction times were creating a negative user experience.

Solutions

Qualitest deployed highly experienced engineering teams who were also skilled in local languages and dialects.

The teams developed automatic tools and mechanisms to improve quality and speed and enable scalability.

Results

Qualitest teams achieved a 50% improvement in search quality, resulting in increased users and ad revenues.

Search processes run time was reduced by 20-30% and projects were completed 6-8 months ahead of plan due to automation.

Client overview

The Client is a multinational technological company and a top provider of search and search-based products, such as voice-activated devices that can search news, music, audio, video and other categories.

With social turmoil, the COVID pandemic and the accelerating pace of world events, more people constantly seek information online in real time. Leading websites rely on the Client not only for general search topics but also for breaking news, progression of events, market trends and other timely information. This requires the Client to be agile and quick, able to scale and deliver flawless performance across numerous devices, locations, scenarios and languages.

Bringing search up to speed across borders

When the Client approached Qualitest, search maturity in English was considered high. However, in many other languages, as well as the dialects and variations of those languages in different countries (for example, French in France vs. French in Canada), the maturity was considered significantly lower.

To elevate overall search quality as well as support its search-based products and create seamless integrations between them in more than 30 different countries, the Client needed a scalable global program. Because search and search-based product capabilities drive a large portion of the Client’s revenues, there was an urgent need to establish such a program quickly and be able to scale it significantly year over year.

To undertake this massive program, the Client wanted a Quality Engineering partner who:

  • Understood the Client’s industry and business rationale, with the eventual goal of a multi-year engagement.
  • Understood the current production usage and metrics.
  • Could support a total of 45 different languages across 30+ locales.
  • Was able to self-manage the program and provide an SLA to hundreds of concurrent projects.
  • Provide global coverage via a 24/7 model.
  • Maintain a model with the ability to be flexible based on changing needs.

A three-phased project built to scale

Qualitest implemented an iterative process model, with three clear, well-defined phases.

Phase 1: Proof of concept with multiple teams across 3 global locations (One location each in the US, Europe and Asia)

To identify the right framework, Qualitest engineers/team leads undertook enterprise-level deep dives and fact-finding excursions into the Client’s existing production systems, documenting findings carefully to create a preliminary knowledge bank that could be used later as a training repository.

The Qualitest teams gained a detailed understanding of the unique logic, tools and ways of use that characterize search.  During this phase, our teams began delivering projects across 6 countries supporting 8 languages and establishing a mutually agreed-upon SLA for the program as a whole.

Our engineers also took it upon themselves to master the understanding of some of the integrations between the search engine and the search-related home products and their release procedures.

Phase 2: Scaling to support all countries, languages and technology stacks involved

With this firm foundation in place, Qualitest began the scale-up, creating an operational engine to:

  • Scale from three teams to 40 teams across 6 geographical locations (more than 400 quality engineers and software developers in test).
  • Support existing search programs and gradually add programs to the support structure
  • Develop more and more automatic tools that would enable the teams to improve from project to project.
  • Define a set of quality metrics that would help in continuous improvement.

The outburst of COVID during this phase brought two unanticipated challenges: the “Great Resignation,” a global trend in which employees were leaving their jobs, and a significant shift in the way people used search and search products, which necessitated rethinking many of the program’s basic assumptions.

Through Qualitest’s internal rotation program and close collaboration with the Client, we were able to retain the vast majority of engineers and successfully scale the program up. Furthermore, the program met more than 90% of its deliverables, hence delivering to its SLA despite some changes across the team.

Phase 3: Steady state project support

Currently more than 80% of the projects are in steady state, while the teams are showing continuous improvement month after month. The program is delivering from five different locations around the world, providing global support to the Client’s programs. The teams that have reached this stage focus on improving a Qualitest delivery model, known as 3-I:

  • Intelligent Optimization
  • Integrated Coverage
  • In-Sprint Agility

Today, the program as a whole is able to develop hundreds of new tools/processes a month and support thousands of incoming projects a year.

Ultimately this program has elevated the quality of search of many different languages by more than 50% over two years. It has also added a lot of different functionalities that navigated the way the world uses search today, satisfying and attracting many more users and resulting in extreme levels of added revenue for our Client.

Key benefits

The Qualitest team of engineers not only met all the Client’s stated goals within the allotted time frame, but also exceeded many of their targets.

  • Supported existing search operations with close to no new failures, while scaling many new programs.
  • Scaled the program successfully from three to more than 40 teams, allowing support from approximately 20 projects a year to more than 1000.
  • Improved a variety of processes through smart automation, reducing about 80% of cycle times with no compromise to ROI.
  • Deployed 30% more improvements than originally planned.

 

quality engineering free assessment

 

download the pdf