Client Overview

Quam is a new mobile brand launched by Telefónica for the youth market. In support of Telefónica’s mission to deliver the best of technology from around the world to their customers in Argentina, the company aims to simplify their product offerings to across the board, including their fixed line, mobile, broadband, and television products.

QUAM’s launch is strategic to Telefónica’s competitive position in Argentina; the new brand targets young people between 15 and 25 years, a market of about 8 million locally. QUAM differentiates itself from competitors by offering simple packages with attractive prices. Because QUAM’s target market is young adults, the brand will focus mainly on attending customers via online channels, such as e-mail and those targeting communities, while also putting an emphasis on self-service options.

Business Needs and Objectives

Qualitest’s background in the telecomm industry, as well as our extensive experience delivering testing services to MVNOs such as giffgaff in the UK, led Telefónica to contact us on Halloween. In addition, Qualitest’s ability to implement and leverage testing solutions such as crowdsourcing was considered valuable to Telefónica in their delivery of their new Argentina-based brand.

A software testing solution was needed to ensure the mobile service launched smoothly and met a range of project requirements for quality, location of testing and speed.  The launch was planned for the start of December that same year to be ready for the Christmas period, over which time a large marketing campaign was scheduled to run. Time was of the essence; less than 2 months from first contact!

The Qualitest Solution

After quickly assessing the requirements of Quam Argentina, Qualitest recommended a crowd-based approach for local resourcing, facilitated and managed by a Qualitest Senior Test Manager, who would be responsible for the test products and coaching the local project team. Jim Brooks, VP of Solution Delivery, flew to Buenos Aires that weekend to kick off the testing effort.

Qualitest created a test approach and plan based on our unique MVNO knowledge to guide the 28 testers from the crowdsourcing community who were identified to participate in a blend of test case execution and functional exploratory testing. These testers were chosen as they fit the Quam target demographic of 18-25 years old and had the skills and access to the technologies required. This capability was ramped up and ready to begin testing within the week.

With Qualitest’s expertise, Quam were able to control the testing effort through the crowdsourcing platform, and supplement the community testing with product managers in the project team, checking back end systems and processes. Together, these would achieve a huge level of testing coverage in a short period of time.

During this phase of test execution a total of 346 defects were identified across 6 cycles of testing, averaging 57 defects per cycle. Overall, the defect approval rate was 86% with 296 defects being approved by Quam over a period of 38 days. This rate experienced a significant increase in later cycles, from 72% in the first cycle to as high as 98% in the cycles that followed as the testers built up their relationship with Quam. The testing was focused mainly on desktop browsers, but also had a secondary focus on mobile browsers.

Key Benefits

  • Speed and Responsiveness – The testing capability was ramped up from zero to 28 local Argentinian testers in the space of a week, plus the senior test manager and crowdsourcing project manager
  • Quality – Defects were of exceptional quality, including screenshots and video where necessary, to a point where only 13% of defects were rejected for any reason. This significantly reduced the “noise” and enabled development teams to focus on resolution of the priority defects
  • Versatility – Approaching the testing as end users and triggering all the in-life processes and user journeys enabled us to minimize formal test case creation and allow testers to begin testing more quickly. Furthermore, taking advantage of the existing community enabled the project to make the most of the time available through testing over weekends and evenings – at no additional cost
  • Cross browser coverage – Although focused on desktop browsers for launch, testing was able to further increase confidence through coverage of a broad range of devices and browsers not available in the lab
  • On-time – Testing completed with a high-quality product in time for the Christmas launch, leading on to an excellent customer take up
  • Fixed cost – The test capability and expertise was provided at a fixed cost, enabling the budget to be managed easily
  • In the following year, Telefónica was able to launch similar offerings in Peru and Mexico.

In addition, some of the testers were so impressed by the end product that they switched providers to Quam following the testing which stands testament to the success of the project and the testing effort.

Working with Qualitest enabled us to launch our mobile brand in a very short period of time, with the confidence that it was a reliable and high-quality product.

– Patricio Lobos, Quam Director in Argentina