News UK is a part of News Corp, a global, diversified media and information services company focused on creating and distributing authoritative and engaging content to consumers throughout the world. Topics for content include world class news, sport, comment, and opinion pieces. News UK has a wide reach within the United Kingdom; an estimated 72% of all adults in Britain read their publications, and the company employs about 5,000 individuals.
The client needed assistance testing the mobile apps linked to their digital and traditional news publications. While News UK has had a presence in the mobile app stores for both Android and iOS for a while, they were unhappy with their ratings and wanted to improve their customers’ experience. They determined that their unsatisfactory ranking was caused by bugs which had not been detected during testing due to the diversity of firmware and devices being used.
The client wanted to find a way to expand their testing efforts to a broader variety of devices and firmware, providing themselves with a more accurate reading on the end user’s experience with their apps. They already had an in-house testing team of mature test and development engineers which used a rigorous testing process made up of an array of methodologies. However, the struggle was to balance the test coverage which could be provided in-house with the needs of their market. As they did not have the resources available to test the diverse scope of firmware and devices, it was necessary for them to enlist a testing company to assist in their efforts.
Qualitest set the client up with a one of our crowdtesting partners, which employed a crowdsourced testing solution for their mobile apps. By crowd testing, News UK was able to focus on a broader range of firmware and devices working in the real world, instead of within the constraints of working solely in-house. By working with the partner, they were able to effectively mimic the various permutations of firmware and devices which they knew their users would be using but had previously been problematic for testing due to lack of time and resources. The QualiTest team worked on the implementation with the client’s IT team, specifically their in-house testing and development teams.
On top of increasing their app functionality and higher defect detection rates, News UK reported that crowdsourcing increased their efficiency impressively, saving them thousands in testing and overhead costs compared to working with contractors. For example, the client recently released a series of apps for football season, the testing of which required 6 regression cycles running in parallel and used around 180 testers across 6 projects. This required 180 people across 6 projects, but with the use of crowdsourcing saved the client thousands and was completed within a week, with operational cycles complete within 48 hours.
“There is no way,” said Paul Byrom, Head of Quality at News UK, “that we could have released at the quality they did without QualiTest to assist us.
They also experienced almost immediate increases in positive customer feedback in the marketplaces through which their apps are circulated, with higher ratings and better subjective feedback; those apps which were newly-released started off at a higher ranking than previously-released ones.
Representatives of News UK says that going forward they will run all of their apps through thorough crowdtesting before release in addition to their in-house functionality and performance testing due to the extra coverage this process can provide. They are eager to continue experimenting with the partner’s “in the wild” approach, as they find its results to be indispensable when paired with their own in-house lab testing and appreciate the wide variety of feedback and information it can provide. Crowd testing allowed their apps a wider range of real-world considerations like poor carrier coverage for devices both current and outdated, reported by an additional on-demand resource pool populated by individuals all over the world. This extended their internal test teams’ capabilities, increasing productivity while keeping their work fresh and interesting by freeing them of the more tedious aspect of testing.
Using crowdsourcing has also provided a means of measuring the quality of third-party apps News UK acquires; passing them through a final test gives the company confidence that these apps are truly as bug-free as possible before release. In the future, all apps, both proprietary and third-party, will crowd tested as a final precaution before release. This is expected to increase positive ratings in iTunes and the Google Play store, which will lead to higher sales and subscription rates.
