Recently Qualitest hosted a Meetup, ‘Crowd Catcher: How to take control of the Crowd’, at One Alfred Place, London. We were there to share our experience of harnessing the power of the crowd and how to best integrate it into the development cycle. Joined by industry experts from Camelot Global and Applause, we were able to deliver a well-rounded session around Crowd Testing. 
With the explosion of mobile and digital application usage, the world has changed. The evolution of all industries moving towards being more digital provides a lot of opportunities, yet similarly this comes with many challenges. Users want broader access and availability, faster time to market and are less tolerant of imperfect quality. Enterprises are being faced with challenges including; device fragmentation, cost, building and maintaining up to date device suits and understanding their users.
These challenges are being conquered daily by Qualitest’s client Camelot Global. Camelot Global is a provider of lottery and technology services to Lotteries around the world, with Camelot UK, the operator of the UK National Lottery, their sister company being their biggest customer.
Biraj Nakarja, Head of Global Test Services, for Camelot Global presented Mobile First, Crowd Second? He discussed how Camelot Global embed crowd testing into their overall testing strategy to help with the ever changing and fragmented mobile world.
Biraj reflected how living in a mobile first world, people should be thinking about the use of crowd second, given the fragmentation challenges that the mobile world brings. Since becoming more tech focused with the release of the iOS app and Android coming soon, quality and integrity have been imperative. He suggested that everything needs to be a ‘Rolls Royce’ solution before it goes into production, yet this can cause challenges as it frequently means a lengthy lead time.
Biraj explained that with careful planning and utilizing the crowd at the right time and with the crowd partner, they have successfully used crowd to complement Camelot Global’s internal test team. Embedding crowd testing into their overall test strategy, Camelot Global have been able to achieve a number of benefits. These benefits are more than just identifying high value defects but also for real user feedback and ratings. Additionally, Camelot Global are able to achieve the full device coverage required, save costs of devices and test cycles, build confidence in the product, the ability to test and learn before going live, testing out of hours and being able to react quickly to any changes from player feedback etc. Interestingly, 47% of interactive sales are on mobile and tablet devices, therefore the investment in mobile will continue.
Applause’ Sanji Alwis, VP of Business Development, continued with From Acceptance To Delight, Crowd Testing Apps which provided examples of how crowdsourced professional testing is helping Qualitest and Applause improve digital experiences for their customers by replicating real world use case, demographic, geographic and device fragmentation.
The apps economy has changed the way users consume content and connect with companies. ‘Apps are essentially the front door to the brand’ and every new app must ‘delight’ in order to differentiate. Poor app quality leads to opportunities for competitors to hijack your customers.
‘Perception is reality’ and the key value with this crowd testing method is the ability to replicate what users experience, i.e. real world testing, which has become a key value for companies. The users essentially dictate a company’s success making it imperative, in this digital world, that business move their focus from requirements and acceptance to delight.
Applause’s crowd community comprises of only professional testers, and allows data capture per issue/cycle, profiles specified for geographies, devices and OS and also gives the visibility into the history of the professional crowd, even down to the defect level. Scoping out the project with the right partner like Qualitest and our managed crowd testing service, ‘in the wild,’ testing services enables real user feedback and coverage where your users work, live and play. Crowd testing in essence allows testing beyond the test lab, across various devices and aims to compliment in the lab and outsourced testing. All of these elements are what can drive business quality, velocity and scale.
Qualitest’s Aviram Shotten, VP of Professional Services, UX Testing using Functional Crowd Testing rounded off the session and shared the unique way in which Qualitest leverage the crowd in order to evolve from functional testing to functional usability testing and how this allows our clients to have continuous feedback around how apps and features are perceived by end users.
Qualitest provide a managed crowd testing service. A dedicated project manager works with our customers to harness the crowd for a number of verticals and are able to adapt the crowd to customers ever changing requirements. Using the power of the crowd for front end testing achieves Functional Exploratory testing by the crowd, not by your testers. Using scripted testing you can specify almost unlimited demographics and geographies. Managed crowd testing provides a single point of contact to manage the crowd, best leveraging the testing best practices while assisting with the end to end aspects such as planning, design and reporting. This method helps unobtrusively embed the service into existing WoW for both agile or waterfall methodology, progressing on demand testing to on demand UX feedback. Through Functional testing with UX focus, we are able to see if the product is able to cope under stress as well as getting user feedback of how the application performs live.
Crowd testing can be implemented through all stages of the development cycle and as a result we are able to cover 80 – 100% of testing that you need, when you need it. With the use of the managed service and Applause’s platform, you are able to target crowd demographics for analysis. This enables Qualitest to incorporate the process seamlessly into agile practices. Through Applause’s analytics we are able to analyze industry competition, focus on areas where competition thrives, provide secondary validation of results and track the competition’s positive and negative reviews in real time.
By leveraging the crowd with your mobile application, you are able to improve the entire experience, maximize conversion and lower the abandonment rate. Crowd will assist with finding high value defects and improving business functionality. Test beyond requirements and acceptance for delight and importantly, use crowd testing to complement existing testing processes.
The combined experience of our speakers gave everyone the opportunity to learn and raise questions about each stage of the crowd testing process, from Applause’s platform, Qualitest’s managed service which takes the pain out of integrating crowd testing into your development process to Camelot Global’s first-hand experience of how they have used crowd testing to drive down costs and speed up testing.
To view the webinar and to see the presentation slide deck click here.
