Our client is giffgaff, a new and innovative MVNO. Qualitest was engaged 5 months prior to launch, with a fixed launch date and pre-defined testing budget.
Qualitest were approached by the giffgaff’s Chief Technology Officer to design, implement and manage the entire end-to-end testing solution prior to launching the brand. The end-to-end testing scope included the complete service proposition and infrastructure, including: web technology, back-office environment (including billing), supply chain, customer services interfaces, business processes and the mobile network integration. The core requirement was to provide a clear quality measure which demonstrated that the newly-formed operations had the ability to meet the launch goals, without losing customer/brand confidence during the early months.
Due to the highly competitive nature of the industry Qualitest was required to sign a strict non-disclosure agreement, binding until giffgaff’s launch. This meant working at giffgaff and being diligent in all communications so as not to risk revealing information to anyone external to the project (including internal QualiTest non-project staff and management).
In order to achieve the goals within the given timeframe it was important that a clear and pragmatic test strategy was created. The strategy included the creation of the right team, adaptable enough to manage multiple vendors but also incorporating the required expertise, and a test automation framework that would meet the business goals in both the short and long term. Chris Quintin, one of Qualitest’s consultancy partners, was chosen because of his experience in delivering high-profile programs in the mobile telecom environment and his extensive knowledge of test strategy design and implementation. An entirely risk-based approach was adopted which ensured testing focused on actual business priority, with the ability to adapt based on the constantly changing business objectives and perceived market demands. This approach also included clear communications checkpoints to manage launch expectations.
Following strategy production, Chris brought in additional resources (3 from Qualitest and 1 from an external supplier) with specific functional experience who were tasked with managing individual components of the end-to-end solution, thus encouraging ownership throughout delivery. The solution comprised of deliverables from 7 core suppliers, each delivering to separate delivery plans and timescales, with whom the team worked closely to enforce bespoke quality criteria and maximize confidence prior to integration testing.
The strategy was based around a comprehensive integration plan, developed in conjunction with the delivery plans of the individual suppliers, which ensured integration testing started immediately as functionality became available to enable critical integration defects to be identified early. This strategy required a clear understanding of the solution, the capability of each release and a clear regression testing plan (based on priority).
As development was delayed (with no impact on the overall end date), Chris introduced a crowd testing phase (in conjunction with Qualitest’s crowdsourcing partner), to validate the general usability and functional quality of the website – This was an invaluable exercise as it enabled the core test team to focus on detailed integration activities and provide an early quality measurement of the look and feel and usability from actual users pre-launch. Prior to deployment the test team provided a detailed report outlining the quality of the solution, the outstanding defects and the risks associated with the launch decision. The business was then able to make a confident decision without fear of unknown, untested priority areas
Following a lengthy decision process, giffgaff ‘beta’ hit the market with a number of known issues outstanding. This proved to be the correct decision and initial market feedback (throughout the following month) has been great. Overall release quality has remained good and customer data has been protected which has enabled the organization to meet their business targets and start generating a buzz around the brand. A substantial release roadmap exists for future development during the next year and, following the success of the test approach, Qualitest has been engaged to continue to deliver all testing activities through a managed test service framework.
Qualitest has also been engaged to develop and implement the on-going test organization and look at ways of optimizing efficiency (cost/effort/time) through the implementation of test automation frameworks and off-shore resourcing. Qualitest has formed a strong and strategic relationship with giffgaff which will hopefully continue for a number of years.
“The experience, approach and quality that QualiTest provided for giffgaff meant the successful delivery of a web 2.0 MVNE in less than 6 months. The QualiTest staff worked tirelessly in an agile environment, recognising and addressing additional risks and issues, bringing together many test teams (across multiple suppliers) for an excellent result. “
Nigel Suddell, Chief Technology Officer
