Client overview
Our Client was a train operating company in the United Kingdom, which operated the InterCity West Coast franchise from 9 March 1997 to 7 December 2019. The franchise covered long-distance passenger services on the West Coast Main Line between London, the West Midlands, North West England, North Wales and southern Scotland, consequently connecting six of the UK’s largest cities: London, Birmingham, Manchester, Liverpool, Glasgow and Edinburgh, which have a combined metropolitan population of over 18 million. It had around 3,400 employees in 2015.
Their brand was also used on the legally and operationally separate East Coast Trains mainline from 2015 until 2018, and previously on Cross Country Trains, which operated between 1997 and 2007, as well as from 2018 to 2020 in Florida, USA.
Next stop: app innovation station
Our Client’s train ticketing app was newly developed to help facilitate members planning and booking UK train tickets using a debit or credit card, and receive credit points for each purchase related to the transaction value. This functionality was built into an app which was available on iOS and Android.
The objective was to test an MVP product ensuring the product’s functionality, reliability and user satisfaction within the constraints of rapid development. The app was inconsistent and had weaknesses from the beginning.
Some of the challenges our Client came up against included:
- Limited teatures: testing the core functionalities in an MVP might be straightforward, but ensuring that they work seamlessly and meet quality standards is essential.
- Unclear requirements: in the early stages, requirements were not well-defined, making it challenging to establish comprehensive test cases and criteria.
- Rapid changes: frequent iterations and updates posed a challenge for quality engineering teams to keep up with changes and conduct thorough testing within tight timelines.
- Balancing speed and quality: sometimes it used to happen to release quickly, and this can sometimes compromise the depth of testing, potentially leading to overlooked issues.
- User feedback integration: incorporating user feedback rapidly and efficiently into the QA process was a challenging, especially when timelines are tight.
- Scaling for growth: as the MVP evolves, ensuring that the QA process can scale to meet the demands of a growing user base, and expanded features becomes crucial.
Addressing these challenges required a strategic approach to quality assurance that considered the unique characteristics and goals of an MVP while maintaining a focus on delivering a reliable and high-quality product.
Choo-choo-choose the best: how the app got a first-class upgrade
Understanding the needs of both the business and the end user POV, several types of functionality testing were deployed including accessibility testing and backend testing. Project requirements were defined, providing comprehensive information about the level of quality of our Client’s product.
There were two distinct phases in the testing process including:
Phase 1 – deploying a MVP product which was crucial to identify and fix potential issues, ensuring the product met basic quality standards
This phase involved thorough testing to ensure that the basic features worked as intended, addressing potential issues, and gathering user feedback. This iterative testing process helped to refine the product, improve its usability, and set the foundation for future enhancements based on user insight.
Phase 2 – a focus on validating new features, assessing their integration with existing ones, and ensuring overall system stability
The focus then shifted to validating new features and functionalities introduced after the initial MVP. This phase involved comprehensive testing to ensure the seamless integration of new components with existing ones, checking for any regressions, and verifying that the product meets quality standards across various testing aspects like functionality, performance, security, and user experience.
In testing the various sources and third-party companies underpinning the ticketing Product, this included:
1. Silver rail – for the backend solution
2. Fast rail ticketing – provide a refunds web form for tickets that can be automatically refunded
3. Braintree – payment method4.Eagle Eye – who will be providing the loyalty points solution
Phase 3 – Accessibility testing to ensure they are inclusive and usable by a diverse range of users
Accessibility testing was conducted following WCAG guidelines, with early accessibility testing increasing user reach. Accessibility-friendly websites rank higher in SEO, driving more traffic and conversions while also offering simpler interfaces, reducing bounce rates and increasing user engagement.
Several solutions for the MVP (Minimum Viable Product) software applications were provided including:
- Early issue detection: issues were detected in the early stages of development, preventing the accumulation of critical bugs and ensuring a more stable MVP.
- Continuous testing: continuous testing processes were implemented so that as new features were added, existing functionalities were not compromised, maintaining overall product quality.
- Risk mitigation: quality assurance strategies on identifying and mitigating potential risks associated with the MVP were deployed, helping to create a more resilient and reliable product.
- Clear requirements definition: clear and comprehensive requirements were established, creating a foundation for effective test planning and execution.
By integrating quality engineering into the development process for an MVP, a more robust and reliable product was delivered for our Client which reduced the likelihood of critical issues and enhancing the overall user experience.
Key benefits
- Executed more than 3500 test cases for Regression for 45 releases.
- Created a regression pack of 80 test cases that helped to maintain the quality, stability and reliability of software products across releases and updates.
- Identified more than 600 defects during the development and testing phase which saves time and resources by preventing the need for extensive rework.
- Implementation of Xray a centralized repository for test cases, eliminating the need for scattered documents or spreadsheets. This helped with accessing, updating, and reusing test cases, saving time on manual document management and version control.
In addition, our Client benefitted from:
- Bug identification: we helped to identify and address bugs and issues early in the development process which reduced the likelihood of critical errors in the final product.
- Enhanced product quality: rigorous testing ensured that the software meets quality standards, resulting in a reliable and high-performance product.
- User satisfaction: a positive user experience was created by ensuring that the software functioned as expected, leading to increased customer satisfaction.
- Cost savings: issues were identified and fixed early in the development process which was more cost-effective than addressing them post-release, saving time and resources.
- A consistent performance: consistent performance was assured across different devices, browsers, and operating systems, enhancing the software’s reliability.
- Early issue resolution: the identification and resolution of issues in the early stages of development was accelerated, preventing them from escalating into more significant problems.
Download the PDF
