Client overview

The Client is a renowned American multinational retail giant with neighborhood and big-box stores as well as supermarkets around the globe. The Client is known for their commitment to quality, and they require that same commitment from all their suppliers and partners. Sales associates relied on multiple mobile applications on store-specific Android mobile devices, which were managed by individual project/app teams, for operational activities and to monitor productivity.

When speed is the need, automation is the answer

To enhance both productivity and the quality of customer service, the Client approached Qualitest to develop a centralized test automation strategy for continuous testing. The Client had invested in multiple sets of commercial testing and test automation tools but had not been able to use any tool to speed up the test-and-release cycle of these mobile applications in a centralized fashion.

Their existing test vendor was entirely dependent on manual test validations and unable to meet Client’s desire to speed up the test turnaround time.  Also, the store system back-office apps had to be tested inhouse in the Client’s internal network and on Client’s devices only. None of the existing device cloud service providers were facilitating a solution to meet these special security and compliance requirements.

The Client needed a partner who could meet these requirements and help transform their development process into an effective Agile project improvement practice.

Specifically, they needed Qualitest to:

  • Find or implement a fast, cost-effective solution for automated testing of applications on Client mobile devices in multiple locations.
  • Find or implement a solution for managing the cost of the mobile devices to be used in the testing.

Remote control: from scattered to centralized

As a first step, Qualitest performed a four-week, comprehensive proof-of-concept test automation strategy for all the Client’s mobile applications. The strategy demonstrated the capability of an on-premise device cloud lab to execute manual and automated tests on devices and mobile apps remotely, including unattended scheduled/nightly continuous integration.

Then we employed a Solution Architect to implement software for device cloud management and all the Client’s desired capabilities. In a phased manner, we implemented an internal device lab controlled by software and utilities that allowed the following capabilities:

  • Reserve devices for manual test execution and exploratory testing remotely using a web application interface.
  • Remote manual test execution on real devices connected into the system using remote device access via a web application interface.
  • Remote automated test execution on real devices connected to the system using a Client library developed for Java and JS-based automated test frameworks.

The right tools for the right solution

We used a full range of tools to implement a feature-rich device lab to cater to a variety of testing and test automation needs of mobile apps for multiple product teams. A glimpse of tools used and solutions builds are as follows:

  • Appium and Selenium servicers for implementation of JSONWire interface of automated tests. Appium test execution stream for automated test execution through a Continuous Integration (CI) system.
  • Java-Javalin-based microservices for device agents running on machines where physical devices were connected to the system server and JSONWire protocol reverse proxy.
  • JavaScripts and ReactJS for functional web interface for remote device access and interaction on web. This enabled both remote manual functional testing and automated test execution using Selenium and Appium.
  • NodeJS-based web socket event-driven microservices for log storage, events storage, device live stream and device user interactions playback. This enabled recording device and application performance logs, live stream of device view for online test monitoring as well as recorded video for test execution for an offline test result debugging.
  • Springboot for web applications and services to manage the access and reservation mechanism of devices on this device lab, as well as management of devices, availability, and scheduling.

Key benefits

Qualitest’s full-fledged internal device lab enabled managed remote testing and test automation on internal devices in an internal device cloud, with numerous features to achieve Client goals.

Unlike other vendors and service providers, Qualitest was able to achieve these goals within the Client’s special security/compliance constraints.

  • Each project saw upwards of 35-40% in savings in test and release-to-market costs, due to time saved through automated tests and device cost optimization.
  • By the end of the first year, the system provided access to 70 devices, with 2-3 devices being added every month.
  • The scalable, repeatable system ensures continuous improvement in savings and productivity, as well as enabling a higher quality of customer service.

 

quality engineering free assessment

 

Download the PDF