Client Overview

The client created a smart home monitoring system which includes HD video (with zoom control) and sensor tracking that includes everything from temperature and air quality to vibration, sound, and movement. Controlled entirely by smartphone, it sends alerts when it senses anything out of the ordinary or allows on-demand live monitoring. Over time, the main device learns your home’s rhythms to send even smarter alerts.

Business Needs and Objectives

The client had 2 on-site dedicated test specialists who were doing a wonderful job, but cost too much money for this startup company. Testing goals/objectives included:

  • Complete app testing cycles in 2 weeks
  • Achieve 99% crash-free user sessions for Android and iOS mobile applications​
  • Expand test library (maintained in TestComplete) with coverage for new mobile application features within first 3 days of the start of a test cycle
  • Completely execute “App Checklist” and “Features and Fixes” for each supported OS weekly or twice per test cycle (App Checklist is a regression set of around 55 test cases, Features and Fixes varies between 20 and 70 test cases emphasizing what’s changed in the latest build)
  • Release on time and according to the mobile app schedule:
    • iOS Mobile Application​ (Apple Test Flight​)
    • Android Mobile Application​ (Google Play Store​, .APK file​)
  • Deliver mobile application test candidates on the first Monday of a 2 week test cycle​
  • Accept delivery of app updates mid-test cycle, depending on the severity of any reported issues
  • Include additional hardware​ (managed switch​, configurable router​, the client’s IoT devices​, mobile devices​) in some test plans/cases
  • Include initial installation of the device via Bluetooth wi-fi synch inside the test suite
  • Log all bugs in JIRA for the next daily sync
  • Communication between the client and Qualitest will occur through daily Google hangouts/Skype video syncs and via the client’s Slack #qa-external chat channel

The Qualitest Solution

Qualitest proposed a manual functional testing effort for the client’s mobile app, utilizing right-shoring to manage the expenses (one U.S.-based test engineer and 2 off-shore test engineers). Qualitest worked with the client’s Development and QA Team and with the existing consultant group to fully understand its priorities and objectives, perform complete knowledge transfer and to clearly understand the requirements of the client, leading to a fast ramp up. The test cases were executed on the iOS 7, 8 & 9 platforms, as well as Android KitKat, Marshmallow and Lollipop. We documented all findings and reported all bugs through the client’s tracking system (with screen captures when applicable)

Key Benefits

  • Grew the regression test set
  • Over 99% crash free user sessions for Android and iOS mobile applications was achieved
  • 2-week test cycle was achieved, and the client was happy with the bugs uncovered
  • Right-shoring significantly reduced cost
  • Full satisfaction from client side
  • Achieved full test coverage of different mobile device OS’s, mobile devices and client devices