When testing mobile applications, there are many approaches that can be used, including manual testing, test automation, and emulator usage. Emulators can be used for both manual and automated testing, and can be used to emulate devices, browsers, and operating systems. These emulators can provide worth to the testing process; they can be faster and cheaper than using real devices, and are equipped with debugging tools. However, there are also intrinsic issues with emulator utilization.
Even if the testing seems to be successful, you cannot be positive that your data can apply to a real device. It is nearly impossible to assume that the results of an emulator’s device are reliable, and tests that are run on emulators should be double-checked on a real device. Furthermore, if a test fails on the emulator, testers need to decide whether to perform the test on a real device or simply assume that the function does not work and requires correction.
Typically, an emulator is a basic version of a device’s operating system and often does not reflect the specific hardware and software features of each supported device. For example, if Google releases and delivers Android emulator software to Motorola, HTC and Samsung, each of those vendors must manipulate and change the product to make an impact on the market, either in terms of hardware or software. Each vendor offers more appealing hardware features and enhance their software (e.g., faster browser, better launcher, more efficient memory management, etc.), which, in many cases, were never tested against the “basic” version of the software.
NOTE: With companies like Apple, the emulator is generally a more accurate representation of the device, since the software and hardware vendors are the same company.
In terms of network configuration, mobile emulators run on the PC, connect to the LAN and access the Internet via your personal/corporate firewall. Using real handsets, the network is connected to the radio interface and from there to the Internet. This issue implies that your network environment will be different from that of a real device, and will result in your application behaving in another way.
As compared to mobile emulators running on PCs, real handsets have limited resources. Because the mobile handset is a compact platform with limited CPU power and memory, an application running on a PC does not accurately reflect the user experience on your handset, which, in extreme cases, it may not even work on at all. Mobile networks affect application behavior. Since the mobile device is still a phone, network-related events like incoming calls, text messages and interactions with other applications must be tested to determine their impact on your application. Because emulators are not connected to the mobile network, they cannot provide an accurate answer for how these will affect your application.
There are thousands of devices with different popularity levels, which should also be taken into consideration when researching and deciding on your testing approach. In fact, this article reveals that in January 2015, the android was by far the most popularly trending operating system! (As an iOS lover, this was shocking for me) Knowing facts like this can be highly influential in narrowing your focuses and concentrations in testing.
The decisions of using emulators vs. real devices is really based on your risk management approach. A real device provides a more accurate depiction of a real-life situation with your applications, while an emulator has the ability to reduce time and expenses. We recommend that the best methodology is to determine your internal needs and your customer’s needs, and develop a merged testing approach with both emulators and real devices.
