Three Ways to Test Mobile Applications

There are three ways to test mobile applications.  Not all three ways may, or should, be required when testing an application.  However, choosing not to use one could carry potential risks.  If the application and context are not properly assessed as to what types of testing should take place, critical defects could go undiscovered during the development and testing phases. Keep in mind that all three ways can be performed using manual or automated testing techniques.

The first way to test a mobile application is by using an emulator.  An emulator is a computer program or device that imitates the behavior of another program or device.  For mobile applications, most emulators are computer programs that imitate another program under development or test.

Pros: The emulator accurately imitates the behavior of the program under test including all functions and disruptions that occur on the selected device (e.g. incoming call, switching applications, behavior when buttons available on the device are pressed, etc.). The emulator can also imitate the device under numerous different types of configuration and user settings.

Cons: The emulator cannot imitate the network that the device will be on (including different physical locations).  Depending of context, this way of testing a mobile application may not be appropriate, especially if the application uses GPS in any way. The emulator also doesn’t provide the user with any hardware settings. Emulators vary in how accurately they imitate the device depending on the operation system or the device.

The second way to test a mobile application is by using a real device tethered to a network.  During this type of testing the device is controlled by the user at the workstation and typically involves third party testing tools to invoke different tests on-demand.

Pros: On-demand execution of test scripts.  Actual device responses can be observed instead of emulator responses. The device accurately imitates the behavior of the program under test including all functions and disruptions that occur on the selected device. The device can be configured to accommodate for specific testing scenarios and user settings.

Cons: Configuration of the device has to be done manually and can become costly and tedious.

The third way to test a mobile application is by testing it on a real device on a real network in real locations.  During this type of testing the tester gets to experience the application as the user would and can test anywhere the network allows.

Pros: Depending on the application, GPS-related functionality can be tested since the tester can change locations. The tester gets to experience real device responses instead of emulator responses.

Cons: Each network varies in performance and network outages becomes a risk.  The tester has to travel to different locations if the application uses GPS, which could be a pro or a con depending on where you live or where you travel.  This type of testing is very tedious, costly, and requires manually changing every hardware setting and every user setting.  Using a third party testing tool also becomes tedious if not impossible. Therefore, capturing test results could become very challenging.  This type of testing is and should be combined with one of the other types of testing mentioned above.

As mentioned earlier, a combination of these would probably most appropriate for most mobile applications.  Context should always be considered on any project, mobile or not.  Successful projects are a result of good planning, so take the time you need up front to determine what types of testing that are best suited for your application to cut project costs.