Mobile Testing Best Practice

In this article, we will discuss some best practices to be applied when developing and performing Quality Assurance of Mobile Applications. While many of the same practices of Software Quality Assurance apply to performing Quality Assurance on mobile applications, the reality is that mobile applications are quite different in some critical ways, which requires some special considerations and attention.

Mobile application testing is a new technique in the field of testing. Mobile app testing best practices involve testing of mobile applications across various platforms with the intent of delivering quality applications.

I Will start with browser & device matrix first.

Browser/Device Matrix

Deciding on browser and device matrix should ideally occur in the early stage of the process that is at requirement gathering. Of course, much depends on the nature of the application being developed, but below are some general criteria for deciding which matrix should be considered.

Browser capability: If the application being developed will be dynamic or interactive, then the browser must support the following:

JavaScript Support: Mobile browsers must support JavaScript in order to build an interactive application UI.

XML HTTP Request Object: XML HTTP request object support is required to communicate with the back-end server and to update the page with new data without reloading the page. This will give the user a smooth browsing experience for the site.

CSS Support: CSS defines how page elements are to be displayed and enables you to change appearance and layout of all of the pages by editing a single file.

The key point that needs to be considered in browser & device matrix:

There should also be performance analysis of the browser. If the browser performance is poor, then it should not be a part of the rich mobile application matrix that is going to be developed.

Browser matrix, of course, is subject to change. This can depend on changes in technology, including the growing popularity of the browser. In this case, when the browser that is not currently part of the matrix is gaining in popularity and major market share, then it should be included in the matrix at some point to support the application.(subject to client approval)

If the browser is initially part of the matrix and later starts to show rendering and performance issues which can’t be fixed at lower cost, then it can be removed out of the matrix if it is not a priority (subject to client approval). Apart from this, there should no major changes in the browser matrix, as this can lead to needless risk. This decision, therefore, should make based on the proper analysis at a higher level.

If the application renders correctly in the higher version of the browser (e.g. browser2.0 which ships with the device X) and has rendering issues on the older version of the same browser (e.g. browser1.0 which ships with device Y) then you must consider whether these issues can be fixed at a lower cost. If the answer is no, then it is advisable to go ahead with browser2.0 and drop browser1.0 from the matrix, since it would be less popular and would incur high development costs to fix the bugs.

While most companies would probably want to target all devices out there, the reality is that because of the actual cost of sourcing, managing and maintaining all the physical devices the only realistic approach is to select a list of devices that your company will ensure compatibility combined with a well-managed user Beta testing program

Having exactly each device model that is supposed to be compatible with the application is the only way you can give 100% assurance that the application will behave as expected. So for example in the case of iPhone as you cannot go back in versions you have to have the same hardware model multiplied by the number of software versions you want to support. This can get very expensive and complex very quickly. Unfortunately, while helpful for initial testing, none of the alternatives such as emulators, simulators, and so on is 100% reliable so far, not even the emulators/simulators that come with the official SDKs.

Device popularity:

There should be analysis on the popular devices in the market, which holds the major market share. This can include iPhone, Samsung, BB, HTC, Motorola, Google or Nexus, which seem to be everywhere these days. Of course, these matrices are also going to change frequently due to rapidly growing mobile industry which frequently causes changes to device popularity.

Matrix for regions, carriers:

There should be analysis for device/carrier popularity in terms of web traffic from devices in particular countries and regions. For example, the iPhone is extremely popular in the US and because of this; they obviously have more web traffic. AdMob serves Graphical Banner and Text Link ads on mobile web pages for more than 15,000 mobile sites and applications. This monthly report offers a snapshot of its data to provide insight into trends in the mobile ecosystem, which can be very helpful when deciding the browser/device matrix for the application.