User testing of mobile apps: the 10 commandments

When a company sets out to develop a mobile application, it is essential for it to plan for several test phases, both during product development and after submitting it in stores.

Within the French Mobile Marketing Association, Ferpection has worked with Orange, StarDust and Open to identify best practice, whether in terms of functional testing or user testing.

Here is our list of the 10 best practices to ensure that you know when, what and how to test! These are excerpts from the French MMA’s white paper which you may download via the link shown at the end of the article.

1. Get the test team involved in the project as soon as possible

The mobile app tests must commence as soon as possible in order to take user feedback into account from the outset. This guarantees that the application will meet their needs as soon as it is submitted in app stores. This also makes it possible to save a lot of time and money, because the later a design error or another anomaly is noticed, the more time consuming and costly it is to resolve.

2. Adopt the best possible test strategy

By defining its testing strategy, you must bear one thing in mind: it is impossible to test everything. Upon the mission’s conclusion, you will not be certain that all flaws have been detected.

Also, it is paramount to pace oneself and prioritize the most important features and ways forward for the app. 

3. Make a proper distinction between the proportion of manual vs automated tests

For the functional tests, one observes the following average spread: 70% automated tests/ 30% manual tests. This proportion may vary and must be tailored to each product, during each of the test phases. In order to take this decision, several vectors must be taken into account: the number of test campaigns, the target demographic composition and the regression tests put in place. 

4. Establish one’s targets carefully

In the case of functional tests, the target demographic must be defined (age, gender, location, interests, customer of such and such company, SPC, mobile consumption habits, etc…) on the basis of the app being tested and the test objectives. It is not always desirable to “over target” or to focus solely on a marketing target. Using personas makes it possible to establish the appropriate target demographic to adopt. When conducting acceptance or regression tests, it is important to identify the terminals concerned (brand, screen size, OS version etc…) as well as the connection method (mobile or Wi-Fi data).

5. Stick as closely as possible to reality

In order to obtain workable results, it is essential for the test conditions and ways forward to be as close as possible to the app’s intended use. Personas are also a means of constructing scenarios for operational tests.

6. Set out a regression initiative

Whenever the development team makes a change (adding a feature, modifying an existing feature), there is a risk that this change could impact upon other features. Prior to implementing this change on the production line, it is therefore necessary to plan for regression tests in order to ensure the smooth running of all of the app’s features.

7. Put the appropriate test tools in place

Using a blend of several test tools makes it possible to cover all aspects of the app’s testing and to back up the recommendations deriving from these tests with concrete proof. Various types of tool may be employed: remote user testing via crowdsourcing, one-to-one tests with friends and family, automated tests, benchmark tests, scenarios, etc…

8. Ensure that there is a stable test environment

During a test phase, it is imperative that no changes are made to the test environment. This could have 2 negative effects:

• Biased results between the different participants, since the app is not identical for everybody.
• Bugs possibly generated by changes (regressions).

9. Maintain test environments and configurations once the product has been put into production

The app will always undergo changes, whether at the behest of its owner (design, marketing and feature changes, etc…) or of the market (OS and browser updates, launching of new terminals, changes in mobile consumption habits, etc…).

It is therefore important to maintain an environment dedicated to developing and testing the app, throughout its life cycle.

10. Assign a dedicated team to carry out the test activities

The test team must comprise a minimum of 3 roles:

• Analyst: sets out the test strategy and scenario
• Tester: performs the tests and ensures that any anomalies are followed up
• Project manager: oversees the test team and ensures the smooth running of the project

If you follow these 10 commandments, your testing strategy should allow you to roll out an app in stores that meets users’ needs and evolves to match their expectations as closely as possible!

In order to obtain more in depth information, download the mobile application test guide produced by members of the Mobile Marketing Association, Orange, StarDust, Open and Ferpection!