September 11, 2017
User testing of mobile apps: the 10 commandments
When a company sets out to develop a mobile application, it is essential for it to plan for several test phases, both during product development and after submitting it in stores.
Within the French Mobile Marketing Association, Ferpection has worked with Orange, StarDust and Open to identify best practice, whether in terms of functional testing or user testing.
is our list of the 10 best practices to ensure that you know when, what and how to test! These are excerpts from the French MMA’s white paper which
you may download via the link shown at the end of the article.
1. Get the test team involved in the project as soon as possible
mobile app tests must commence as soon as possible in order to take user
feedback into account from the outset. This guarantees that the application
will meet their needs as soon as it is submitted in app stores. This also makes
it possible to save a lot of time and money, because the later a design error
or another anomaly is noticed, the more time consuming and costly it is to
2. Adopt the best possible test strategy
By defining its testing strategy, you must bear one thing in mind: it is impossible to test everything. Upon the mission’s conclusion, you will not be certain that all flaws have been detected.
Also, it is paramount to pace oneself and prioritize the most important features and ways forward for the app.
3. Make a proper distinction between the proportion of manual vs automated tests
For the functional tests, one observes the following average spread: 70% automated tests/ 30% manual tests. This proportion may vary and must be tailored to each product, during each of the test phases. In order to take this decision, several vectors must be taken into account: the number of test campaigns, the target demographic composition and the regression tests put in place.
4. Establish one’s targets carefully
the case of functional tests, the target demographic must be defined (age, gender,
location, interests, customer of such and such company, SPC, mobile consumption
habits, etc…) on the basis of the app being tested and the test objectives. It
is not always desirable to “over target” or to focus solely on a marketing
target. Using personas makes it possible to establish the appropriate target
demographic to adopt. When conducting acceptance or regression tests, it is
important to identify the terminals concerned (brand, screen size, OS version
etc…) as well as the connection method (mobile or Wi-Fi data).
5. Stick as closely as possible to reality
order to obtain workable results, it is essential for the test conditions and
ways forward to be as close as possible to the app’s intended use. Personas are
also a means of constructing scenarios for operational tests.
6. Set out a regression initiative
the development team makes a change (adding a feature, modifying an existing
feature), there is a risk that this change could impact upon other features.
Prior to implementing this change on the production line, it is therefore
necessary to plan for regression tests in order to ensure the smooth running of
all of the app’s features.
7. Put the appropriate test tools in place
a blend of several test tools makes it possible to cover all aspects of the
app’s testing and to back up the recommendations deriving from these tests with
concrete proof. Various types of tool may be employed: remote user testing via crowdsourcing,
one-to-one tests with friends and family, automated tests, benchmark tests, scenarios,
8. Ensure that there is a stable test environment
During a test phase, it is imperative that no changes are made to the test environment. This could have 2 negative effects:
- Biased results between the different participants, since the app is not identical for everybody.
- Bugs possibly generated by changes (regressions).
9. Maintain test environments and configurations once the product has been put into production
The app will always undergo changes, whether at the behest of its owner (design, marketing and feature changes, etc…) or of the market (OS and browser updates, launching of new terminals, changes in mobile consumption habits, etc…).
is therefore important to maintain an environment dedicated to developing and
testing the app, throughout its life cycle.
10. Assign a dedicated team to carry out the test activities
The test team must comprise a minimum of 3 roles:
- Analyst: sets out the test strategy and scenario
- Tester: performs the tests and ensures that any anomalies are followed up
- Project manager: oversees the test team and ensures the smooth running of the project
If you follow these 10 commandments, your testing strategy should allow you to roll out an app in stores that meets users’ needs and evolves to match their expectations as closely as possible!
In order to obtain more in
depth information, download the mobile
application test guide produced by members of the Mobile Marketing Association, Orange,
StarDust, Open and Ferpection!