July 5, 2018
The different ways of testing a mobile app
When a company decides to develop a mobile app, it does so with three main objectives in mind:
Generate user loyalty
Attract positive comments and reviews on app stores
In order to achieve these objectives, it is essential to carry out user tests at several stages in the life cycle of an app.
1. Adopt an agile approach during the development stage
The project team can choose to test the app either during development or after development, i.e.
Develop the app and test it once it's finished, then have to go back to the development stage in order to fix any issues found
Carry out tests over the course of different version deliveries and thus have the ability to update the app immediately and ensure a satisfactorily working version is made available from the outset
As you can see, testing an app during its development saves a tremendous amount of time. The tests carried out will provide the opportunity to check several different aspects of the app:
- That it works properly (with as few bugs as possible)
- That the user interface provides a cohesive experience with respect to what's expected of the app (in terms of design and user-friendliness)
Performance (does the technology work well with the app?)
This phase of testing is often driven by the marketing team.
2. The acceptance phase: the technical control and validation for apps
Once development is completed (after each release when employing the agile approach), it is essential to carry out extensive acceptance tests.
This consists of defining a set of scenarios that correspond to the behaviors of future web users and describing what should happen at each stage of each scenario. This is referred to as a test plan, or a list of use cases. The role of the tester is therefore to reproduce these scenarios and ensure that everything is fully specification compliant.
This type of test primarily achieves two main objectives:
Detection of errors, malfunctions etc.
Detection of any incoherence in the user journey
These tests can be carried out manually or via the use of tools designed to automate the process.
When testing applications, the 10 main aspects to focus on are:
- Installing / uninstalling and launching the app
- Performance: speed and battery consumption
- Online connection: the impact the network on the app
- Interruptions: how the app behaves when calls or notifications are received
- The interface: how well adapted is the design?
- Language: appropriate translations and correct use of currencies, date/number formats etc.
- Memory: the application's memory management
- Stability: ensuring the app doesn't crash at all
- Submission: compliance with app store rules
Once these 10 aspects have been checked and found to be okay, it's then usually time to begin user testing!
3. User tests: our specialty!
The main principles behind user testing
What is the purpose of a user test?
To understand the mobile web user, i.e. how the user perceives the app, what they understand about it, how they use it, and the app's mobility.
What should you test?
As with the user tests described earlier, you need to ensure that the application is working properly in functional and technical terms and with respect to performance and the user interface.
Why is it essential for testers to be external to the project, to be from outside the company?
Because the project team knows the company's objectives, has worked on the app from the beginning, and its members are therefore too close to the app to be able to use it as a "real" user would.
Setting up and organizing a user test
Once the team is aware of all this, the process of organizing the test can begin. Several things first need to be defined in order to do this:
What do you want to test? What procedures should the testers follow?
How many mobile web users do you intend to survey?
What kind of user are you targeting? Who do you want to ask to do the testing? (and for this you can define "personas", i.e. create fictional users and imagine yourself in their place).
The method you're going to use: individual interviews, soft launches accompanied by surveys, the setting up of a group of beta-testers, remote testing, whether moderated or non-moderated – and this needs to be carried out with respect to the target sample and within the constraints of the budget.
How will the testers access the application?
Once all this information has been formally defined, the work is still far from complete. The list of tasks to be assigned to the tester will now need to be drawn up, and you need to ensure these are formulated in as neutral and precise a manner as possible.
Example: If testing an online appointment booking application, you would instruct the tester to make an appointment with Doctor Jones on the 6 October at 10:30 am and not simply tell them to "make an appointment with the doctor".
It is essential to clearly formulate instructions in this way: it is critical to the successful execution of the tests and the quality of the feedback received from the testers.
Next comes the tester recruitment stage, followed by the testing phase itself.
Making use of the data
Once the data has been obtained, it is time to analyze it in as operationally beneficial a way as possible! The objective is to find 5 to 10 ways of optimizing the application and thus improving it.
This exercise requires both meticulousness and objectivity. To ensure it's properly carried out:
Read all the feedback provided by testers in full
Identify all the problems raised and count the number of times each is mentioned
Read between the lines. e.g. try to notice if there are any elements that the web user has failed to "see" on the page, perhaps because they're too small, badly positioned, not relevant, etc. Nothing should be excluded from the process.
And finally, here are the kinds of issues you should deal with first, as a matter of priority:
Issues that keep occurring on a frequent basis
"Quick win" improvements that can be made in a few minutes
Critical issues – bugs that interfere with the correct operation of the application.
By carrying out frequent tests, the application can be constantly improved. Over time, the identified changes will become increasingly minor in scale, resulting in an application that meets user requirements and expectations as closely as possible.
So don't wait a second longer. Start testing and re-testing your mobile apps right away, and remember to follow the 10 rules of user testing!