According to Wikipedia, Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. It's a very important usability practice, since it gives you direct input on how real users use your system. It allows you to discover bottlenecks in your design you overlooked yourself, and hands you specific guidelines to improve your system instantly. This blogpost will show you that usability testing isn't something new: people have been testing their products on their users for quite some time. In addition, we will demonstrate how to conduct such a usability test in practice.
First, let us start with some history!
In the 1940’s American Export Lines planned to build two ocean liners. Henry Dreyfuss, a prominent American Industrial designer, was contracted to design these steamships. Dreyfuss, who had done extensive studies on living habits and comforts, charged to provide the utmost comfort for the passengers in each of the three classes as well as for the crew. After Dreyfuss’ team created the designs and subsequently the prototypes of the interior, they invited customers to live in the cabins as if they were on a cruise. While these customers were experiencing the prototypes, Dreyfuss’ team was observing them to detect what was causing frustrations. One of the main issues that Dreyfuss’ team noticed, was that in the smallest cabins, it was impossible to open a suitcase. Based on that observation, Dreyfuss’ team adapted the designs of the cabins making them more usable.
And so Dreyfuss’ performed usability testing avant la lettre. Instead of building the cabins immediately, Dreyfuss and his team first prototyped them, tested the cabins with some customers and finally improved the designs before building the cabins. The essence of the current day usability testing is still the same: inviting customers, letting them use your design and discover potential improvements.
“Design is a process — an intimate collaboration between engineers, designers & clients.”
— Henry Dreyfuss
A good usability testing approach starts with the preparation. When the designs & prototypes are ready, one can start with writing some test scenario's. These should be clear and purely instructional. Such as “Buy a green jacket (size M) from Brand X” when designing for an e-commerce website. Be concise, but concrete. Don’t mention any screens in the scenario's!
Simultaneously, the recruitment of the participants can be started. It’s advisable to narrow down your target audience to the users which are matching your primary personas or, in case you don’t have personas, the people most likely to use your product. When it’s clear who you want to recruit, the actual recruiting can start. For the recruitment itself, there are a bunch of options. You can contact your current user base, involve a recruitment company, use testing tool (e.g. usertesting.com, userzoom) or go on the streets… Based on research of NNG, 5 people are enough to do usability testing. But, I always try to recruit 7 people per persona which I want to test. In practice, there will always be people who are not showing up,… So, if you recruit 7 participants, you are sure that you will have enough people to get relevant information from your sessions.
“You are not your user!”
Don’t test the designs yourself. Everyone involved in the project or even the company is familiar with the lingo, mindset, concepts, etc. used in the design. So, they are biased. On top of that, big chance that people working in the IT sector are not really representative for your target audience. A study from the OECD concluded that that the computer skills of the general population are worse that you would expect. Only 5 percent of the population has a high level of computer skill. Consequently, all people working in IT-related domains are considered as computer experts. Just as everyone who is reading this post…
The testing itself starts with welcoming the participants and making them at ease. A participant who is relaxed will behave as if he is performing the tasks in his natural habitat, leading to better test results. So, invest enough time and effort in welcoming participants.
Next, explain participants the goals and procedures of the session. The most important part is that you emphasize that the participant should think out loud as this will give you insights in their mental models. Don’t forget to ask for the consent of your participants if you (video or audio) record them. In case you pay people, do it upfront to prevent people thinking that they only get paid when they praise your designs. To get people going, ask some simple questions to collect some general feedback about the product.
During the test, one person should be moderator. As moderator, you shouldn’t interfere with the participant but lead them through the test procedure and ask questions to surface the rationale behind the participants’ actions. It’s advisable to have at least one dedicated observer. So, without any other tasks. My experience tells me that combining a moderator and observer role is very hard and will lead to missing relevant information. The observer should make notes of what the participant does, thinks, says, reacts,… It can be very useful to invite a few business stakeholders to play the role of observer. At the end, this will lead to a better understanding and more empathy for their users. Oftentimes, observers are following the session in another room to prevent disturbing the session.
Depending on the type of project and the allocated budget, the setup of the testing can be different. If the project has a substantial budget, you can hire participants, organize an in lab or in situ testing session with a few observers where you record the screen behavior and track the facial mimicry of the participants. This can be in one location or through videoconferencing. When budget is limited and consequently an extended testing set up is out of question, you can perform guerilla testing. This means that you go out on the streets or cafés, look for people who match your personas and ask them to perform a task with your prototypes. Probably, the context won’t be ideal and it will be hard to capture the thinking, doing, remarks and facial mimicry of the participants but at least you let people test the prototypes!
After the testing sessions, the results should be analyzed. You will have some quantitative and qualitative data. The quantitative data will be some metrics defining the success rate of a task, the lostness of participants, time on task, etc. This data will indicate what people are doing. The qualitative data will give you an idea on how people are feeling and what people are thinking while executing the tasks by observing people. This data will tell you why people are doing things and what is their mental model of your product.
Based on the learnings from the testing sessions, you can start redesigning! If it seemed that your prototype was not working, don’t hesitate to throw it away and start from scratch!
The concept of usability testing can be applied on different items, ranging from a paper prototype to a product which is on the market for quite some time. Some companies organize monthly usability testing sessions to evaluate the current state of their product. Others — mostly in the beginning of a project — let participants test a competitor’s product to ensure they are not making the same mistakes as their competitors.
Was this article useful? Which systems in your organization would benefit from usability testing? Let us know in the comments!
Do you want to dive deeper into the world of User Centered design? Have a look at our Experience days training and experience the customer side of your service!
