What is user testing?
User testing, or usability testing, is a key component in designing, finalizing and launching a great digital product. It’s about detecting usability issues at any stage of the product development cycle with the aim of improving the overall user experience. This is done by monitoring how representative users interact with the product in a realistic setting. This way, you can see exactly what works and — more importantly — what doesn’t.
To ensure you set up a valid usability test you need to understand the user testing process.
The problem many designers and product development teams have, however, is conducting user testing that will identify actual product issues and design flaws. This is understandable. There are a lot of variables that can influence the quality and validity of a usability test. Things like budget, tight deadlines, participants, shortages in staff, poorly structured questions, unclear criteria, and different testing environments can — and generally do — devalue the final results.
To ensure you set up a valid usability test you need to understand the user testing process. We’ve jotted down 6 easy-to-follow steps for successful user testing. Let’s get straight into it.
User Testing Checklist
Download our free usability testing checklist to empower your product with the knowledge that comes directly from your users:
Download Free ChecklistHow to conduct user testing
Step 1: Define Testing Objectives
Start by clearly outlining your user testing goals and UX metrics. What is it you want to test and what answers do you expect to obtain?
Maybe it's:
Task completion. Can the user complete the task? What’s stopping the user from completing the task? How far into the task completion funnel can users get?
Time for task. How much time does the user need to complete the task? Does the task take too much or too little time to complete?
Flow efficiency. Can users easily navigate through the product? Was any part of the user flow confusing? Do you need to add/remove any steps?
Error detection. Did users experience any errors? When did the errors occur? Were they avoidable or unavoidable? Could the user recover from the setback?
Or maybe it’s something else entirely.
What you want to test is completely up to you. Just make sure you define your testing objectives beforehand and maintain consistency. Don’t expect to just ‘wing it’ because you won’t. Usability testing takes a lot of research, planning, and preparation. So be ready with a solid test plan.
Another reason you can't just play by ear is that you need to collect consistent sets of data to track progress.
Step 2: Choose a User Testing Method
Once you’ve decided what you want to get out of testing, it’s time to choose the most adequate user testing method. There are numerous testing methods to consider. Here are some common ones:
Individual in-depth interviews. This method takes the longest to complete and requires a trained test moderator to run the user test with the help of a prepared script and task scenarios.
Remote user testing. A cost-effective technique, remote user testing can be conducted in a natural setting, like at home or work, and can be moderated or unmoderated.
Card-sorting. Participants organize content cards into a dendrogram that makes the most sense to them.
A/B testing. Users are given two alternatives of a digital product to determine which version performs better. This method needs more participants than others for accurate results.
Eye movement tracking. This method employs eye-tracking software to trace where users are looking when interacting with your product.
Focus groups. This test involves a moderated discussion between small groups of users and takes place in a room equipped with a two-way mirror.
When choosing a UX testing method, have in mind where your product is currently at in the product development cycle. Some testing methods are better to use early on, while some work better on a finished product. Of course, you also want to consider things like budget, resources, and testing facilities before you choose a testing method.
Step 3: Find Representative Users
When recruiting users to sit through user testing, make sure you have established clear selection criteria. This is so that you test people who match your target user profile.
Around 3–5 users (preferably – 5) are generally more than enough to identify key issues. Segment test participants according to certain characteristics that reflect your perfect user persona, like:
Age
Gender
Location
Education
Profession
Income
Technical proficiency
Other characteristics specific to your product
Choose people who are not familiar with your product for more reliable results. Users who are already familiarized with your product may have preconceived ideas about completing tasks. Or their wish to maintain the status quo may result in serving up false flattery over constructive criticism. For this reason, you want to avoid recruiting fellow employees, their immediate family members and friends. That way you can avoid users' bias. It’s highly likely that testing your product on these groups of people will contaminate the reliability of the final results.
In addition to test participants, you also need an experienced facilitator to guide the test and see to it that everything runs smoothly, as well as several observers who will take notes during the test.
Tip: Make sure your screening questions are inclusive when asking personal questions. For instance, if one of your questions is about income, provide income brackets in order to respect the applicant’s privacy. Don’t ask them to reveal their exact salary, as this crosses the line of confidentiality.
Step 4: Create Task Scenarios
The way you ask users to complete tasks is paramount to user testing success. You don’t want to reveal the exact order of actions users need to take in order to get from point A to point B. The purpose of user testing isn’t to direct users towards the finish line, but to give them enough information to complete the journey intuitively — on their own accord.
Be vague in your task scenarios and be mindful of formulating the right questions. Share as little information as needed to complete the task. Providing too many instructions can ruin the purpose of the test. Think of it like a mathematical equation. You give users a problem and they have to find the correct formula to solve it.
Here’s an example of a bad task scenario:
🔴 Go to the search bar, type in red sandals, select size 8, click on the ‘add to cart’ button and continue to the checkout.
Why it’s bad: You’re not leaving much room for the user to figure out the user flow. Therefore, you can’t guarantee that the user would achieve the same results without explicit instructions.
And here’s an example of a good task scenario:
✅ Explore the website and buy a pair of red sandals in your size using the credit card number provided.
Why it’s good: You’re giving the user a sufficient amount of information on what you want them to do, without revealing too much.
Furthermore, when formulating task scenarios, be sure to incorporate both open-ended and goal-specific tasks. Open-ended questions are great for observing users interact with a product freely without guided interruption, while goal-specific tasks are effective in testing particular features. Both are extremely valuable in user testing when used correctly.
Step 5: Replicate the Testing Environment
Replicate everything you do and say during user testing. This includes when and how you say and do something. While this may seem a bit over the top, trust us — it’s not. Recreating the testing environment is the only way you can be 100% certain that your results are accurate.
Scripts also play a very important role in user testing. By having a script you eliminate the chances of inconsistencies occurring during testing. Interestingly, it is generally the test facilitator that prompts inconsistencies to happen. For example, they might greet one participant in one way, and emphasize particular parts of a task to another. Or they may disclose certain details about the testing process to some and not to others. All of this adds up to an inconsistent testing environment.
For this reason, it’s best to use a script to replicate the testing environment and ensure consistency is maintained throughout. Additionally, if you're dealing with a product that heavily relies on context (e.g. a product for ER nurses), you might want to go as far as conducting a contextual inquiry.
Why is designing for an ER Nurse and accountant different? – The context of usage.
Step 6: Analyze the Findings
This is the moment you’ve been waiting for. It’s finally time to gather the results and make sense of them. How have the participants responded to the test? What key issues did they uncover? Have any of your hypotheses been proven true? Share and summarize your findings with your team.
After you’ve finished summarizing the findings, work together with the design and development team to prioritize issues, identify possible solutions, and determine the best course of action to improve the user experience.
User Testing: Test, Analyze, Repeat
Successfully conducting user testing doesn’t have to be rocket science. It’s quite easy once you define your objectives, choose your testing method, create your task scenarios, formulate your questions, and document the results.
Follow these 6 steps for user testing and get the answers you need to improve user experience, increase retention rates and reel in the profit you deserve. Test, analyze, and repeat. Good luck and happy testing!
Another handy resource you might want to try is our checklist for usability testing
Further Readings
Q&A
How to conduct user testing?
It takes 6 steps: defining the goals, choosing the methods, finding the users, creating task scenarios, replicating the user environment and analyzing the findings.
How to conduct remote user testing?
The process is identical to the in-person testing sessions. You'd just need to make sure the remote meeting infrastructure is in place.
