Intro
Most if not all established UX professionals would agree that usability testing is of paramount importance. That’s the only way to know for sure whether your users find your design useful and useable. However, the insights gathered during usability testing sessions are only as useful, as your questions are deep and adequate. This is what this blog post is about – making sure you’re asking the right questions.
Let’s get started.
It all starts with a plan
As you might already know, usability testing sessions require a fair amount of planning. In short, you need a functional prototype, a list of hypotheses, and a rough interview script. Otherwise, you’ll wind up with a bunch of chaotic data that’s hard to draw conclusions from.
Once you have an idea of a plan, you should frame your questions around your hypotheses. Note, that if your hypothesis is “users find onboarding difficult”, that doesn’t mean that it’s exactly what you should ask. Not only it’s a leading question (more on that later), it’s also a question that might be subject to users’ bias.
Good questions lead you closer to achieving the goal (e.g. testing a hypothesis) of your user testing; the ones that do not should be reconsidered, since your time with the user is limited.
Avoid leading questions
Before going through which questions to ask, it’s worthwhile to understand which ones are better left unasked. Questions that are too personal or downright rude aside, leading questions are bad for a variety of reasons.
Leading questions are the ones that already suggest an answer or limits the answering options to just a few. Here are a few examples:
Either or questions;
“Yes or no” questions;
“Do you like…” questions.
The aforementioned questions will give a very surface-level understanding of how your users think. It’s not useful, and may even lead to you to a worse design than you started with.
Now that we know what not no ask, let’s go through the ones that you should.
Pre-test usability testing questions
Before diving into the meat of the interview, it’s worth spending some time on non-design-related questions. There could be a few reasons for that. Firstly, if you have any gaps in understanding of your users such as their demographics, you should ask for clarifications. Secondly, the first few interactions with the user set the tone for the whole interview. You should make sure the interviewee is relaxed and predisposed to being honest, so small talk goes a long way.
Many resources we’ve checked while comparing our processes with those of our fellow designers, indicate that before the actual test you should ask questions about age, gender, education, etc. We’re not sure how useful it would be. The only people you should talk to are the ones that qualify for your prior persona research/user profile. You should only double-check if you have doubts, preferably before the actual call.
Before the actual testing session, we like to start with small talk to create a relaxed atmosphere. Then, we make sure to make it extremely clear that we’re testing the design, not the user. Not doing this simple procedure, is likely to cause bias, i.e. a user might associate giving feedback on the design, as giving feedback on the designer. You can read more on that in our article on bias in usability testing.
Once we’re done with the preliminary chat, it’s time to move on to the actual testing.
Questions to ask during the usability testing session
Let’s take a minute to distinguish between moderated and unmoderated usability testing sessions. Moderated ones have you, the designer/researcher, as a guide, while the unmoderated ones leave users to their own devices. There’s a continuum of designers’ involvement in these testings. Generally, you would want to have users as free as possible, while also nudging them to complete specific tasks or actions.
The questions below are a mix of specific and generic questions to get you in the right line of thinking and give you examples of questions we think are good.
What do you think about the way the information in front of you is presented?
How has your experience been with using this particular modal?
What do you think this button does? Do you think there other ways to perform this action? If so, how would you do it?
Try to leave the most general questions for after the user completed the whole series of tasks you’ve prepared. Of course, the specificity or genericness of your questions will also depend on the scope of the actions you want your users to take.
Questions to ask after the test
Once that user completed the testing session, it’s time to ask general questions about the overall experience. Here are a few examples below:
Overall, how has your experience been with the app?
If you could tweak just one thing, what would it be? Why do you think that?
Would you be willing to refer your friends and colleagues to this app?
Is there anything that you absolutely loved about using this prototype?
Does this way of tackling your problem, better than the one you currently use?
Once you’re done asking questions, make sure to thank users for their time. Also, if there are any incentives, make sure to instruct a user on how to get their reward. Finally, you could also ask if that user has any friends/colleagues who are of a similar demographic for you to test.
What’s next
Once you’re done with the interview, you should have a recording of it available (if the user provides consent). At the very least you should have made extensive notes. Note, that the more direct quotes these notes contain, the better. All of the notes you’ve made during a usability testing campaign should be structured and organized. What you do next will depend on the insights you’ve gathered.
We also have an article on the tools that can help you make the most of the usability testing sessions. Check out our blog post on usability testing tools here.
Outro
Although usability testing is a discipline in itself, we hope that navigating through its intricacies became a tad easier. Thank you for sticking with us to the very end, and good luck on your quest for a useful and useable product!
