User Testing, Continued

So you've planned the setting, found the users and facilitator, and you're ready to go. Make the most of your pre-test planning by preparing your questions and approach as best you can. During a user test, your main goal is passive observation, but you still need to be prepared.

What should you ask?
Before you begin, you'll need to know a few basic facts about your participants. This doesn't mean handing them a 5-page survey about their habits and preferences. What you need to know can be covered in a few quick questions. It's important to get the user's name -- more for social reasons than anything else. It's usually also important to find out what their web skill level is. This becomes useful later when you sort results. If you decide to ask other preliminary questions, make sure they're tightly focused and essential to how you view results. Overwhelming users with too many personal questions up front is no way to begin a test.

Once you've gotten the initial information out of the way, you'll need to briefly reiterate what is expected of users. There's a lot you can do at this point to prejudice people. "Take a look at this site and tell me how we can fix it," for example, biases users toward the negative and provides no real instruction.

Instead, try to explain briefly that you are interested in discovering what they think of the site, both good and bad. Reassure them that you are an objective observer and won't be insulted by anything they say (whether this is true or not, say it anyway). Let them know that this is not a quiz and there are no wrong answers. And finally, tell them that you will not be touring them through the site; instead, it will be the reverse. Ask them to "talk out loud" about what they see and experience as they move through the site.

You may need to give users tasks to get them started or keep them focused during the test. If it's a personal banking site, you might begin by asking them what they would do to sign up for a new account. Watch where they go, when they feel unsure or take the wrong path, and what makes them react positively (or curse like a sailor). When they seem to stall or go around in circles, you can give them another task or ask them to explore another part of the site. Balancing tasks with free exploration is essential, since it increases opportunities for discovery.

When the test is over (30 minutes is usually ideal), you can ask users some more focused questions. If you're concerned that graphic design might be affecting company image, don't ask "What do you think of that red logo?" Instead ask something like, "How do you feel about this company?" or "What kind of company do you think this is? What words would you use to describe the people behind this site?" Once the test is done, you should ask for general impressions. If problems are mentioned, you may want to ask for suggestions on solving them. Avoid asking these more focused questions up front, since they can bias or confuse participants.

What can go wrong?
Some users will not react well to the test or to the site, and may be uncommunicative, belligerent, or distracted. If you've focused on tasks, try giving them more freedom. If you've focused on freedom, try giving them tasks. Different users will react better to different techniques. Avoid simply guiding them through the site, though some will plainly ask you to do so. If necessary, reiterate that you are here as an observer, and remain friendly and calm.

Some tests will be a wash despite all your best efforts, but don't feel bad. People have different personalities, and some users just won't take to the testing. There's nothing you can or should do to weed these people out, but be aware you may have an occasional uncommunicative or unpleasant tester.

useit.com: Usability Inspection Methods
Describes several methods for conducting a usability inspection, including heuristic evaluations and cognitive walkthroughs.

view a printable version of this article