User Testing, Continued

After each test you conduct, you will want to summarize your notes. Try to do this as soon as possible after the test, since unless you know shorthand, you'll probably need your short term memory to help you decipher what you've recorded. If you wait until all your tests are completed, you'll risk losing information or misinterpreting your notes.

What should you look for in the results?
Remember that user testing is not like science class. You don't have a hypothesis to prove. Approach it more like an anthropologist or detective, watching for clues in the results that will lead you to findings. The best clues are in patterns, the repetition of remarks or observations.

For example, if every person you test with rants about your navigation system, here's a news flash: you've got a problem with your navigation system. If only one person rants about it, that's a different story. Weed out individual reactions and focus on the shared issues that were raised, both specifics and generalities. This helps you to define where your site's weaknesses -- and strengths -- lie.

What other tools help in presenting results?
Entering your results into a database such as Microsoft Access or Filemaker Pro makes it easier to sort results by factors like skill level. The bulk of your "data" will probably not be data. It should instead be insight, which is far more important. Still, archiving and organizing results this way can be a useful final step, particularly in a large organization where test results will need to be shared among departments.

I like to document user test results with narratives for each user. These explain what happened, start to finish, during the test. I quote users throughout each narrative, especially where they've used strong language (such as words your mother would never let you say in public).

Whoever conducted the tests should prepare a report presenting the trends in the test results and providing a summary of problem areas. He or she should also be prepared to discuss test findings with development team members, some of whom may need handholding. It can be a stressful experience for developers, but remember that user testing is a powerful tool -- and the only clear antidote to the "disease of familiarity."

Jennifer Fleming is an Anchor producer and the author of the upcoming book Web Navigation: Designing the User Experience (O'Reilly, 1998). She runs Square Circle Solutions, a Boston-based web firm specializing in user experience consulting and idea generation.

Find user testing titles in the bookstore. In association with Amazon.com.

view a printable version of this article