Usability Evaluation

What: In-class user testing
When: Monday, 9 April.

For the first part of the project, you will evaluate existing products to identify key usability strengths and weaknesses. To do so, you will design and execute a user test on actual alarm clocks with actual users.

I will give you time in class to perform the user testing. We will begin the class by dividing into two groups: users and testers. The testers will be given 20 minutes to perform their tests with their assigned group of 3–4 users. During the second half of the class, the roles will reverse.

About Usability Testing

User testing—or usability testing—is where you get to try out artifacts with a potential real-life user. Usability testing reveals unforeseen—and often surprising—problems with documents, artifacts, and interfaces as they are used by a real audience.

In practice, there are many different kinds and ways of administering usability tests. Some are done in laboratories using one-way mirrors, cameras, intercoms, and strict empirical protocols. Others are done less formally in the context of actual use.

For this assignment, you should perform an informal version of the "think aloud protocol" with a test user. In this type of usability testing, you invite your user to vocalize their thinking as they perform tasks prompted by you. Then, as the user goes through the steps, carefully record their actions and reactions.

Keep in mind that a user test is meant to help you determine the ways in which the interface succeeds or fails to accommodate to the users' needs and expectations. It is not necessarily the role of the user to give you design advice or even to tell you how to fix the problems they find. It is also not your role to defend any aspect of the interfaces' design.

The General Process

  1. Developing the test. In developing a sound user test, it may be helpful to first perform a generic task analysis of an alarm clock to delineate the typical tasks a user performs with one. (Be sure to think in terms not just of what the user does in a given a day, but also in the course of a week, month, and year.) Then, because your user testing time is limited, identify the specific, most important set of tasks to concentrate on in your testing. Next, develop a script to prompt and direct the user during the test.
  2. Orienting your user and explaining the test. It's important to stress several things to your user about the test in order to put them at ease and to get good results. Most importantly, you should stress that the usability test is not about gauging the user's abilities or intelligence; rather, it's about gauging the effectiveness of the interface. Let your user know what types of data you will be recording about them, and reassure them that there is no wrong course of action.
  3. Performing the test and recording the data. One by one, prompt your user to accomplish a set of tasks with the clock. Without coaching the user through the process, record what the user did, how they did it, what comments they made, and whether the task was accomplished successfully or not. Additional data that you record could include the time it took to complete the task, how many times your user pressed a button, any of the user's reactions or verbalizations, etc. Ideally, the other members of your user group will not be within sight or earshot of your testing while waiting to test with you.
  4. Expanding on the test. Your usability evaluation is not restricted to the in-class user test. You can also perform informal usability assessment of the clocks within your group, interviews with friends and family, and extra user tests on your own. Be sure to record the relevant data and results for the final client report.
  5. Analyzing your results. After the tests, determine in your group what works about the interfaces and what doesn't work. Then, determine and what new interface features could solve the problems you identified, and what existing interface features should be retained.