Usability Evaluation
What: In-class user testing
When: Monday, 9 April.
For the first part of the project, you will evaluate existing products to identify
key usability strengths and weaknesses. To do so, you will design and execute
a user test on actual alarm clocks with actual users.
I will give you time in class to perform the user testing. We will begin the
class by dividing into two groups: users and testers. The testers will be given
20 minutes to perform their tests with their assigned group of 3–4 users.
During the second half of the class, the roles will reverse.
About Usability Testing
User testing—or usability testing—is where you
get to try out artifacts with a potential real-life user. Usability
testing reveals unforeseen—and often surprising—problems
with documents, artifacts, and interfaces as they are used
by a real audience.
In practice, there are many different kinds and ways of administering
usability tests. Some are done in laboratories using one-way
mirrors, cameras, intercoms, and strict empirical protocols.
Others are done less formally in the context of actual use.
For this assignment, you should perform an informal version
of the "think aloud protocol" with a test user. In
this type of usability testing, you invite your user to vocalize
their thinking as they perform tasks prompted by you. Then,
as the user goes through the steps, carefully record their
actions and reactions.
Keep in mind that a user test is meant to help you determine
the ways in which the interface succeeds or fails to accommodate
to the users' needs and expectations. It is not necessarily
the role of the user to give you design advice or even to tell
you how to fix the problems they find. It is also not your
role to defend any aspect of the interfaces' design.
The General Process
- Developing the test. In developing a sound user test, it may
be helpful to first perform a generic task analysis of an alarm clock to delineate
the typical tasks a user performs with one. (Be sure to think in terms not just
of what the user does in a given a day, but also in the course of a week, month,
and year.) Then, because your user testing time is limited, identify the specific,
most important set of tasks to concentrate on in your testing. Next, develop
a script to prompt and direct the user during the test.
- Orienting your user and explaining the test. It's
important to stress several things to your user about the
test in order to put them at ease and to get good results.
Most importantly, you should stress that the usability test
is not about gauging the user's abilities or intelligence;
rather, it's about gauging the effectiveness of the interface.
Let your user know what types of data you will be recording
about them, and reassure them that there is no wrong course
of action.
- Performing the test and recording the data. One
by one, prompt your user to accomplish a set of tasks with
the clock. Without coaching the user through the process,
record what the user did, how they did it, what comments
they made, and whether the task was accomplished successfully
or not. Additional data that you record could include the
time it took to complete the task, how many times your user
pressed a button, any of the user's reactions or verbalizations,
etc. Ideally, the other
members of your user group will not be within sight or earshot
of your testing while waiting to test with you.
- Expanding on the test. Your usability
evaluation is not restricted to the in-class user test. You
can also perform informal usability assessment of the clocks
within your group, interviews with friends and family, and
extra user tests on your own. Be sure to record the relevant
data and results for the final client report.
- Analyzing your results. After the tests,
determine in your group what works about the interfaces and
what doesn't work. Then, determine and what new interface
features could solve the problems you identified, and what
existing interface features should be retained.