Usability testing

From Wikipedia, the free encyclopedia

Jump to: navigation, search
Usability testing is a technique used to evaluate a
product by testing it on users. This can be seen as an
irreplaceable usability practice, since it gives direct
input on how real users use the system.[1] This is in
contrast with usability inspection methods where
experts use different methods to evaluate a user
interface without involving users.
Usability testing focuses on measuring a human-
made product's capacity to meet its intended
purpose. Examples of products that commonly
benefit from usability testing are foods, consumer
products, web sites or web applications, computer
interfaces, documents, and devices. Usability testing
measures the usability, or ease of use, of a specific
object or set of objects, whereas general human-
computer interaction studies attempt to formulate
universal principles.
Contents
[hide]
  1 History of usability testing
 2 Goals of usability testing
 3 What usability testing is not
 4 Methods
o 4.1 Hallway testing
o 4.2 Remote testing
 5 How many users to test?
 6 See also
 7 References
 8 External links

[edit] History of usability testing

A Xerox Palo Alto Research Center (PARC)
employee wrote that PARC used extensive usability
testing in creating the Xerox Star, introduced in
1981.[2]] Only about 25,000 were sold, leading many
to consider the Xerox Star a commercial failure.
The Inside Intuit book, says (page 22, 1984), "... in
the first instance of the Usability Testing that later
became standard industry practice, LeFevre recruited
people off the streets... and timed their Kwik-Chek
(Quicken) usage with a stopwatch. After every test...
programmers worked to improve the program."[1])
Scott Cook, Intuit co-founder, said, "... we did
usability testing in 1984, five years before anyone
else... there's a very big difference between doing it
and having marketing people doing it as part of
their... design... a very big difference between doing
it and having it be the core of what engineers focus
on.[3]
[edit] Goals of usability testing
Usability testing is a black-box testing technique.
The aim is to observe people using the product to
discover errors and areas of improvement. Usability
testing generally involves measuring how well test
subjects respond in four areas: efficiency, accuracy,
recall, and emotional response. The results of the
first test can be treated as a baseline or control
measurement; all subsequent tests can then be
compared to the baseline to indicate improvement.
 Performance -- How much time, and how many
steps, are required for people to complete basic
tasks? (For example, find something to buy,
create a new account, and order the item.)
  Accuracy -- How many mistakes did people
make? (And were they fatal or recoverable with
the right information?)
 Recall -- How much does the person remember
afterwards or after periods of non-use?
 Emotional response -- How does the person feel
about the tasks completed? Is the person
confident, stressed? Would the user recommend
this system to a friend?
[edit] What usability testing is not
Simply gathering opinions on an object or document
is market research rather than usability testing.
Usability testing usually involves systematic
observation under controlled conditions to determine
how well people can use the product.[4]
Rather than showing users a rough draft and asking,
"Do you understand this?", usability testing involves
watching people trying to use something for its
intended purpose. For example, when testing
instructions for assembling a toy, the test subjects
should be given the instructions and a box of parts.
Instruction phrasing, illustration quality, and the
toy's design all affect the assembly process.
[edit] Methods
Setting up a usability test involves carefully creating
a scenario, or realistic situation, wherein the person
performs a list of tasks using the product being
tested while observers watch and take notes. Several
other test instruments such as scripted instructions,
paper prototypes, and pre- and post-test
questionnaires are also used to gather feedback on
the product being tested. For example, to test the
attachment function of an e-mail program, a scenario
would describe a situation where a person needs to
send an e-mail attachment, and ask him or her to
undertake this task. The aim is to observe how
people function in a realistic manner, so that
developers can see problem areas, and what people
like. Techniques popularly used to gather data
during a usability test include think aloud protocol
and eye tracking.
[edit] Hallway testing
Hallway testing (or Hall Intercept Testing) is a
general methodology of usability testing. Rather
than using an in-house, trained group of testers, just
five to six random people, indicative of a cross-
section of end users, are brought in to test the
product, or service. The name of the technique refers
to the fact that the testers should be random people
who pass by in the hallway. [5]
[edit] Remote testing
Remote usability testing (also known as
unmoderated or asynchronous usability testing)
involves the use of a specially modified online
survey, allowing the quantification of user testing
studies by providing the ability to generate large
sample sizes. Similar to an in-lab study, a remote
usability test is task-based and the platforms allow
you to capture clicks and task times. Hence, for
many large companies this allows you to understand
the WHY behind the visitors intents when visiting a
website or mobile site. Additionally, this style of
user testing also provides an opportunity to segment
feedback by demographic, attitudinal and
behavioural type. The tests are carried out in the
user’s own environment (rather than labs) helping
further simulate real-life scenario testing. This
approach also provides a vehicle to easily solicit
feedback from users in remote areas.
[edit] How many users to test?
In the early 1990s, Jakob Nielsen, at that time a
researcher at Sun Microsystems, popularized the
concept of using numerous small usability tests—
typically with only five test subjects each—at
various stages of the development process. His
argument is that, once it is found that two or three
people are totally confused by the home page, little
is gained by watching more people suffer through
the same flawed design. "Elaborate usability tests
are a waste of resources. The best results come from
testing no more than five users and running as many
small tests as you can afford." [6]. Nielsen
subsequently published his research and coined the
term heuristic evaluation.
The claim of "Five users is enough" was later
described by a mathematical model[7] which states
for the proportion of uncovered problems U
U = 1 − (1 − p)n
where p is the probability of one subject identifying
a specific problem and n the number of subjects (or
test sessions). This model shows up as an asymptotic
graph towards the number of real existing problems
(see figure below).
In later research Nielsen's claim has eagerly been
questioned with both empirical evidence[8] and more
advanced mathematical models [9]. Two key
challenges to this assertion are:
1.since usability is related to the specific set of
users, such a small sample size is unlikely to be
representative of the total population so the data
from such a small sample is more likely to
reflect the sample group than the population they
may represent
2. Not every usability problem is equally easy-to-

detect. Intractable problems happen to decelerate

the overall process. Under these circumstances
the progress of the process is much shallower
than predicted by the Nielsen/Landauer formula
[10]
.
Most researchers and practitioners today agree that,
although testing 5 users is better than not testing at
all, a sample size larger than five is required to
detect a satisfying number of usability problems.
[edit] See also
Software Testing portal
 ISO 9241
 Software testing
 Educational technology
 Universal usability
 Commercial eye tracking
 Don't Make Me Think
 Performance testing
 System Usability Scale (SUS)
 Test method
 Tree testing
 RITE Method