Scribd
Upload
91 views19 pages

Evaluation: Inspections, Analytics & Models

This document discusses various methods for evaluating user interfaces, including inspections, heuristic evaluations, walkthroughs, analytics, and predictive models. It describes heuristic evaluation as involving experts reviewing a system based on usability heuristics. Walkthroughs involve experts stepping through scenarios and noting potential problems. Analytics evaluate user traffic patterns. Predictive models like Fitts' Law can evaluate designs without users by predicting expert task completion times.

Uploaded by

Cristi Popa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
91 views19 pages

Evaluation: Inspections, Analytics & Models

This document discusses various methods for evaluating user interfaces, including inspections, heuristic evaluations, walkthroughs, analytics, and predictive models. It describes heuristic evaluation as involving experts reviewing a system based on usability heuristics. Walkthroughs involve experts stepping through scenarios and noting potential problems. Analytics evaluate user traffic patterns. Predictive models like Fitts' Law can evaluate designs without users by predicting expert task completion times.

Uploaded by

Cristi Popa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 19

Chapter 15

Evaluation: Inspections, Analytics & Models


Aims:
• Describe the key concepts associated with
inspection methods.
• Explain how to do heuristic evaluation and
walkthroughs.
• Explain the role of analytics in evaluation.
• Describe how to use Fitts’ Law – a predictive
model.

www.id-book.com 2
Inspections
• Several kinds.

• Experts use their knowledge of users & technology to


review software usability.

• Expert critiques can be formal or informal.

• Heuristic evaluation is a review guided by a set of


heuristics.

• Walkthroughs involve stepping through a pre-planned


scenario noting potential problems.

www.id-book.com 3
Heuristic evaluation
• Developed by Jacob Nielsen in the early 1990s.
• Based on heuristics distilled from an empirical
analysis of 249 usability problems.
• These heuristics have been revised for current
technology by Nielsen and others for:
– mobile devices,
– wearables,
– virtual worlds, etc.

• Design guidelines form a basis for developing


heuristics.
www.id-book.com 4
Revised version (2014) of Nielsen’s original
heuristics

• Visibility of system status.


• Match between system and real world.
• User control and freedom.
• Consistency and standards.
• Error prevention.
• Recognition rather than recall.
• Flexibility and efficiency of use.
• Aesthetic and minimalist design.
• Help users recognize, diagnose, recover from errors.
• Help and documentation.

www.id-book.com 5
No. of evaluators & problems

www.id-book.com 6
Number of evaluators

• Nielsen suggests that on average 5


evaluators identify 75-80% of usability
problems.
• Cockton and Woolrych (2001) point out
that the number of users needed to find
75-80% of usability problems depends on
the context and nature of the problems.

www.id-book.com 7
Heuristics for websites focus on key
criteria (Budd, 2007)
• Clarity
• Minimize unnecessary complexity &
cognitive load
• Provide users with context
• Promote positive & pleasurable user
experience

www.id-book.com 8
Using heuristics to evaluate to evaluate
ambient displays

www.id-book.com 9
3 stages for doing heuristic
evaluation
• Briefing session to tell experts what to do.
• Evaluation period of 1-2 hours in which:
– Each expert works separately;

– Take one pass to get a feel for the product;

– Take a second pass to focus on specific features.

• Debriefing session in which experts work


together to prioritize problems.
www.id-book.com 10
Advantages and problems
• Few ethical & practical issues to consider
because users not involved.
• Can be difficult & expensive to find experts.
• Best experts have knowledge of application
domain & users.
• Biggest problems:
– Important problems may get missed;
– Many trivial problems are often identified;
– Experts have biases.

www.id-book.com 11
Cognitive walkthroughs
• Focus on ease of learning.
• Designer presents an aspect of the
design & usage scenarios.
• Expert is told the assumptions about user
population, context of use, task details.
• One or more experts walk through the
design prototype with the scenario.
• Experts are guided by 3 questions.

www.id-book.com 12
The 3 questions
• Will the correct action be sufficiently evident
to the user?
• Will the user notice that the correct action is
available?
• Will the user associate and interpret the
response from the action correctly?

As the experts work through the scenario


they note problems.

www.id-book.com 13
Pluralistic walkthrough
• Variation on the cognitive walkthrough theme.

• Performed by a carefully managed team.


• The panel of experts begins by working separately.
• Then there is managed discussion that leads to
agreed decisions.
• The approach lends itself well to participatory design.
• Also other adaptations of basic cognitive
walkthroughs.

www.id-book.com 14
Evaluation using analytics
• A method for
evaluating user
traffic through a
system or part of a
system.

• Many examples:
Google Analytics
(chapter 7),
Visistat (shown
below), Learning
Analytics.

• Times of day &


visitor IP
addresses

www.id-book.com 15
Social action analysis
(Perer & Shneiderman, 2008)

www.id-book.com 16
Predictive models
• Provide a way of evaluating products or
designs without directly involving users.

• Less expensive than user testing.

• Usefulness limited to systems with


predictable tasks - e.g., telephone answering
systems, mobiles, cell and smart phones.

• Based on expert error-free behavior.

www.id-book.com 17
Fitts’ Law (Fitts, 1954)
• Fitts’ Law predicts that the time to point at an
object using a device is a function of the
distance from the target object & the object’s
size.
• The further away and the smaller the object,
the longer the time to locate it and point to it.
• Fitts’ Law is useful for evaluating systems for
which the time to locate an object is
important, e.g., a cell and smart phones,
a handheld and mobile devices.
www.id-book.com 18
Key points
• Inspections can be used to evaluate requirements, mockups, functional
prototypes, or systems.

• User testing & heuristic evaluation may reveal different usability problems.

• Design guidelines can be used to develop heuristics

• Walkthroughs are focused so are suitable for evaluating small parts of a


product.

• Analytics involves collecting data about users activity on a website or


product

• Fitts’ Law can be used to predict expert, error-free performance for clearly
defined tasks with limited key presses, eg. data entry and smart phone
use.

www.id-book.com 19

You might also like