Chapter Seven
Evaluation Techniques
and Universal Design
Evaluation Techniques
Evaluation
tests usability, accessibility and functionality of
system
occurs in laboratory, field and/or in
collaboration with users
evaluates both design and implementation
should be considered at all stages in the design
life cycle
Goals of Evaluation
To assess extent and accessibility of system
functionality- users functionality and expectation
To assess the user’s experience of the interaction-
systems usability, user’s satisfaction and
enjoyment.
To identify specific problems- unexpected result-
confusion amongst users
Evaluation Criteria
We will consider evaluation techniques under two broad
headings.
Evaluation through Expert analysis
Evaluation through user Participation
Evaluation through Expert analysis
Cognitive Walkthrough
Heuristic Evaluation
Model-based evaluation
Using previous studies/ work
Cognitive Walkthrough
Proposed by Polson et al.
based on software engineering walkthrough (sequence of
steps)
The programmers shows her source code to other and
explain line by line
It can be done at any stage of development process when
your prototype or actual system implementation will
work.
Heuristic Evaluation
Proposed by Nielsen and Molich.
usability criteria (heuristics) are identified
Performed in design phase, useful for evaluating early
design, Prototype or early system.
Used in early stage of development from first sketches
and outline descriptors
There are Nielsen’s ten heuristics, i.e. Visibility of system
status, Match between system and the real world, User
control and freedom, Consistency and standards, Error
prevention, Recognition rather than recall, Flexibility,
Aesthetic, diagnose, recover and Help.
Using previous studies in evaluation
Based on previous results evidence to supports
aspects of the design.
For e.g Menu, Command Name, Colour,
Schemes, Buttons, Labels, Icons Design etc.
Model-based evaluation
Use high level models that predict user performance
e.g. GOMS (goals, operators, methods and selection)
model predicts user performance with a particular
interface.
Evaluating through user Participation
Styles of Evaluation: Laboratory studies
Field Studies
Query techniques: Interviews
Questionnaires
Experimental evaluation: Participants
Variables
Hypothesis
Experimental design
Statistical measures
Studies of groups of users
Evaluating through user Participation…
Observational techniques:
Think aloud and cooperative evaluation: direct
asking
Protocol analysis: Methods for recording user actions
Automatic protocol analysis tools: tools for handling
the data
Post-task walkthroughs
Evaluation through monitoring physiological responses:
Eye tracking for usability
evaluation
Physiological measurements
Choosing an Evaluation Method
We can identify at least eight factors that distinguish different
evaluation techniques and therefore help us to make an
appropriate choice. These are:
stage in development cycle:design stage involve experts
only. implementation stage developers need user’s
participation.
the style of evaluation: laboratory vs. field
the level of subjectivity or objectivity of the technique:
subjectivity techniques depends on knowledge and expertise.
objective techniques depends on particular evaluations.
the type of measures provided: subjectivity techniques
provides qualitative measures vs. objective techniques
provides quantitative measures.
Choosing an Evaluation Method…
the information provided: high level information decides
Is system usable? vs. low level information includes
decisions such as Which font is most readable?
the immediacy of the response: Get immediate response
of experiments and analysis.
the level of interference implied: how users use the
interface? obtrusive vs. unobtrusive
the resources required: time, money participants,
equipment, expertise of
evaluation etc.
Universal Design
Introduction
Universal design principles
Multi-modal interaction
Designing for diversity
Universal Design
Universal design is the process of designing products so
that they can be used by as many people as possible in as
many situations as possible.
Universal design is about designing systems that are
accessible by all users in all circumstances, taking account
of human diversity in disabilities, age and culture.
universal design principles
Equitable use: No user is excluded or stigmatized
Flexibility in use:-allows for a range of ability and preference
Simple and intuitive to use: should not be unnecessarily complex
Perceptible information:- provide effective communication of
information
Tolerance for error:-minimizing the damage caused by mistakes
Low physical effort:-systems should be designed to be
comfortable to use, minimizing physical effort and fatigue
Multi-Modal interaction
providing access to information through more than one
mode of interaction is an important principle of universal
design.
More than one sensory channel in interaction
e.g. sounds, text, hypertext, animation, video, gestures,
vision
Used in a range of applications:
particularly good for users with special needs, and
virtual reality
Will cover
general terminology
speech
non-speech sounds
handwriting
Usable Senses
As we saw in Chapter 2 : There are 5 senses (sight, sound,
touch, taste and smell) are used by us every day
each is important on its own
together, they provide a fuller interaction with the
natural world
Computers rarely offer such a rich interaction
Can we use all the available senses?
ideally, yes
practically – no
We can use • sight • sound • touch (sometimes)
We cannot (yet) use • taste • smell
Designing for diversity
people are diverse and there are many factors that must be
taken into account if we are to come close to universal
design.
some of these factors: disability, age and culture
Users with disabilities
How might universal design of technology assist
people?
visual impairment
screen readers, SonicFinder
hearing impairment
text communication, gesture, captions
physical impairment
speech I/O, eyegaze, gesture, predictive systems (e.g.
Reactive keyboard)
speech impairment
speech synthesis, text communication
dyslexia
speech input, output/ spelling correction program
Autism: Communication(Computer-mediated
cont…
age groups
older people e.g. disability aids, memory aids,
communication tools to prevent social isolation
children e.g. appropriate input/output devices,
involvement in design process
cultural differences
influence of nationality, generation, gender, race,
sexuality, class, religion, political influence etc. on
interpretation of interface features
e.g. interpretation and acceptability of language,
cultural symbols, gesture and colour
End of the chapter
