Human Computer Interaction

Fundamentals and Practice [ SWE - 431 ]

Gerard Jounghyun Kim

Chapter: 8
User Interface Evaluation

Mahfuzur Rahman Emon

Lecturer, IICT, SUST
Human Computer Interaction
Chapter 8:

Evaluation Criteria

When evaluating the interaction model and interface, there are largely two criteria

● Usability: Refers to the ease of use and learnability of the user interface

● User Experience

Usability: Refers to the ease of use and learnability of the user interface. Can be measure in two ways

● Quantitatively
● Qualitatively
Human Computer Interaction
Chapter 8:

Quantitative Assessment:

● Involves task-performance measurements. An interface is “easy to use and learn” if the subject/subjects is/are able to show
some minimum user performance on typical application tasks.

● Popular choices of such performance measures are task completion time, task completion amount in a unit time (e.g., score),
and task error rate

● For example, we would like to test a new motion-based interface for a smartphone game. We could have a pool of subjects
play the game, using both the conventional touch-based interface and also the newly proposed motion-based one. We could
compare the score and assess the comparative effectiveness of the new interface.

● The underlying assumption is that task performance is closely correlated to the usability (ease of use and learnability), such
an assumption is quite arguable.

● Task-performance measures, while quantitative, only reveal the aspect of efficiency (or merely the aspect of ease of use) and
not necessarily the entire usability.
Human Computer Interaction
Chapter 8:

Quantitative Assessment:

● The aspect of learnability should be and can be assessed in a more explicit way by measuring the time and effort (e.g.,
memory) for users to learn the interface.

● The problem is that it is difficult to gather a homogeneous pool of subjects with similar backgrounds (in order to make the
evaluation fair)

● Measuring the learnability is generally likely to introduce much more biasing factors such as differences due to
educational/experiential/cultural background, age, gender, etc.

● Finally, quantitative measurements in practice cannot be applied to all the possible tasks for a given application and
interface. Usually, a very few representative tasks are chosen for evaluation. This sometimes makes the evaluation only
partial.
Human Computer Interaction
Chapter 8:

Qualitative Evaluations:

● To complement the shortcomings of the quantitative evaluation, qualitative evaluations often are conducted together with
the quantitative analysis.

● Conducting a usability survey, posing usability-related questions to a pool of subjects after having them experience the
interface.

● Includes questions involving the ease of use, ease of learning, fatigue, simple preference, and other questions specific to the
given interface.

● Ex. NASA TLX (Task Load Index) and the IBM Usability Questionnaire
Human Computer Interaction
Chapter 8:

NASA TLX Usability Questionnaire

The NASA Task Load Index method assesses the workload on a seven-point
scale. Increments of high, medium, and low estimates for each point result in 21
gradations on the scale
Human Computer Interaction
Chapter 8:

IBM Usability Questionnaire for

computer systems.
Human Computer Interaction
Chapter 8:

User Experience: There's no precise definition, A holistic concept encompassing more than just the interface, extending to the
entire product or application, and even the product family. It involves the user's emotions and perceptions arising from using the
application through a given interface.

Various aspects to be considered in

totality for assessing user
experience (UX).
Human Computer Interaction
Chapter 8:

Evaluation Methods

Whether it is for the user experience or more narrow usability, or whether for the qualitative feelings or quantitative performance,
there is a variety of evaluation methods.

A given method may be general and applicable to many different situations and objectives, or it may be more specific and fitting
for a particular criterion or usage situation. Overall, an evaluation method can be characterized by the following factors:

● Timing of analysis (e.g., throughout the application development stage: early, middle, late/after)

● Type and number of evaluators (e.g., several HCI experts vs. hundreds of domain users)

● Formality (e.g., controlled experiment or quick and informal assessment)

● Place of the evaluation (laboratory vs. field testing)

Human Computer Interaction
Chapter 8:

Focus Interview/Enactment/Observation Study:

● One of the easiest and most straightforward evaluation methods is to simply interview the actual/potential users and observe
their interaction behavior, either with the finished product or through a simulated run.

● The interview can involve actual system usage, or a paper/digital mock-up can be used for enacted scenarios. Mock-ups offer
a tangible feel early in development, but may lack some interactive features.

● Wizard of Oz testing, with a human simulating system responses, is used when features are not implemented.

● The interview is often focused on particular user groups (e.g., elderly) or on the features of the system/interface (e.g.,
information layout) to save time.

● A particular interviewing technique, The cognitive walkthrough where the subject talks through their thought process. It's
designed to identify gaps between the system's interaction model and the user's understanding. This method is suitable for
evaluating the early stages of design, such as interaction modeling or interface selection, rather than specific interface
design.
Human Computer Interaction
Chapter 8:

Interviewing a subject upon simulating the usage of the interface with a mock-
up.
Human Computer Interaction
Chapter 8:

A cognitive walkthrough with the

interviewer.
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8:

Focus Interview/Enactment/Observation Study:

● Notable variation of the actual usage-based testing is the “Can you break this?” type of testing in which the subject is given
the mission to explicitly expose interface problems, e.g., by demonstrating interface flaws and interface-design-related bugs.

● Interview/simulation method, due to its simplicity, can be used not only for evaluation, but also for interaction modeling and
exploration of alternatives at the early design stage

● The interview is free form, easy to administer but not structured or comprehensive.
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8:
Human Computer Interaction
Chapter 8: