Chapter 2

The Interaction
Overview
• Interaction models help us to understand what is going on in the
interaction between user and system.
• They address the translations between what the user wants and what the
system does.
• Ergonomics looks at the physical characteristics of the interaction and
how these influence its effectiveness.
• The dialog between user and system is influenced by the style of the
interface.
• The interaction takes place within a social and organizational context
that affects both user and system.

2
Introduction
• The Interaction
• interaction models
– translations between user and system
• ergonomics
– physical characteristics of interaction
• interaction styles
– the nature of user/system dialog
• context
– social, organizational, motivational

3
MODELS OF INTERACTION
• Models help us to understand complex behavior and complex systems.
• Interaction involves at least two participants: the user and the system.
• Both are complex and are very different from each other in the way that
they communicate
• The interface must therefore effectively translate between them to allow
the interaction to be successful
• This translation can fail at a number of points and for a number of
reasons.
• The use of models of interaction can help us to understand exactly what
is going on in the interaction and identify the likely root of difficulties

4
Models of interaction
terms of interaction
Norman model
interaction framework
• Some terms of interaction:
domain – the area of work under study
• A domain defines an area of expertise and knowledge in some real-world
activity.
• Some examples of domains are graphic design, authoring and process
control in a factory.
• A domain consists of concepts that highlight its important aspects.
• In a graphic design domain, some of the important concepts are
geometric shapes, a drawing surface and a drawing utensil. 5
… Some terms of interaction
• Tasks
• are operations to manipulate the concepts of a domain.
• How you go about doing it
• For example, one task within the graphic design domain is the construction of
a specific geometric shape with particular attributes on the drawing surface
• A goal is the desired output from a performed task
Goal – what you want to achieve
e.g. create a solid red triangle
• An intention is a specific action required to meet the goal.
• Two models of interaction:
• Norman’s execution–evaluation cycle
• The interaction framework
6
The execution–evaluation cycle
• Norman’s model of interaction is perhaps the most influential in HCI,
• because it is closer to our intuitive understanding of the interaction between
human user and computer
• The user formulates a plan of action, which is then executed at the
computer interface.
• When the plan, or part of the plan, has been executed, the user
observes the computer interface to evaluate the result of the
executed plan, and to determine further actions
• The interactive cycle can be divided into two major phases: execution
and evaluation.
• These are then subdivided into seven stages.

7
… Donald Norman’s model
• The stages in Norman’s model of interaction are as follows:
– user establishes the goal
– formulates intention
– specifies actions at interface
– executes action
– perceives system state
– interprets system state
– evaluates system state with respect to goal

• Norman’s model concentrates on user’s view of the interface

8
… execution/evaluation loop

goal

execution evaluation
system

• user establishes the goal

• formulates intention
• specifies actions at interface
• executes action
• perceives system state
• interprets system state
• evaluates system state with respect to goal

9
… execution/evaluation loop

goal

execution evaluation
system

• user establishes the goal

• formulates intention
• specifies actions at interface
• executes action
• perceives system state
• interprets system state
• evaluates system state with respect to goal

10
… execution/evaluation loop

goal
execution evaluation
system

• user establishes the goal

• formulates intention
• specifies actions at interface
• executes action
• perceives system state
• interprets system state
• evaluates system state with respect to goal

11
… execution/evaluation loop

goal
execution evaluation
system

• user establishes the goal

• formulates intention
• specifies actions at interface
• executes action
• perceives system state
• interprets system state
• evaluates system state with respect to goal

12
Using Norman’s model
• Norman uses a simple example of switching on a light to illustrate this
cycle
• Imagine you are sitting reading as evening falls. You decide you need
more light; that is you establish the goal to get more light.
• From there you form an intention to switch on the desk lamp, and
• you specify the actions required, to reach over and press the lamp switch.
• If someone else is closer the intention may be different – you may ask
them to switch on the light for you.
• Your goal is the same but the intention and actions are different.
• When you have executed the action you perceive the result, either the
light is on or it isn’t and you interpret this, based on your knowledge of
the world.
13
… Using Norman’s model
• If the light does not come on you may interpret this as indicating the bulb has
blown or the lamp is not plugged into the mains, and
• you will formulate new goals to deal with this.
• If the light does come on, you will evaluate the new state according to the
original goals –
• is there now enough light? If so, the cycle is complete.
• Norman uses this model of interaction to demonstrate why some interfaces
cause problems to their users.
• He describes these in terms of the gulfs of execution and the gulfs of
evaluation.
• the user and the system do not use the same terms to describe the domain
and goals
• We call the language of the system the core language and the language of the
user the task language 14
… Using Norman’s model
• The gulf of execution is the difference between the user’s formulation
of the actions to reach the goal and the actions allowed by the
system.
• If the actions allowed by the system correspond to those intended by
the user, the interaction will be effective.
• The interface should therefore aim to reduce this gulf
• The gulf of evaluation is the distance between the physical
presentation of the system state and the expectation of the user
• If the user can readily evaluate the presentation in terms of his goal, the gulf
of evaluation is small.
• The more effort that is required on the part of the user to interpret the
presentation, the less effective the interaction.

15
… Using Norman’s model
Gulf of Execution
user’s formulation of actions
≠ actions allowed by the system

Gulf of Evaluation
user’s expectation of changed system state
≠ actual presentation of this state

16
Human error - slips and mistakes
slip
understand system and goal
correct formulation of action
incorrect action

mistake
may not even have right goal!

Fixing things?
slip – better interface design
mistake – better understanding of system

18
… Norman’s model
• Norman’s model is a useful means of understanding the interaction, in
a way that is clear and intuitive.
• It allows other, more detailed, empirical and analytic work to be
placed within a common framework.
• However, it only considers the system as far as the interface.
• It concentrates wholly on the user’s view of the interaction.
• It does not attempt to deal with the system’s communication through
the interface.
• An extension of Norman’s model, proposed by Abowd and Beale,
addresses this problem

19
The interaction framework
Abowd and Beale framework
• attempts a more realistic description of interaction by including the system
explicitly, and breaks it into four main components:
• extension of Norman… O
• the interaction framework has 4 parts output
– user
– input S U
– system core task
– output I
input

• Each component has its own language.

• In addition to the User’s task language and the System’s core language, there are
languages for both the Input and Output components.
• Input and Output together form the Interface. 20
… Abowd and Beale framework
• As the interface sits between the User and the System, there are four
steps in the interactive cycle,
• each corresponding to a translation from one component to another
• interaction  translation between languages
• problems in interaction = problems in translation
• The User begins the interactive cycle with
the formulation of a goal and a task
to achieve that goal (articulation) - input
• The input language is translated into the core
language as operations to be performed by
the System
• The System then transforms itself as described
Translations between components
by the operations
21
… Abowd and Beale framework
• the execution phase of the cycle is complete and the evaluation phase now
begins.
• The System is in a new state, which must now be communicated to the
User.
• The current values of system attributes are rendered as concepts or
features of the Output.
• It is then up to the User to observe the Output.
• There are four main translations involved in the interaction: articulation,
performance, presentation and observation
• The User’s formulation of the desired task to achieve some goal needs to be
articulated in the input language.
• The tasks are responses of the User and they need to be translated to
stimuli for the Input 22
Using Abowd & Beale’s model
user intentions
 translated into actions at the interface
 translated into alterations of system state
 reflected in the output display
 interpreted by the user

general framework for understanding interaction

– not restricted to electronic computer systems
– identifies all major components involved in interaction
– allows comparative assessment of systems
– an abstraction

23
24
ERGONOMICS
• Ergonomics (or human factors) is the study of the physical characteristics of
the interaction:
• how the controls are designed, the physical environment in which the interaction
takes place, and the layout and physical qualities of the screen.
• A primary focus is on user performance and how the interface enhances or
detracts from this.
• In seeking to evaluate these aspects of the interaction, ergonomics will
certainly also touch upon human psychology and system constraints
• It is a large and established field, which is closely related to but distinct from
HCI.
• Also known as human factors – but this can also be used to mean much of
HCI!
• Ergonomics good at defining standards and guidelines for constraining the
way we design certain aspects of systems 25
Ergonomics - examples
• arrangement of controls and displays
• Inappropriate placement of controls and displays can lead to inefficiency and frustration
e.g. controls grouped according to function or frequency of use, or
sequentially
• sequential controls and displays are organized to reflect the order of their use in a typical
interaction
• the most commonly used controls being the most easily accessible (frequency control)
• surrounding environment
e.g. seating arrangements adaptable to cope with all sizes of user
• the user should be able to reach all controls necessary and view all displays
without excessive body movement
• Critical displays should be at eye level.
• Lighting should be arranged to avoid glare and reflection distorting displays

26
• health issues
e.g. physical position, environmental conditions (temperature, humidity),
lighting, noise,
• Physical position: users should be able to reach all controls comfortably and see all
displays. Users should not be expected to stand for long periods and, if sitting, should be
provided with back support

• adequate lighting should be provided to allow users to see the computer screen without
discomfort or eyestrain
• use of colour
e.g. use of red for danger & warning, green for okay, and yellow for standby;
awareness of colour-blindness etc.
• Red is associated with danger and warnings in most western cultures, but
in China it symbolizes happiness and good fortune.
• The color of mourning is black in some cultures and white in others
27
Industrial interfaces
Office interface vs. industrial interface?

Context matters!
Office Industrial
Type of data Textual numeric
Rate of change Slow Fast
Environment Clean Dirty

… the oil soaked mouse!

• industrial interfaces raise some additional design issues rarely
encountered in the office

28
Interaction Styles
• Interaction can be seen as a dialog between the computer and the
user.
• The choice of interface style can have a profound effect on the nature
of this dialog.
• Common interaction styles:
• command line interface
• menus
• natural language
• question/answer and query dialogue
• form-fills and spreadsheets
• WIMP
• point and click
• three–dimensional interfaces 29
Command line interface
• The command line interface was the first interactive dialog style to be
commonly used,
• in spite of the availability of menu-driven interfaces, it is still widely used.
• It provides a means of expressing instructions to the computer
directly, using
• function keys, single characters, short abbreviations, whole words, or a
combination

Command line
interface

30
… Command line interface
• In some systems the command line is the only way of communicating
with the system, especially for remote access using telnet.
• More commonly today it is supplementary to menu-based interfaces,
• providing accelerated access to the system’s functionality for experienced
users.
• suitable for repetitive tasks
• better for expert users than novices
• offers direct access to system functionality
• command names/abbreviations should be meaningful!
Typical example: the Unix system

31
… Command line interface
• Command line interfaces are powerful in that they offer direct access to system
functionality (as opposed to the hierarchical nature of menus), and
• can be combined to apply a number of tools to the same data.
• They are also flexible: the command often has a number of options or
parameters that will vary its behavior in some way, and
• it can be applied to many objects at once, making it useful for repetitive tasks.
• However, this flexibility and power brings with it difficulty in use and learning.
• Commands must be remembered, as no cue is provided in the command line
to indicate which command is needed.
• can be alleviated a little by using consistent and meaningful commands and
abbreviations.
• The commands used should be terms within the vocabulary of the user rather
than the technician.
32
Menus
• In a menu-driven interface, the set of options available to the user is
displayed on the screen, and
• selected using the mouse, or numeric or alphabetic keys.
• Since the options are visible they are less demanding of the user,
relying on recognition rather than recall.
• However, menu options still need to be meaningful and logically
grouped to aid recognition.
• Often menus are hierarchically ordered and the option required is not
available at the top layer of the hierarchy.
• The grouping and naming of menu options then provides the only cue
for the user to find the required option.

33
… Menus
• Set of options displayed on the screen
• Options visible
– less recall - easier to use
– rely on recognition so names should be meaningful
• Selection by:
– numbers, letters, arrow keys, mouse
– combination (e.g. mouse plus accelerators)
• Often options hierarchically grouped
– sensible grouping is needed
• Restricted form of full WIMP system

Menu-driven interface

34
Natural language
• Familiar to user
• speech recognition or typed natural language
• Problems
– vague
– ambiguous
– hard to do well!
• Language is by nature vague and imprecise: this gives it its flexibility and allows
creativity in expression.
• Computers, on the other hand, require precise instructions.
• Solutions
– try to understand a subset
– pick on key words

35
Query interfaces
• Question/answer interfaces
– user led through interaction via series of questions (mainly with yes/no
responses, multiple choice, or codes) E.g Web based questionnaire
– These interfaces are easy to learn and use
– suitable for novice users but restricted functionality and power
– often used in information systems

• Query languages (e.g. SQL)

– used to to construct queries to retrieve information from database
– They use natural-language-style phrases, but in fact require specific
syntax, as well as knowledge of the database structure.
– Queries usually require the user to specify an attribute or attributes for
which to search the database
– requires understanding of database structure and language syntax, hence
requires some expertise
36
 Form-fills
• Primarily for data entry or data retrieval
• Screen like paper form.
• Data put in relevant place
• Requires
– good design
– obvious correction
facilities
• allow easy movement around the
form and allow some fields to be
left blank.
• require correction facilities, as users may change their minds or make a mistake about the
value that belongs in each field
• useful primarily for data entry applications and, it is easy to learn and use, for novice
users
• assuming a design that allows flexible entry, form filling is also appropriate for expert user37
Spreadsheets
• first spreadsheet VISICALC, followed by Lotus 1-2-3
• MS Excel is most common today
• sophisticated variation of form-filling.
– grid of cells contain a value or a formula
– formula can involve values of other cells
e.g. sum of all cells in this column
– user can enter and alter data, spreadsheet maintains
consistency

38
The WIMP interface
Windows
Icons
Menus
Pointers
• Currently many common environments for interactive computing are
examples of the WIMP interface style, often simply called windowing
systems.
• WIMP stands for windows, icons, menus and pointers (sometimes
windows, icons, mice and pull-down menus)
• It is the default style for majority of interactive computer
systems, especially PCs and desktop machines

39
Point and click interfaces
• used in ..
– multimedia
– web browsers
– hypertext
• just click something!
– icons, text links or location on map
• minimal typing
• This point-and-click interface style is obviously closely related to the WIMP style.
• It clearly overlaps in the use of buttons, but may also include other WIMP
elements.
• However, the philosophy is simpler and more closely tied to ideas of hypertext.
• the point-and-click style is not tied to mouse-based interfaces, and is also
extensively used in touchscreen information systems. 40
elements of the wimp interface
windows, icons, menus, pointers
+++
buttons, toolbars,
palettes, dialog boxes

• the four key features of the WIMP interface that give it its name –
windows, icons, pointers and menus.
• There are also many additional interaction objects and techniques
commonly used in WIMP interfaces, some designed for specific
purposes and others more general
• These include buttons, toolbars, palettes and dialog boxes.
41
… elements of the wimp interface
• Together, these elements of the WIMP interfaces are called widgets,
and they comprise the toolkit for interaction between user and
system.

Elements of the
WIMP interface

42
Windows
• Areas of the screen that behave as if they were independent
– can contain text or graphics
– can be moved or resized
– can overlap and obscure each other, or can be laid out next to one
another (tiled)

• scrollbars
– allow the user to move the contents of the window up and down or from
side to side
• title bars
– describe the name of the window

43
Icons
• Windows can be closed and lost for ever, or they can be shrunk to some very
reduced representation.
• A small picture is used to represent a closed window, and this representation is
known as an icon.
• small picture or image
• represents some object in the interface
• often a window or action

• windows can be closed down (iconised)

• Shrinking a window to its icon is known as iconifying the window
• small representation of many accessible windows
• icons can be many and various
• highly stylized
• realistic representations of the objects that they stand for. 44
• Variety of icons

45
Pointers
• important component
– WIMP style relies on pointing and selecting things
• uses mouse, trackpad, joystick, trackball, cursor keys or
keyboard shortcuts
• wide variety of graphical images

A variety of pointer
cursors

46
Menus
• Choice of operations or services offered on the screen
• Required option selected with pointer

File Edit Options Font

Typewriter
Pull-down Screen
menu Times

• problem – take a lot of screen space

• solution – pop-up: menu appears when needed

47
Kinds of Menus
• Menu Bar at top of screen (normally), menu drags down
– pull-down menu - mouse hold and drag down menu
– drop-down menu - mouse click reveals menu (Like Drop box)
– fall-down menus - mouse just moves over bar!

• Contextual menu appears where you are

– pop-up menus - actions for selected object (Like Right click- shortcut
menus)
– pie menus - arranged in a circle
• easier to select item (larger target area)
• quicker (same distance to any option)
… but not widely used!
• Menus are inefficient when they have too many items, and so cascading menus
are utilized, in which item selection opens up another menu adjacent to the item,
allowing refinement of the selection.
• Several layers of cascading menus can be used. 48
… Kinds of Menus
• Pull-down menus are dragged down from the title at the top of the
screen, by moving the mouse pointer into the title bar area and
pressing the button. (Shown in the previous figure)
• Fall-down menus are similar, except that the menu automatically
appears when the mouse pointer enters the title bar, without the user
having to press the button.
• Pop-up menus appear when a particular region of the screen, maybe
designated by an icon, is selected, but they only stay as long as the
mouse button is depressed.

49
… Kinds of Menus
• Arranging the options in circular fashion (Pie menus):
• The pointer appears in the center of the circle, and so there is the
same distance to travel to any of the selections.
• This has the advantages that it is easier to select items, since they can
each have a larger target area, and that the selection time for each
item is the same, since the pointer is equidistant from them all.
• Compare this with a standard menu: remembering Fitts’ law,
• we can see that it will take longer to select items near the bottom of the
menu than at the top in standard menu.
• However, these pie menus take up more screen space and are
therefore less common in interfaces.

50
Menus extras
• Cascading menus
– hierarchical menu structure
– menu selection opens new menu
– and so in ad infinitum

• Keyboard accelerators
– key combinations - same effect as menu item
– two kinds
• active when menu open – usually first letter
• active when menu closed – usually Ctrl + letter
usually different !!!

51
Menus design issues
• The major problems with menus in general are deciding what items to
include and how to group those items.
• In pull-down menus the menu label should be chosen to reflect the
function of the menu items, and items grouped within menus by
function.
• These groupings should be consistent across applications so that the
user can transfer learning to new applications.
• Menu items should be ordered in the menu according to importance
and frequency of use, and
• opposite functionalities (such as ‘save’ and ‘delete’) should be kept
apart to prevent accidental selection of the wrong function, with
potentially disastrous consequences.
52
… Menus design issues
• which kind to use
• what to include in menus at all
• words to use (action or description)
• how to group items
• choice of keyboard accelerators

53
Buttons & Toolbars
• individual and isolated regions within a display that can be selected to
invoke an action

• Special kinds
– radio buttons
– set of mutually exclusive choices
– check boxes
– set of non-exclusive choices
• Toolbars
• long lines of icons …
… but what do they do?
• fast access to common actions
• often customizable:
• choose which toolbars to see
• choose what options are on it 54
Design Focus

55
Dialogue boxes
• information windows that pop up to inform of an important event
or request information.

e.g: when saving a file, a dialogue box is displayed to allow the user to
specify the filename and location. Once the file is saved, the box
disappears.
- When deleting a file a dialogue box displayed for user’s confirmation that
helps to avoid accidental file deletion

56
 The Context of the Interaction
• Interaction between a user and a system is affected by interface design.
• One of the context in which the interaction takes place is the physical factors in
the environment that can directly affect the quality of the interaction
• Social and organizational factors affect the user’s interaction with the system

57
… Context
Interaction affected by social and organizational context

• The presence of other people in a work environment affects the performance of

the worker in any task:
• In the case of peers, competition increases performance, at least for known tasks.
• Similarly the desire to impress management and superiors improves performance on these
tasks
• When it comes to acquisition of new skills, the presence of these groups can inhibit
performance, owing to the fear of failure.
• privacy is important to allow users the opportunity to experiment
• motivation
– fear, allegiance, ambition, self-satisfaction
• inadequate systems
– cause frustration and lack of motivation
58
59
… Context
• the introduction of new technology may prove to be a motivation to
users, particularly if it is well designed, integrated with the user’s
current work, and challenging.
• Providing adequate feedback is an important source of motivation for
users.
• If no feedback is given during a session, the user may become bored,
unmotivated or, worse, unsure of whether the actions performed
have been successful.
• If system delays occur, feedback can be used to prevent frustration on
the part of the user

60
Experience, engagement and fun

designing experience
physical engagement
managing value

61
 Usability and User Experience Goals
• Traditionally, usability goals are concerned with meeting specific
usability criteria, such as efficiency,
• whereas user experience goals are concerned with explicating the
nature of the user experience, for instance, to be aesthetically
pleasing
• The distinction between the two types of goals is not clear-cut
• usability is often fundamental to the quality of the user experience
• Aspects of the user experience, such as how it feels and looks, are linked with
how usable the product is

62
Usability Goals
• Usability refers to ensuring that interactive products are easy to learn,
effective to use, and enjoyable from the person’s perspective.
• It involves optimizing the interactions people have with interactive
products to enable them to carry out their activities
• Usability is broken down into the following goals:
• Effective to use (effectiveness)
• Efficient to use (efficiency)
• Safe to use (safety)
• Having good utility (utility)
• Easy to learn (learnability)
• Easy to remember how to use (memorability)
• Enjoyable to use (satisfaction)
… Usability Goals
• Effectiveness is a general goal, and it refers to how good a product is at
doing what it is supposed to do.
• Question: Is the product capable of allowing people to carry out their work
efficiently, access the information that they need, or buy the goods that they
want?
• Efficiency refers to the way a product supports people in carrying out their
tasks.
• Once people have entered the necessary personal details in an online form
for a purchase, they can let the website/app save their personal details.
• Then, if they want to make another purchase at that site, they don’t have to re-enter
their personal details.
• Question: How many steps does it take to complete a task? How does
storing a person’s personal details make it more efficient?
… Usability Goals
• Safety involves protecting a person from dangerous conditions and
undesirable situations.
• the first ergonomic aspect w.r.t. safety refers to the external
conditions where people work.
• Example: where there are hazardous conditions—such as X-ray machines or
toxic chemicals—operators should be able to interact with and control
computer-based systems remotely.
• The second aspect refers to helping anyone in any kind of situation to
avoid the dangers of carrying out unwanted actions accidentally.
• Making interactive products safer in this sense involves:
• (1) preventing the user from making serious errors by reducing the
risk of wrong keys/buttons being mistakenly activated.
… Usability Goals (… Safety… )
• Example: not placing the quit or delete-file command right next to
the save command on a menu.
• (2) providing people with various means of recovery should they make
errors, such as an undo function
• Another safety mechanism is confirming dialog boxes that give users
another chance to consider their intentions
• Ex: the appearance of a dialog box after issuing the command to
delete everything in the trash, saying:
• “Are you sure you want to remove the items in the Trash permanently?”
… Usability Goals
• Utility refers to the extent to which the product provides the right
kind of functionality so that users can do what they need or want to
do
• An example of a product with low utility is a software drawing tool that does
not allow users to draw freehand but forces them to use a mouse to create
their drawings
• Learnability refers to how easy a product is to learn to use.
• People want to get started right away and become competent at carrying out
basic tasks without too much effort.
• Memorability refers to how easy a product is to remember how to
use, once learned.
• This is especially important for tasks and interactive products that are used
infrequently.
User Experience Goals
• Desirable and undesirable aspects of the user experience
• Undesirable aspects of the user experience
 Experience?
• home, entertainment, shopping
– not enough that people can use a system
– they must want to use it!

• psychology of experience
– flow (Csikszentimihalyi): looked at extreme experiences such as climbing a rock
face in order to understand that feeling of total engagement
– balance between anxiety and boredom
– If you do something that you know you can do it is not engaging; you may do it
automatically while thinking of something else, or you may simply become bored.
– If you do something completely outside your abilities you may become anxious.

• education
– zone of proximal development is those things that you cannot quite do yourself,
but you can do with some support, whether from teachers, fellow pupils, or
electronic or physical materials
– things you can just do with help

70
• Read the following sections from the text book (Exercise)
• Designing experience
• Physical design
• Design trade-offs

71