ITE 254

Human-Computer Interaction
A. Mintra Ruensuk
[email protected]
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Emotions and the user experience

• HCI has traditionally been about designing efficient
and effective systems

• Now more about how to design interactive systems

that make people respond in certain ways

– e.g. to be happy, to be trusting, to learn, to be

motivated

• Emotional interaction is concerned with how we

feel and react when interacting with technologies
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Is this form fun to fill in?

• “My goal was to
design Wufoo to feel
like something Fisher-
Price would make.”
- Kevin Hale, Wufoo
director
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Emotional interaction
• What makes us happy, sad, annoyed, anxious, frustrated,
motivated, delirious and so on

– translating this into different aspects of the user experience

• Why people become emotionally attached to certain products

(e.g. virtual pets)

• Can social robots help reduce loneliness and improve

wellbeing?

• How to change human behavior through the use of emotive

feedback
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Activity
• Try to remember the emotions you went through
when buying a big ticket item online (e.g. a fridge,
a vacation, a computer)

• How many different emotions did you go through?

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Emotional design model

• Norman, Ortony and Revelle (2004) model of
emotion
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Claims from model

• Our emotional state changes how we think

– when frightened or angry we focus narrowly and

body responds by tensing muscles and sweating

• more likely to be less tolerant

– when happy we are less focused and the body

relaxes

• more likely to overlook minor problems and be

more creative
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Activity
• Do you feel more creative when you are in a happy
mood?

• Do you get less work done when you are feeling

stressed?
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Expressive interfaces
• Provide reassuring feedback that can be both informative and fun

• But can also be intrusive, causing people to get annoyed and even
angry

• Color, icons, sounds, graphical elements and animations are used

to make the ‘look and feel’ of an interface appealing

– conveys an emotional state

• In turn this can affect the usability of an interface

– people are prepared to put up with certain aspects of an

interface (e.g. slow download rate) if the end result is appealing
and aesthetic
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Friendly interfaces
• Microsoft pioneered friendly interfaces for
technophobes - ‘At home with Bob’ software

• 3D metaphors based on familiar places (e.g. living

rooms)

• Agents in the guise of pets (e.g. bunny, dog) were

included to talk to the user

– Make users feel more at ease and comfortable

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Bob
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Clippy
• Why was Clippy
disliked by so many

• Was it annoying, 

distracting,

patronising or other?

• What sort of user 


liked Clippy?
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Frustrating interfaces
• Many causes:
– When an application doesn’t work properly or crashes

– When a system doesn’t do what the user wants it to do

– When a user’s expectations are not met

– When a system does not provide sufficient information to enable the user to
know what to do

– When error messages pop up that are vague, obtuse or condemning

– When the appearance of an interface is garish, noisy, gimmicky or patronizing

– When a system requires users to carry out too many steps to perform a task,
only to discover a mistake was made earlier and they need to start all over
again
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Gimmicks
• Amusing to the designer but not the user, e.g.

• Clicking on a link to a website only to discover that

it is still ‘under construction’
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Error messages
• “The application Word Wonder has unexpectedly quit
due to a type 2 error.”

• Why not instead: “the application has expectedly quit due

to poor coding in the operating system”

• Shneiderman’s guidelines for error messages include:

• avoid using terms like FATAL, INVALID, BAD
• Audio warnings
• Avoid UPPERCASE and long code numbers
• Messages should be precise rather than vague
• Provide context-sensitive help
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Website error messages

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Should computers say they’re sorry?

• Reeves and Naas (1996) argue that computers should be made
to apologize

• Should emulate human etiquette

• Would users be as forgiving of computers saying sorry as

people are of each other when saying sorry?

• How sincere would they think the computer was being? For
example, after a system crash:

– “I’m really sorry I crashed. I’ll try not to do it again”

• How else should computers communicate with users?

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Detecting emotions and emotional technology

• Sensing technologies used to measure GSR, facial

expressions, gestures, body movement

• Aim is to predict user’s emotions and aspects of

their behavior –

• E.g. what is someone most likely to buy online

when feeling sad, bored or happy
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Facial Coding
• Measures a user’s emotions as they interact with a
computer or tablet

• Analyses images captured by a webcam of their

face

• Uses this to gauge how engaged the user is when

looking at movies, online shopping sites and ads

• 6 core expressions - sadness, happiness, disgust,

fear, surprise and anger
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Facial Coding
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

How to use the emotional data?

• If user screws up their face when an ad pops up ->
feel disgust

• If start smiling -> they are feeling happy

• Website can adapt its ad, movie storyline or

content to match user’s emotional state

• Eye-tracking, finger pulse, speech and words/

phrases also analyzed when tweeting or posting to
Facebook
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Indirect emotion detection

• Beginning to be used more to infer or predict
someone’s behavior

• For example, determining a person’s suitability for a

job, or how they will vote at an election

• Do you think it is creepy that technology can read

your emotions from your facial expressions or from
your tweets?
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Persuasive technologies and behavioral

change
• Interactive computing systems deliberately
designed to change people’s attitudes and
behaviors (Fogg, 2003)

• A diversity of techniques now used to change what

they do or think
• Pop-up ads, warning messages, reminders, prompts,
personalized messages, recommendations, Amazon 1-click

• Commonly referred to as nudging

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Nintendo’s Pocket Pikachu

• Changing bad habits and improving well being
– Designed to motivate children to be more physically active on a
regular basis

– owner of the digital pet that ‘lives’ in the device is required to

walk, run, or jump

– If owner does not exercise the virtual pet becomes angry and
refuses to play anymore
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

How effective?
• Is the use of novel forms of interactive technologies
(e.g., the combination of sensors and dynamically
updated information) that monitor, nag, or send
personalized messages intermittently to a person
more effective at changing a person’s behavior
than non-interactive methods, such as the
placement of warning signs, labels, or ads in
prominent positions?
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Which is most effective?

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Tracking devices
• Mobile apps designed to help people monitor and
change their behavior (e.g. fitness, sleeping,
weight)

• Can compare with online leader boards and charts,

to show how they have done in relation to their
peers and friends

• Also apps that encourage reflection that in turn

increase well-being and happiness
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Energy reduction
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

The Tidy Street project

• large-scale visualization of the street’s electricity
usage
• stenciled display on the road surface using chalk

• provided realtime feedback that all could see change each day

• reduced electricity consumption by 15%

ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Phishing and trust

• Web used to deceive people into parting with
personal details e.g. Paypal, eBay and won the
lottery letters

• Allows Internet fraudsters to access their bank

accounts and draw money from them

• Many vulnerable people fall for it

• The art of deception is centuries old but internet

allows ever more ingenious ways to trick people
ITE254: Human-Computer Interaction
Week8: Emotional Interaction

Some interesting researches

• https://www.youtube.com/watch?
v=1gJA3NkmlUY&feature=youtu.be

• https://www.youtube.com/watch?v=eP6sW2-Hwdg

• https://www.youtube.com/watch?v=Q5ujdXhFGSY

• https://www.psych.ox.ac.uk/research/csr

• https://skybiometry.com