Scribd
Upload
160 views19 pages

Human Computer Interaction Evaluation Through User Participation

The document discusses various methods for evaluating user interfaces through user participation, including controlled experiments, observational methods, query techniques, and physiological measurements. Controlled experiments manipulate independent variables and measure effects on dependent variables to test hypotheses. Observational methods include think-aloud protocols, cooperative evaluation, and post-task walkthroughs. Query techniques are interviews and questionnaires. Physiological methods such as eye tracking and sensors can measure physical responses.

Uploaded by

vicky aroora
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
160 views19 pages

Human Computer Interaction Evaluation Through User Participation

The document discusses various methods for evaluating user interfaces through user participation, including controlled experiments, observational methods, query techniques, and physiological measurements. Controlled experiments manipulate independent variables and measure effects on dependent variables to test hypotheses. Observational methods include think-aloud protocols, cooperative evaluation, and post-task walkthroughs. Query techniques are interviews and questionnaires. Physiological methods such as eye tracking and sensors can measure physical responses.

Uploaded by

vicky aroora
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 19

Human Computer Interaction

Lecture 18

Evaluation Through User


Participation
- Controlled evaluation of specific aspects of interactive
behaviour

- Evaluator chooses hypothesis to be tested

- A number of experimental conditions are considered


which differ only in the value of some controlled variable.

- Changes in behavioural measure are attributed to


different conditions
Experimental factors
Participants
Users – representative, sufficient sample
Variables
Things to modify and measure
Hypothesis
What you’d like to show
Experimental design
How you are going to do it
Participants

- Participants should be chosen to match the expected user


population as closely as possible.

-If participants are not actual users they should be a similar


age, education or profession.

- Sample size must be large enough to be considered to


represent the actual user population.
Variables

- independent variable (IV)


characteristic changed to produce different conditions
e.g. interface style, number of menu items etc.

- dependent variable (DV)


characteristics measured in the experiment
e.g. time taken, number of errors etc.
Hypothesis
- Prediction of outcome
States that a variation in the independent variables will
cause a difference in the dependent variables.

e.g. “error rate will increase as font size decreases”

- Null hypothesis:
- states no difference between conditions
- aim is to disprove this

e.g. null hypothesis. = “no change with font size”


Observational Methods

Think Aloud
Cooperative evaluation
Protocol analysis
Post-task walkthroughs
Think Aloud
- user observed performing task
- user asked to describe what he is doing and why, what he
thinks is happening etc.

Advantages
- simplicity - requires little expertise
- can show how system is actually being used
Disadvantages
-Subjective
Cooperative evaluation
- variation on think aloud
- user collaborates in evaluation
- both user and evaluator can ask each other questions
throughout

Additional advantages
- less constrained and easier to use
- user is encouraged to criticize system
- clarification possible
Protocol Analysis
Methods for recording user actions including:
- paper and pencil – cheap, limited to writing speed
- audio – good for think aloud, difficult to match with other protocols
- video – accurate and realistic, needs special equipment.
- computer logging – automatic and modest, large amounts of data
difficult to analyze
- user notebooks – records every few minutes or hourly, good for
longitudinal studies
- Mixed use in practice.
audio/video transcription difficult and requires skill.
Post-Task walkthroughs
- user reacts on action after the event

Advantages
- analyst has time to focus on relevant incidents
-avoid excessive interruption of task

Disadvantages
- lack of freshness
Query Techniques
Interviews
Questionnaires
Interviews
• analyst questions user on one-to-one basis
usually based on prepared questions
• Informal and relatively cheap

• Advantages
– can be varied to suit context
– issues can be explored more fully
– can extract user views and identify unexpected problems
• Disadvantages
– time consuming
Questionnaires
• Set of fixed questions given to users

• Advantages
– quick and reaches large user group
• Disadvantages
– less flexible
– less questioning
Questionnaires (ctd)
• Need careful design
– what information is required?
– how are answers to be analyzed?

• Styles of question
– general
– open-ended
– multi-choice
– ranked
Physiological methods
Eye tracking
Physiological measurement
Eye Tracking
• head or desk mounted equipment tracks the position of the
eye
• eye movement reflects the amount of cognitive processing a
display requires
• measurements include
– fixations: eye maintains stable position. Number and duration indicate
level of difficulty with display
– saccades: rapid eye movement from one point of interest to another
– scan paths: moving straight to a target with a short fixation at the
target is optimal
• Still a lot of research is needed in this area
Physiological Measurements
• emotional response is linked to physical changes
• these may help determine a user’s reaction to an
interface
• Measurements are made by attaching various sensors to
users
Physiological Measurements
• Measurements include:
– heart activity, including blood pressure and pulse.
• May represent stress or anger
– activity of sweat glands: indicated by skin resistance
• Represent mental effort needed
– electrical activity in muscle: measured by electromyogram (EMG)
• Reflect involvement in a task
– electrical activity in brain: electroencephalogram (EEG)
• Reflect decision making and motivation
• Some difficulty in interpreting these physiological
responses
– Increased pulse rate <- (frustration with the interface OR Stress at
being unable to complete the task)
• More research needed

You might also like