Human centered design week 3

Chapter 8: Data gathering

8.1 Introduction

8.2 Five key issues

8.3 Interviews

8.5 Questionnaires

8.6 Observation

8.7 choosing and combining techniques

8.1: Introduction

- What is data gathering?

 Gathering of any information, that is useful for understanding a particular
design, user needs, and user behavior.
- Data gathering can be quantitative or qualitative.
- Data gathering is very important to set the requirements of a design and the
evaluation
- 3 main techniques:
o Interviews: an interviewer asks questions to interviewees
o Questionnaires: a series of questions that can be answered
asynchronously
o Observation:
 Direct: spend time with individuals observing their activities
 Indirect: make a record of the user’s activity as it happens to be
studied at later date

8.2: Five key issues:

Five key issues require attention for any data gathering session to be successful:
goal setting, identifying participants, the relationship between the data collector and
the data provider, triangulation, and pilot studies.
1) Setting goals:
Many reasons to collect data, so important to identify exactly what data we
need, and why, by setting goals. Those will influence the nature of data
gathering sessions, techniques, and the analysis.
2) Identifying Participants
Once the goals are set, you need to identify the type of people from whom
data is to be gathered: (study) population
- Sampling: choosing participants to be included in data gathering ( for bigger
groups, you cannot consult everyone)
- Saturation sampling: all members of the target population are accessible
- Probability sampling:
o Apply statistical tests and generalize to the whole population
o By random sampling ( using a random number generator) ot stratified
sampling ( dividing population into groups and applying random
sampling)
- Nonprobability sampling:
o You cannot really generalize the results
o Convenience sampling (includes those who were available rather than
those specifically selected) or volunteer panels
- How many participants are needed for a study?
o Depends on the technique used, the kind of study, …
o Can use previous studies that are similar to get an idea
o There are also local standards

3) Relationship with participants

- Keep it professional contract, reassuring the person providing the data that
it will be used for its intended purpose and keep privacy
- The informed consent form is intended to protect the interests of both the data
gatherer and the data provider
- Make it advantageous for the people providing data so it is motivating.

4) Triangulation:
- Use of different investigations
- 4 types:
o Triangulation of data (different sources at different times…)
o Investigator triangulation (different investigators)
o Triangulation of theories
o Methodological triangulation (MOST COMMON)
- There are some limits to this data must be compatible to be able to be used
- Ground truth  other form of triangulation to check an original study

5) Pilot studies:
- Small trial of the main study to make sure that the proposed method is viable
before embarking on the real study
- Anyone involved in a pilot study cannot be involved in the main study itself
Conclusion: five steps, to make sure that the data we pick up is trustworthy and
reliable. You need to have a clear view of the problem and make sure that the way of
collecting data is suited

8.3: Data recording

Some forms of data gathering are self-documenting and no further recording is
necessary. But for others you might need to take notes, and other.
In most settings, audio recording, photographs, and notes will be sufficient. In others,
it is essential to collect video data so as to record in detail the intricacies of the
activity and its context.
3 common data recording:
1) Notes plus photographs
- Handwritten notes allow the researcher to be more concentrated on the data
and start thinking about it.

2) Audio plus photographs

- Useful alternative to note taking
- Less intrusive than video
- Allows the interviewer to be focused on the activity rather than taking notes
- Supplemented by photographs

3) Video
- Is practical but you need to find a way that it is not intrusive
Conclusion: different recording methods that are suited for different situations.
Careful to ethics and conditions of the experiment.

8.4: Interviews

4 main types of interview. Choose the most appropriate one based on the purpose of
the interview, etc.
1) Unstructured interviews
- Are open so no particular expectation, similar to a conversation
- The interviewer still needs a plan, but the format of the interview is not set
- PROS: generates rich data that provides a deep understanding of the topic
- CONS: time-consuming to analyze

2) Structured interviews
- Predetermined questions, that are all the same for each participant
- Similar to a questionnaire
 - Short and clearly worded, closed questions
- PROS: direct info that is clear to understand and use
- CONS: you need to have clear goals in order to ask specific questions
3) Semi-structured interviews
- combine features of structured and unstructured interviews
- the interviewer has a script, with closed questions but once they answer it
they can ask more about why they chose that answer
- PROS: collects more data than structured but it is easier to understand and
replicate than unstructured
- CONS: do not influence the interviewee
Interviews must be adapted to the user’s group (children or other)

4) Focus groups
- Interviews in groups
- Everybody gives their opinion
- Same guideline as for semi-structured interviews
- PROS: allows diverse or sensitive issues to be raised
- CONS: you need every person to express themselves and you do not want
them to be influenced by other participants
Conclusion: there are four types of interviews: you can either have an interview with
few guidelines and open possibilities that will allow you to collect a lot of data, or
have a structured interview that will work like a questionnaire but allows the
interviewer to look at body language, or have a semi structured interview that is a bit
of both. Lastly you can have group interviews, the interviewer needs to make sure
everyone is heard.

Planning and conducting an interview:

 Developing interview questions
o Open questions  goal of session is exploratory
o Closed questions  possible answers are known in advance
o Long or compound questions can be difficult to use  split them
o Do not use jargon, be straightforward
o Neutral questions
o When asking closed questions, you can prepare a list of question with
answers to be checked (like a questionnaire)
 Running the interview
Common sequence for an interview:
1) Intro: explains goals, reassures interviewees, etc
2) Warm up session with easy questions, can be about the person’s life
3) Main session
4) Cooling off period with easy questions
5) Closing, thanks
Other forms of interview:
If not face-to-face  remote interviews
PROS:
- Participants are more relaxed
- They don’t have to travel
- Anonymous, sense of security
CONS:
- Cannot see body-language
- Tend to multitask

Enriching the interview experience:

Neutral environment, interviewees could not feel at ease.
Solution: make them bring props, to illustrate what they want to say, or provide some
Conclusion: different kinds of interviews, they all need to be planned and the
interviewers need to keep in mind what the goals are for this interview. Once you
have prepared the interview you also want the interviewee to feel comfortable. If it is
not possible to have a face to face meeting, it can also be done remotely, which
presents both advantages and disadvantages.

8.5: Questionnaires:
Similar to interviews, as they can have closed or open questions, and once they are
produced they can be distributed to a large number of participants.
 More data collected
 But the questionnaire needs to be clear for the participant to understand
without additional help from an interviewer

Questionnaire structure:
Start: basic demographic information and details of relevant experience to put the
questionnaire responses into context
Advices:
 Ordering questions is important
 Different versions of the questionnaire might be needed
 Provide clear instructions
  Think about the length of the questionnaire, if it is long, allow them to
complete it in steps
 Think about layout, keep it compact but readable

Question and response format:

There are different formats:
1) Check boxes and ranges:
- Ranges for age, ( 15-20 or 21-25 …)
- The interval does not have to be equal
- Used for demographic questions

2) Rating scales:
- Good for getting people to make judgments
o Likert scales: identifying a set of statements representing a range of
possible opinions. Widely used for evaluating user satisfaction with
products
 Gather a pool of short statements about the subject
 Decide on the scale
 Select items for the final questionnaire

o Sematic differential scales: choosing pairs of words that represent the

range of possible opinions
- What scale to use?
o Many use five or seven
o Small scale people tend to chose the center
o Large scale difficult to pick up subtilities

Administering Questionnaires:
2 issues: reaching a representative sample and ensuring a reasonable response rate
Online questionnaires:
PROS:
- Interactive, help screens…
- Faster response rates
- Automatic transfer of responses into database
CONS:
- Difficult to obtain a random sample of respondents
- Results cannot be generalized
Steps to deploy online questionnaire:
1) Plan the survey timeline
 2) Design the questionnaire offline
3) Program the online survey
4) Test the survey
5) Recruit respondents
Do people answer online questionnaires differently than paper and pencil?
 Computer: more revealing and consistent
 People answer more and are more honest in online questionnaires
 Why? The way the information is structured, online questionnaires provide
more presenting information.
Conclusion: it is important that the questionnaire is easy to understand so that the
answers remain as accurate as possible. It also helps to get more information and
context about the respondents. The questionnaire can present open questions and
closed questions with check boxes or ranges. There are five steps to a successful
survey: plan the timeline, design the questionnaire offline, program the online survey,
test the survey, and recruit respondents.

8.6: Observation

1) Direct observation in the field

- Allows to get the true story

- Understanding context
- But it is a lot of data and it can be tedious to analyze
- Important to have a clear goal
- Careful balance between being guided by goals and being open to modifying

 Structuring frameworks for observation in the field:

o Capture the most important parts
o Keep the goals in mind
 Degree of participation
o It’s a spectrum
o Depends on the practical and ethical issues
o Passive observer = observer that is an outsider. More appropriate to
lab studies
o Participant observer = observer that is at the insider end of the
spectrum. They attempt to be a member of the group being studied. It
is important to keep the roles of insider and observer apart, to have an
objective view on what is happening. It is not always possible.
 Planning and conducting an observation in the field
 o There are a lot of factors to take in account before doing the study
( framework, level of participation, how to record data, acceptance in
the group…)
o How to ensure that the study uses different perspectives?
Work as a team:
 You focus on different parts
 More data is collected
 Different perspectives
 More objective conclusions
o Diary/ blog keep track of observation after each day to prevent
loosing data
 Ethnography
o uncover the organization of societies and their activities
o explain why and how people do what they do
o participant observer
o understanding behavior in certain environments
o online ethnography = virtual ethnography= netnography
is the study of social media and the various forms of computer-
mediated communication.
o You need to consider ethnical issues

2) Direct Observation in Controlled Environments

In laboratories. Can be portable.
Use of a script ensures that each participant will be treated in the same way.
Difference with field:
- Lab = emphasis on details
- Field = global understanding of interaction with others, technology and the
environment

 The think aloud technique

o In a controlled environment the observer can afford to be a little more
intrusive
o Understand what is going on in someone’s head
o Problem : silence moments where you cannot know what is going on
o When the task becomes difficult, people become silent. They can be
embarrassed etc.
o Solution to the silence: have people work together so that the
communication is more natural

3) Indirect Observation: tracking Users’ Activities

Sometimes activities need to be tracked indirectly, else it would be too intrusive.
  Diaries:
o Participants need to write in a diary on a daily basis
o Practical when the participants are scattered, if the study is personal
(emotions) or in a private environment (home)
o PROS: time efficient, does not require special equipment or expertise,
suited for long term studies.
o CONS: participants have to be reliable and remember to complete the
diaries at the assigned time and as instructed>
It is tricky to determine how long the study should be
Memories can be exaggerated and details can be forgotten
 Interaction logs, web analytics, and Data scraping
o Uses software to record user’s activity
o Ethical concerns
o Large volumes of data can be logged automatically

8.7: Choosing and Combining Techniques

 Depends on a variety of factors related to the study goals

 Influenced by practical issues

CONCLUSION:
1) five steps, to make sure that the data we pick up is trustworthy and reliable.
You need to have a clear view of the problem and make sure that the way of
collecting data is suited

2) different recording methods that are suited for different situations. Careful to
ethics and conditions of the experiment

3) there are four types of interviews: you can either have an interview with few
guidelines and open possibilities that will allow you to collect a lot of data, or
have a structured interview that will work like a questionnaire but allows the
interviewer to look at body language, or have a semi structured interview that
is a bit of both. Lastly you can have group interviews, the interviewer needs to
make sure everyone is heard.
 different kinds of interviews, they all need to be planned and the interviewers
need to keep in mind what the goals are for this interview. Once you have
prepared the interview you also want the interviewee to feel comfortable. If it
is not possible to have a face to face meeting, it can also be done remotely,
which presents both advantages and disadvantages
 4) it is important that the questionnaire is easy to understand so that the answers
remain as accurate as possible. It also helps to get more information and
context about the respondents. The questionnaire can present open questions
and closed questions with check boxes or ranges. There are five steps to a
successful survey: plan the timeline, design the questionnaire offline, program
the online survey, test the survey, and recruit respondents

Chapter 9: Data analysis, interpretation and presentation

9.1: Introduction:

Kind of analysis depends on the goals and data gathered

Data cleansing looking for errors before interpreting data
COMMON MISTAKES:
 investigator’s existing beliefs or biases influence the interpretation
 make claims that go beyond data

9.2: Quantitative and Qualitative

Quantitative: data in the form of numbers, quantitative analysis uses numerical

methods to ascertain the magnitude, amount or size of something
Qualitative: data in the form of words, images, descriptions. Qualitative analysis
focuses on the nature of something, and can be represented in patterns, stories…

! Careful when using numbers it must be relevant and useful

- you cannot translate small populations into percentages
- don’t transform qualitative data into quantitative if it is not necessary

1) First steps in analysing data

o Interviews:
  Write down all the notes right after the interview so it is still fresh
and clear
 Transcription takes effort select information you want to keep,
revisiting goals can help this selection
 Closed ended questions = quantitative
Open ended questions = qualitative

o Questionnaires:
 Data can easily be analyzed using computers
o Observation:
 Transcriptions and the observer’s notes are most likely to be
analyzed using qualitative approaches, while photographs
provide contextual information.

9.4: Basic qualitative analysis

3 basic approaches:
 Identifying themes
 Categorizing data
 Analyzing critical incidents
First step gain an overall impression of the data and start looking for anything that
stands out
1) Identifying themes:
o Aims to identify, analyze and report patterns in data
o Theme = something important about data in relation to the study goal
o When you identify a theme you always need to double check it to see if
it is relevant
o depending on the study goal, you can identify major and minor themes,
and when looking at the bigger picture identify an overarching theme
o  get an overall picture of data
o Affinity diagram: The approach seeks to organize individual ideas and
insights into a hierarchy showing common structures and themes. It is
used a lot in contextual design and interaction design. Uses sticky
notes
2) Categorizing data:
o Can be deductive or inductive if the goals lead the categorization.
o Results can be quantified
o Make a list of possible problems, sorted in categories and then pick up
arguments in the recordings to support those problems
3) Critical incident analysis:
o Helps identify significant subsets of the data for more detailed analysis.
o First principle: reporting facts regarding behavior is preferable to the
collection of interpretations, ratings, and opinions based on general
opinions. >> use of well planned sessions
o Second principle: reporting should be limited to those behaviors which
make a significant contribution to the activity. >> critical incidents