Methods of data

collection
Mekdes W. (MPH)

1 Mekdes W.(Epid)
 Introduction
2

 Data Collection Strategies

 Quantitative and Qualitative Data
 Tools for Collecting Data

Mekdes W.(Epid)
 Data Collection Strategies
3

 No one best way: decision depends on:

 What you need to know: numbers or stories
 Where the data reside: environment, files,
people
 Resources and time available
 Complexity of the data to be collected
 Frequency of data collection
 Intended forms of data analysis

Mekdes W.(Epid)
 Rules for Collecting Data
4

 Use multiple data collection methods

 Use available data, but need to know
 how the measures were defined
 how the data were collected and cleaned
 the extent of missing data
 how accuracy of the data was ensured

Mekdes W.(Epid)
 Rules for Collecting Data
5

 If must collect original data:

 be sensitive to burden on others
 Conduct pretest
 establish procedures and follow them
(protocol)
 maintain accurate records of definitions
and coding
 verify accuracy of coding, data input

Mekdes W.(Epid)
 Source of data collection
1 Primary data: data measured or collected by the
investigator or the user directly from the source.
Primary sources are sources that can supply first hand

information for immediate user.

2 Secondary data: When an investigator uses data, which

have already been collected by others, such data are
called secondary data.
Data gathered or compiled from published and

unpublished sources
 Approaches in data
7
collection
 Generally there are two approaches of
data collection. These are
 Quantitative approach
 Qualitative approach

Mekdes W.(Epid)
 Quantitative Approach
8

 Data in numerical form

 Data that can be precisely measured
 age, cost, length, height, area, volume,
weight, speed, time, and temperature
 Harder to develop
 Easier to analyze

Mekdes W.(Epid)
 Qualitative Approach
9

 Data that deal with description

 Data that can be observed or self-
reported, but not always precisely
measured
 Less structured, easier to develop
 Can provide “rich data” — detailed and
widely applicable
 Is challenging to analyze
 Is labor intensive to collect
 Usually generates longer reports
Mekdes W.(Epid)
 Which Data?
10

If you: Then Use:

- want to conduct statistical analysis
- want to be precise Quantitati
- know what you want to measure
ve
- want to cover a large group
- want narrative or in-depth information
- are not sure what you are able to measure Qualitati
- do not need to quantify the results ve

Mekdes W.(Epid)
 How to Decide on Data
11
Collection Approach

 Choice depends on the situation

 Each technique is more appropriate in
some situations than others
 Caution: All techniques are subject to
bias

Mekdes W.(Epid)
 Triangulation to Increase
12
Accuracy of Data

 Triangulation of methods
 collection of same information using
different methods
 Triangulation of sources
 collection of same information from a
variety of sources
 Triangulation of evaluators
 collection of same information from more
than one evaluator
Mekdes W.(Epid)
 Data Collection Tools
13

 Participatory Methods
 Records and Secondary Data
 Observation
 Surveys/questionnaires
 Interviews
 Focus Groups
 Diaries, Journals, Self-reported
Checklists
 Other Tools
Mekdes W.(Epid)
 Tool 1: Participatory
14
Methods
 Involve groups or communities heavily in
data collection
 Examples:
 community meetings
 mapping
 transect walks

Mekdes W.(Epid)
 Community Meetings
15

 One of the most common participatory

methods
 Must be well organized
 agree on purpose
 establish ground rules
 who will speak
 time allotted for speakers
 format for questions and answers

Mekdes W.(Epid)
 Mapping
16

 Drawing or using existing maps

 Useful tool to involve stakeholders
 increases understanding of the
community
 generates discussions, verifies secondary
sources of information, perceived changes
 Types of mapping:
 natural resources, social, health,
individual or civic assets, wealth, land use,
demographics
Mekdes W.(Epid)
 Transect Walks
17

 Evaluator walks around community

observing people, surroundings, and
resources
 Need good observation skills
 Walk a transect line through a map of a
community — line should go through all
zones of the community

Mekdes W.(Epid)
 Tool 2: Records and Secondary
18
Data
 Examples of sources:
 files/records
 computer data bases
 industry or government reports
 other reports or prior evaluations
 census data and household survey data
 electronic mailing lists and discussion groups
 documents (budgets, organizational charts,
policies and procedures, maps, monitoring
reports)
 newspapers and television reports
Mekdes W.(Epid)
 Using Existing Data Sets
19

Key issues: validity, reliability,

accuracy, response rates, data
dictionaries, and missing data rates

Mekdes W.(Epid)
 Advantage/Challenge:
20
Available Data
Advantages
•Often less expensive
•Fast to collect
Challenges • There may be coding
errors or other problems.
• Data may be incomplete
• Difficult to access
• Less accurate, reliable,
valid
Mekdes W.(Epid)
 Tool 3: Observation
21

 See what is happening

 traffic patterns
 land use patterns
 layout of city and rural areas
 quality of housing
 condition of roads
 conditions of buildings
 who goes to a health clinic

Mekdes W.(Epid)
 Observation is Helpful
22
when:
 need direct information
 trying to understand ongoing behavior
 there is physical evidence, products, or
outputs than can be observed
 need to provide alternative when other
data collection is infeasible or
inappropriate

Mekdes W.(Epid)
 Ways to Record Information from
Observations
23

 Observation guide
 printed form with space to record
 Recording sheet or checklist
 Yes/no options; tallies, rating scales
 Field notes
 least structured, recorded in narrative,
descriptive style

Mekdes W.(Epid)
 Guidelines for Planning
24
Observations
 Have more than one observer, if
feasible
 Train observers so they observe the

same things
 Pilot test the observation data

collection instrument
 For less structured approach, have a

few key questions in mind

Mekdes W.(Epid)
 Advantages and Challenges:
25
Observation

Advantage Challenges
 Collects data on • Observer bias
actual behavior or • Potentially unreliable
perceptions • Interpretation and
coding challenges
• sampling can be a
problem
• Cost and time taking

Mekdes W.(Epid)
 Tool 4: Surveys
26

 Excellent for asking people about:

 perceptions, opinions, ideas
 Less accurate for measuring behavior
 Sample should be representative of the
whole

Mekdes W.(Epid)
 Why we use surveys?
27

 Cheap & effective way of collecting data

 It can be simple & very focused
 Can collect vast quantities of data from
variety of respondents
 Relatively little training to develop
 Produce rich data that can easily & quickly
be analysed

Mekdes W.(Epid)
 When not to use
28
surveys/questionnaire?
 Little knowledge
 Type of the variables is difficult to quantify

 Lack of adequate time:

 Questionnaires compete for respondents time
 Lack of adequate time to complete the instrument may result
in the return of superficial data
 Low response rate due to lack of personal
contact:
 Lack of personal contact (if questionnaire is mailed) may
mean that response rates suffer, requiring the expense of
follow up letters, telephone calls & other means of chasing
the respondent
Mekdes W.(Epid)
29
Structures for Surveys
 Structured:
 Precisely worded with a range of pre-
determined responses that the respondent
can select
 Everyone asked exactly the same questions
in exactly the same way, given exactly the
same choices
 Semi-structured
 Asks same general set of questions but
answers to the questions are predominantly
open-ended
Mekdes W.(Epid)
 Types of questions
30

 Closed questions
– Dichotomous responses (yes/no), and
– Multiple-choice questions:
• All possible answers covered (and include other)
• Ordinal responses (Likert-scale)
 Open-ended questions
– No restrictions of answers
• But, can make analysis more difficult
• Must be carefully coded

Mekdes W.(Epid)
 Closed Vs open
31

 Closed questions are usually better:

– Easier for the respondent

– Less coding is required later

– Categories help define the question

 Disadvantage of close ended:

• Categories may be ‘leading’ to respondents
• May make it too easy to answer without thinking
• Not best when:
– asking for frequency of sensitive behaviors
– there are numerous possible responses

Mekdes W.(Epid)
Rules/Requirements for questionnaire development

1. Start with easy questions that all respondents can answer

with little effort

2. Avoid leading questions

3. Avoid sensitive or touchy questions

4. Avoid negatively phrased items

5. Should be simple & specific (stick to one idea)

6. Questions should be arranged in logical sequences

7. Must be in the language respondents understand well

32
 Modes of Survey
33
Administration
 Telephone surveys
 Self-administered questionnaires
distributed by mail, e-mail, or websites
 Administered questionnaires

Mekdes W.(Epid)
34
Tool 5:Interviews
 Often semi-structured
 Used to explore complex issues in
depth
 Can provide evaluators with an intuitive
sense of the situation

Mekdes W.(Epid)
 Challenges of Interviews
35

 Can be expensive, labor intensive, and

time consuming
 Selective hearing on the part of the
interviewer may miss information that
does not conform to pre-existing beliefs
 Cultural sensitivity: e.g., gender issues

Mekdes W.(Epid)
 Tool 6: Focus Groups
36

 Type of qualitative research where small

homogenous groups of people are
brought together to informally discuss
specific topics under the guidance of a
moderator
 Purpose: to identify issues and themes,
not just interesting information, and not
“counts”

Mekdes W.(Epid)
 Focus Groups Are
37
Inappropriate when:
 language barriers are insurmountable
 evaluator has little control over the
situation
 trust cannot be established
 free expression cannot be ensured
 confidentiality cannot be assured

Mekdes W.(Epid)
 Focus Group Process
38

Phase Action
1 Openin Ice-breaker; explain purpose; ground rules;
g
introductions
2 Warm- Relate experience; stimulate group
up interaction; start with least threatening and
simplest questions
3 Main Move to more threatening or sensitive and
body complex questions; elicit deep responses;
connect emergent data to complex, broad
participation
4 Closur End with closure-type questions; summarize
e and refine; present theories, etc; invite final
Mekdes W.(Epid)
comments or insights; thank participants
 Advantages and Challenges of
39
Focus Groups

Advantage Can be conducted relatively quickly and

s easily; may take less staff time than in-
depth, in-person interviews; allow
flexibility to make changes in process
and questions; can explore different
perspectives; can be fun
Challenge Analysis is time consuming; participants
s not be representative of population,
possibly biasing the data; group may be
influenced by moderator or dominant
group members
Mekdes W.(Epid)
 Tool 7: Diaries and Self-Reported
Checklists
40

 Use when you want to capture

information about events in people’s
daily lives
 Participants capture experiences in real-
time not later in a questionnaire
 Used to supplement other data collection

Mekdes W.(Epid)
 Other Measurement Tools
41

- scales (weight)
- tape measure - health testing
- stop watches tools:
i.e. blood pressure
- chemical tests :
- aptitude and
i.e. quality of
water achievement
tests
-citizen report
cards
Mekdes W.(Epid)
 Data Collection Summary
42

 Choose more than one data collection

technique
 No “best” tool

 “Do not let the tool drive your work but

rather choose the right tool to address the
evaluation question”
Mekdes W.(Epid)
 Question
s?

43 Mekdes W.(Epid)