Introduction To Data Analysis

What is Data?
⁺ Data is raw information.
⁺ Extraction of meaningful data from raw
data is termed as information.
⁺ Raw data is collected as a part of research,
observations and surveys.
Why data is required?
What we want to know?

drives

Collection of data
What is Statistics?
⁺ Statistics is the study of how
to collect, organise, analyse,
Statistics and interpret information.
⁺ Statistics is a tool for
converting data into
information.
Data Information
What is Data Analysis?
⁺ Data analysis is defined as a
process of collecting, modeling,
and analyzing data to extract
insights that support decision-
making.

⁺ The purpose of data analysis is to extract useful information

from data and taking the decision based upon the data
analysis.
 Data Analysis Process

Collecting Storage Processing Describing Modeling

Data Data Data Data Data
Data Collection
⁺ Data Collection is the process of gathering information on
targeted variables identified as data requirements.
⁺ Data Collection ensures that data gathered is accurate such
that the related decisions are valid.
⁺ Data is collected from various sources ranging from
organizational databases to the information in web pages.
⁺ The data thus obtained, may not be structured and may
contain irrelevant information.
Data Collection Strategies
No one best way: decision depends on:
What you need to know: numbers or stories.
Where the data reside: environment, files, people.
Resources and time available.
Complexity of the data to be collected.
Frequency of data collection.
Intended forms of data analysis.
Rules for Collecting Data
Use multiple data collection methods.
Use available data, but need to know
How the measures were defined?
How the data were collected and cleaned ?
 The extent of missing data.
How accuracy of the data was ensured?
Rules for Collecting Data
If must collect original data:
Establish procedures and follow them (protocol).
Maintain accurate records of definitions and coding.
Verify accuracy of coding, data input.
Structured Approach
All data collected in the same way.
Especially important for multi-site and
cluster evaluations so you can compare.
Important when you need to make
comparisons with alternate interventions.
When to use Structured Approach ?
Need to address extent questions.
Have a large sample or population.
Know what needs to be measured.
Need to show results numerically.
Need to make comparisons across different sites or
interventions.
Semi-structured Approach
Systematic and follow general procedures but data are
not collected in exactly the same way every time.
More open and fluid.
Does not follow a rigid script.
May ask for more detail.
People can tell what they want in their own way.
When to use Semi-structured Approach ?
Conducting exploratory work.
Seeking understanding, themes, and/or issues.
Need narratives or stories.
Want in-depth, rich, “backstage” information.
Seek to understand results of data that are
unexpected.
Characteristics of Good Measures
Is the measure relevant?
Is the measure credible?
Is the measure valid?
Is the measure reliable?
Relevance
Does the measure Do not measure
capture what matters? what is easy instead
of what is needed?
Credibility
 Is the measure believable? Will it
be viewed as a reasonable and
appropriate way to capture the
information sought?
Internal Validity

 How well does the Are waiting lists a

measure capture what valid measure of
it is supposed to? demand?
Reliability
 A measure’s precision How reliable are:
and stability- extent to • Birth weights of
which the same result newborn infants?
would be obtained with • Speeds measured by a
repeated trials. stopwatch?
Quantitative Approach
Data in numerical form.
Data that can be precisely measured
age, cost, length, height, area, volume, weight,
speed, time, and temperature
Harder to develop.
Easier to analyze.
Qualitative Approach
Data that deal with description.
Data that can be observed or self-reported, but not always
precisely measured.
Less structured, easier to develop.
Can provide “rich data” — detailed and widely applicable.
Is challenging to analyze.
Is labor intensive to collect.
Usually generates longer reports.
Obtrusive vs. Unobtrusive Methods
Obtrusive Unobtrusive
Data collection methods that Data collection methods that
directly obtain information do not collect information
from those being evaluated. directly from evaluees.

Example interviews, surveys, Example document analysis,

focus groups Google Earth, observation at
a distance, trash of the stars
How to Decide on Data Collection Approach?
Choice depends on the
situation.
Each technique is more
appropriate in some situations
than others.
Caution: All techniques are
subject to bias.
Triangulation to Increase Accuracy of Data
Triangulation of methods
collection of same information
using different methods.
Triangulation of sources
collection of same information
from a variety of sources.
Triangulation of evaluators
collection of same information
from more than one evaluator.
Data Collection Tools
Participatory Methods
Records and Secondary Data
Observation
Surveys and Interviews
Focus Groups
Diaries, Journals, Self-reported Checklists
Expert Judgment
Delphi Technique
Other Tools
Tool 1: Participatory Methods
Involve groups or communities
heavily in data collection.
Examples:
Community meetings
Mapping
Transect walks
Community Meetings
One of the most common
participatory methods.
Must be well organized.
Agree on purpose.
Establish ground rules.
Who will speak?
Time allotted for speakers.
Format for questions and
answers.
Mapping
Drawing or using existing maps.
Useful tool to involve stakeholders.
Increases understanding of the community.
Generates discussions, verifies secondary sources of
information, perceived changes.
Types of mapping:
Natural resources, social, health, individual or civic assets,
wealth, land use, demographics.
Transect Walks
Evaluator walks around community observing
people, surroundings, and resources.
Need good observation skills.
Walk a transect line through a map of a
community — line should go through all zones of
the community.
Tool 2: Records and Secondary Data
Examples of sources:
Files/records
Computer data bases
Industry or government reports
Other reports or prior evaluations
Census data and household survey data
Electronic mailing lists and discussion groups
Documents (budgets, organizational charts, policies and
procedures, maps, monitoring reports)
Newspapers and television reports
Using Existing Data Sets
 Key issues: Validity,
reliability, accuracy,
response rates, data
dictionaries, and
missing data rates.
Advantage/Challenge: Available Data
Advantages Often less expensive and faster than
collecting the original data again

Challenges There may be coding errors or other

problems. Data may not be exactly what is
needed. You may have difficulty getting
access. You have to verify validity and
reliability of data
Tool 3: Observation
See what is happening
Traffic patterns
Land use patterns
Layout of city and rural areas
Quality of housing
Condition of roads
Conditions of buildings
Who goes to a health clinic
Observation is Helpful when:
Need of direct information.
Trying to understand ongoing behavior.
There is physical evidence, products, or
outputs than can be observed.
Need to provide alternative when other
data collection is infeasible or
inappropriate.
Degree of Structure of Observations
Structured: Determine, before the observation, precisely what
will be observed before the observation.
Unstructured: Select the method depending upon the
situation with no pre-conceived ideas or a plan on what to
observe.
Semi-structured: A general idea of what to observe but no
specific plan.
Ways to Record Information from Observations
Observation guide
Printed form with space to record
Recording sheet or checklist
Yes/no options; tallies, rating scales
Field notes
Least structured, recorded in narrative, descriptive style
Guidelines for Planning Observations
Have more than one observer, if feasible.
Train observers so they observe the same
things.
Pilot test the observation data collection
instrument.
For less structured approach, have a few key
questions in mind.
Advantage/Challenge: Observation
Advantages Collects data on actual vs. self- reported
behavior or perceptions. It is real-time vs.
retrospective.

Challenges Observer bias, potentially unreliable;

interpretation and coding challenges;
sampling can be a problem; can be labor
intensive; low response rates.
Tool 4: Surveys and Interviews
Excellent for asking people about:
Perceptions, opinions, ideas
Less accurate for measuring behavior
Sample should be representative of the whole
Big problem with response rates
Structures for Surveys
Structured:
Precisely worded with a range of pre-determined responses
that the respondent can select.
Everyone asked exactly the same questions in exactly the
same way, given exactly the same choices.

Semi-structured
Asks same general set of questions but answers to the
questions are predominantly open-ended.
Structured vs. Semi-structured Surveys
Structured  Harder to develop.
 Easier to complete.
 Easier to analyze.
 More efficient when working with large numbers.
Semi-  Easier to develop: open ended questions.
structured  More difficult to complete: burdensome for people
to complete as a self-administrated questionnaire.
 Harder to analyze but provide a richer source of
data, interpretation of open-ended responses
subject to bias.
Modes of Survey Administration
Telephone surveys.
Self-administered questionnaires distributed by
mail, e-mail, or websites.
Administered questionnaires, common in the
development context.
In development context, often issues of language
and translation.
Mail / Phone / Internet Surveys
Literacy issues.
Consider accessibility
Reliability of postal service
Turn-around time
Consider bias
What population segment has
telephone access? Internet
access?
Advantage/Challenge: Survey
Advantages Best when you want to know what people
think, believe, or perceive, only they can tell
you that.
Challenges People may not accurately recall their
behavior or may be reluctant to reveal their
behavior if it is illegal or stigmatized. What
people think they do or say they do is not
always the same as what they actually do.
Interviews
Often semi-structured.
Used to explore complex
issues in depth.
Forgiving of mistakes:
unclear questions can be
clarified during the interview
and changed for subsequent
interviews.
Can provide evaluators with
an intuitive sense of the
situation.
Challenges of Interviews
Can be expensive, labor
intensive, and time
consuming.
Selective hearing on the
part of the interviewer may
miss information that does
not conform to pre-
existing beliefs.
Cultural sensitivity: e.g.,
gender issues.
Tool 5: Focus Groups
Type of qualitative research
where small homogenous
groups of people are brought
together to informally discuss
specific topics under the
guidance of a moderator.
Purpose: To identify issues
and themes, not just
interesting information, and
not “counts”.
Focus Groups Are Inappropriate when:
Language barriers are insurmountable.
Evaluator has little control over the situation.
Trust cannot be established.
Free expression cannot be ensured.
Confidentiality cannot be assured.
 Focus Group Process
Phase Action
1 Opening Ice-breaker; explain purpose; ground rules;
introductions.
2 Warm-up Relate experience; stimulate group interaction; start
with least threatening and simplest questions.
3 Main body Move to more threatening or sensitive and complex
questions; elicit deep responses; connect emergent
data to complex, broad participation.
4 Closure End with closure-type questions; summarize and
refine; present theories, etc; invite final comments
or insights; thank participants.
Advantage/Challenge: Focus Groups
Advantages • Can be conducted relatively quickly and easily.
• May take less staff time than in-depth.
• Allow flexibility to make changes in process and
questions.
• Can explore different perspectives.
Challenges • Analysis is time consuming.
• Participants not be representative of population.
• Possibly biasing the data.
• Group may be influenced by moderator or
dominant group members.
Tool 6: Diaries and Self-Reported Checklists
Use when you want to capture
information about events in
people’s daily lives.
Participants capture experiences in
real-time not later in a
questionnaire.
Used to supplement other data
collection.
Self-reported Checklists
Cross between a questionnaire and a diary.
The evaluator specifies a list of behaviors
or events and asks the respondents to
complete the checklist.
Done over a period of time to capture the
event or behavior.
More quantitative approach than diary.
 Advantage/Challenge: Diaries and Self-
reported Checklists
Advantages • Can capture in-depth, detailed data that might
be otherwise forgotten.
• Can collect data on how people use their time
• Can collect sensitive information.
• Supplements interviews provide richer data
Challenges • Require commitment and self-discipline.
• Data may be incomplete or inaccurate.
• Poor handwriting, difficult to understand
phrases.
Tool 7: Expert Judgment
Use of experts, one-on-one or as a panel.
Can be structured or unstructured.
Issues in selecting experts.

Example: Government task forces,

Advisory Groups
Selecting Experts
Establish criteria for selecting experts not only
on recognition as expert but also based on:
Areas of expertise
Diverse perspectives
Diverse political views
Diverse technical expertise
Advantage/Challenge: Expert Judgment
Advantages • Fast
• Relatively inexpensive
Challenges • Weak for impact evaluation.
• May be based mostly on perceptions.
• Value of data depends on how credible
the experts are perceived to be .
Tool 8: Delphi Technique
Enables experts to engage remotely in a
dialogue and reach consensus, often about
priorities.
Experts asked specific questions; often
rank choices.
Responses go to a central source, are
summarized and feedback to the experts
without attribution.
Experts can agree or argue with others’
comments.
Process may be iterative.
Advantage/Challenge: Delphi Technique
Advantages • Allows participants to remain anonymous.
• Is inexpensive.
• Is free of social pressure, personality influence, and
individual dominance.
• Is conducive to independent thinking.
• Allows sharing of information.
Challenges • May not be representative.
• Has tendency to eliminate extreme positions.
• Requires skill in written communication.
• Requires time and participant commitment.
Data Collection Summary
Choose more than one
data collection technique.
No “best” tool.
Do not let the tool drive
your work but rather
choose the right tool to
address the evaluation
question.