Evaluation Briefs
Data Collection Methods for Program
Evaluation: Observation
No. 16 | December 2008
This brief is about observation as a data collection method for evaluation. It includes a basic overview of
observation; when to use it; how to plan and conduct it; and its advantages and disadvantages.
What is observation?
Observation is way of gathering data by watching
behavior, events, or noting physical characteristics in
their natural setting. Observations can be overt
(everyone knows they are being observed) or covert
(no one knows they are being observed and the
observer is concealed). The benefit of covert
observation is that people are more likely to behave
naturally if they do not know they are being
observed. However, you will typically need to
conduct overt observations because of ethical
problems related to concealing your observation.
Observations can also be either direct or indirect.
Direct observation is when you watch interactions,
processes, or behaviors as they occur; for example,
observing a teacher teaching a lesson from a written
curriculum to determine whether they are delivering
it with fidelity. Indirect observations are when you
watch the results of interactions, processes, or
behaviors; for example, measuring the amount of
plate waste left by students in a school cafeteria to
determine whether a new food is acceptable to them.
How do you plan for observations?
When should you use observation for
evaluation?
When you are trying to understand an
ongoing process or situation. Through
observation you can monitor or watch a process
or situation that your are evaluating as it occurs.
When you are gathering data on individual
behaviors or interactions between people.
Observation allows you to watch peoples
behaviors and interactions directly, or watch for
the results of behaviors or interactions.
When you need to know about a physical
setting. Seeing the place or environment where
something takes place can help increase your
understanding of the event, activity, or situation
you are evaluating. For example, you can
observe whether a classroom or training facility
is conducive to learning.
When data collection from individuals is not
a realistic option. If respondents are unwilling
or unable to provide data through questionnaires
or interviews, observation is a method that
requires little from the individuals for whom you
need data.
Determine the focus. Think about the
evaluation question(s) you want to answer
through observation and select a few areas of
focus for your data collection. For example, you
may want to know how well an HIV curriculum
is being implemented in the classroom. Your
focus areas might be interactions between
students and teachers, and teachers knowledge,
skills, and behaviors.
Design a system for data collection. Once
you have focused your evaluation think about
the specific items for which you want to collect
data and then determine how you will collect the
information you need. There are three primary
ways of collecting observation data. These three
methods can be combined to meet your data
Evaluation Briefs
No. 16
collection needs.
o Recording sheets and checklists are the
most standardized way of collecting
observation data and include both preset
questions and responses. These forms are
typically used for collecting data that can be
easily described in advance (e.g., topics that
might be covered in an HIV prevention
lesson).
o Observation guides list the interactions,
processes, or behaviors to be observed with
space to record open-ended narrative data.
o Field notes are the least standardized way of
collecting observation data and do not
include preset questions or responses. Field
notes are open-ended narrative data that can
be written or dictated onto a tape recorder.
Select the sites. Select an adequate number of
sites to help ensure they are representative of the
larger population and will provide an
understanding of the situation you are observing.
Select the observers. You may choose to be the
only observer or you may want to include others
in conducting observations. Stakeholders, other
professional staff members, interns and graduate
students, and volunteers are potential observers.
Train the observers. It is critical that the
observers are well trained in your data collection
process to ensure high quality and consistent
data. The level of training will vary based on the
complexity of the data collection and the
individual capabilities of the observers.
Time your observations appropriately.
Programs and processes typically follow a
sequence of events. It is critical that you
schedule your observations so you are observing
the components of the activity that will answer
your evaluation questions. This requires advance
planning.
What are the advantages of observation?
Allows you to directly see what people do rather
than relying on what people say they did.
What are the disadvantages of
observation?
Susceptible to observer bias.
Susceptible to the hawthorne effect, that is,
people usually perform better when they know
they are being observed, although indirect
observation may decrease this problem.
Can be expensive and time-consuming compared
to other data collection methods.
Does not increase your understanding of why
people behave as they do.
Resources
Taylor-Powell E, Steele S. Collecting Evaluation Data:
Direct Observation. University of Wisconsin
Cooperative Extension. 1996. Available at
http://learningstore.uwex.edu/pdf/G3658-5.PDF
arningstore.uwex.edu/pdf/G3658-5.PDF
Evaluation and Civic Activism Curriculum. Module 3: Data
Collection Methods and Evaluation Tools. Social Policy
Research Associates. Available at
http://www.evaluationtools.org/files/Guidelines%2
0for%20Observations.pdf
Performance Evaluation and Monitoring TIPS. No.
4: Using Direct Observation Techniques. USAID Center
for Development Information and Evaluation. 1996.
Available at
http://www.usaid.gov/pubs/usaid_eval/ascii/pnaby
208.txt
Section 6: Direct Observation and Photovoice (Rapid
Appraisal). UCLA Center for Health Policy and
Research.
http://www.healthpolicy.ucla.edu/HealthData/tw_c
ba25.pdf
Collect data where and when an event or activity
is occurring.
Does not rely on peoples willingness or ability
to provide information.
For further information or assistance, contact the Evaluation Research Team at
[email protected]. You can also contact us via our Website at:
http://www.cdc.gov/healthyyouth/evaluation/index.htm.
