Mo n ito r in g & Eva l uatio n

Monitoring
and Evaluation
Planning
Guidelines and Tools

by Scott G. Chaplowe
American Red Cross
Since 1943, Catholic Relief Services (CRS) has held the privilege of serving the poor
and disadvantaged overseas. Without regard to race, creed, or nationality, CRS
provides emergency relief in the wake of natural and manmade disasters. Through
development projects in fields such as education, peace and justice, agriculture,
microfinance, health and HIV/AIDS, CRS works to uphold human dignity and
promote better standards of living. CRS also works throughout the United States
to expand the knowledge and action of Catholics and others interested in issues of
international peace and justice. Our programs and resources respond to the U.S.
Bishops’ call to live in solidarity—as one human family—across borders, over oceans,
and through differences in language, culture and economic condition.

The American Red Cross helps vulnerable people around the world prevent, prepare
for, and respond to disasters, complex humanitarian emergencies, and life-threatening
health conditions through global initiatives and community-based programs. With a
focus on global health, disaster preparedness and response, restoring family links, and
the dissemination of international humanitarian law, the American Red Cross provides
rapid, effective, and large-scale humanitarian assistance to those in need. To achieve
our goals, the American Red Cross works with our partners in the International Red
Cross and Red Crescent Movement and other international relief and development
agencies to build local capacities, mobilize and empower communities, and establish
partnerships. Our largest program is currently the Tsunami Recovery Program,
which is improving community health and preventing disease outbreaks, supporting
communities as they rebuild their lives and reestablish their livelihoods, and helping
affected Red Cross and Red Crescent Societies and their communities develop disaster
preparedness capabilities.

Published in 2008 by:

Catholic Relief Services American Red Cross

228 W. Lexington Street 2025 E Street, NW
Baltimore, MD 21201-3413 Washington, DC 20006
USA USA

Author: Scott G. Chaplowe (American Red Cross)

Series Editor: Guy Sharrock (CRS)
Readers/Editors: Cynthia Green and Dina Towbin
Graphic Designer: Jeanne Ivy
Cover Photograph: Sean Sprague

This module was produced by CRS and the American Red Cross with financial support
from the United States Agency for International Development (USAID) Food for Peace
(FFP) grants: CRS Institutional Capacity Building Grant (AFP-A-00-03-00015-00) and
American Red Cross Institutional Capacity Building Grant (AFP-A-00-00007-00).
The views expressed in this document are those of the author and do not necessarily
represent those of the USAID or FFP.

Recommended citation: Chaplowe, Scott G. 2008. “Monitoring and Evaluation

Planning.” American Red Cross/CRS M&E Module Series. American Red Cross and
Catholic Relief Services (CRS), Washington, DC and Baltimore, MD.

To access the full series, please visit: www.crs.org or www.redcross.org.

Contents

iii Preface
iii Acknowledgments
iv Acronyms

Monitoring and Evaluation Planning

1 Introduction
3 The M&E System – An Overview
5 The Four Key Components of an M&E System
15 Other Aspects of M&E Planning

Tools and Resources Annexes

19 Annex I References and Resources
22 Annex II Logframe Example
23 Annex III Indicator Matrix Example
25 Annex IV Data Collection Tools and Techniques
Preface
Monitoring and evaluation (M&E) are core responsibilities of American Red
Cross and Catholic Relief Services (CRS) program managers and help ensure
quality in our programming. Monitoring and Evaluation Planning is one in a
series of M&E training and capacity-building modules that the American
Red Cross and CRS have agreed to collaborate on under their respective
Institutional Capacity Building Grants. These modules are designed to
respond to field-identified needs for specific guidance and tools that did not
appear to be available in existing publications. Although examples in the
modules focus on Title II programming, the guidance and tools provided have
value beyond the food-security realm.

Our intention in writing the Monitoring and Evaluation Planning module was to
provide concise guidance to readers to develop a comprehensive M&E system
for international humanitarian relief and development programs. Please send
any comments or suggestions for this module to m&[email protected].

Acknowledgments
The author would like to acknowledge the considerable contribution and
guidance of Cynthia Green, formerly with the American Red Cross, in her
patient review and editing of this module, as well as the work of Dina Towbin
(consultant), in shepherding the module through the final stages, and Joe
Schultz (CRS) and Jeanne Ivy (consultant), who were responsible for the
graphic design work.

Monitoring and Evaluation Planning • iii

Acronyms
CRS Catholic Relief Services
HH Household
KAP Knowledge, attitudes, and practices
M&E Monitoring and evaluation
MOH Ministry of Health
PRA Participatory rapid (or rural) appraisal
UNAIDS Joint United Nations Programme on HIV/AIDS
UNDP United Nations Development Programme
USAID United States Agency for International Development

Monitoring and Evaluation Planning • iv

 Introduction
This Monitoring and Evaluation Planning module is intended to provide
This Monitoring and concise guidance to develop a comprehensive monitoring and evaluation
(M&E) system for international humanitarian relief and development
Evaluation Planning
programs. It covers the key planning documents and processes needed to
module is designed set up and implement an M&E system for project planning, implementation,
for use by M&E and evaluation. It is designed for use by M&E specialists, managers of
specialists, managers humanitarian and development programs, and decision makers responsible
for program oversight and funding.
of humanitarian and
development programs, In developing this module, it became apparent that some people working
and decision makers in international programs use the term “M&E plan” to refer to a planning
responsible for program document for a project’s entire M&E system, whereas others use it to refer
to a specific document that defines project indicators and how they will be
oversight and funding.
measured—an indicator matrix. For clarity, this module adopts the broader
usage of an M&E plan as a key planning document for coherence and
continuity within a project’s M&E system. The specific format or content for
an M&E plan should be tailored and adopted to specific project needs. This
module focuses on the key components of an M&E system that inform M&E
planning for projects.

This module focuses on the key components of an M&E system that inform
M&E planning for projects. These components trace a logical train of thought
from hypotheses on how the project will bring about change in a specific
sector, to the specific objectives needed for these changes, methods for
measuring the project’s achievement of its stated objectives, and protocols
for collecting and analyzing data and information used in the measurement.
The four key components of an M&E system are:

1. A causal analysis framework

2. A logframe or logical framework
3. An indicator matrix
4. A data collection and analysis plan.

Monitoring and Evaluation Planning • 1

 Introduction

Following an overview of the M&E system, this module examines these four
key M&E components. It is important to stress that the various components
of an M&E system are interdependent and that M&E planning requires other
elements, whether stated explicitly or implicitly. Other key considerations
for M&E planning are presented in the final section of the module and
highlighted in relevant boxes throughout.

Box 1. Plan Early and Involve Stakeholders

M&E planning should begin during or immediately after the project design stage.
Early planning will inform the project design and allow for sufficient time to
arrange for resources and personnel prior to project implementation. M&E planning
should also involve those using the M&E system. Involvement of project staff and
key stakeholders ensures feasibility, understanding, and ownership of the M&E
system.

Monitoring and Evaluation Planning • 2

 The M&E System–An Overview
The M&E system provides the information needed to assess and guide the
M&E should be an project strategy, ensure effective operations, meet internal and external
reporting requirements, and inform future programming. M&E should be
integral part of project
an integral part of project design as well as project implementation and
design as well as project completion. Accordingly, this module will begin by describing the overall
implementation and M&E system as it corresponds with these key stages in a project’s lifecycle (see
completion. Figure 1).

Figure 1. M&E and the Project Cycle

Project Follow-Up

Project
Dissemination Design
& Use of Needs
Project Lessons Assessment
End

Final Evaluation
(With Endline Logframe
Survey) &
Indicators
Ongoing Project
Project
Monitoring Planning
Midterm or &
Project Annual Reflection M&E Plan
Evaluation Development
Middle [Indicator Table]

Project
Monitoring Baseline
(QPR) Survey

Project Implementation

An M&E system is built on the key parameters of a project:

▪▪ The overall goal or desired change or effect

▪▪ The main beneficiaries or audience that the project seeks to benefit
▪▪ The hypotheses or assumptions that link the project objectives to
specific interventions or activities
▪▪ The project scope and size

Monitoring and Evaluation Planning • 3

 The M&E System—An Overview

▪▪ The extent of participation in and capacity for M&E

▪▪ The project duration
▪▪ The overall project budget.

Each project may have different M&E needs, depending on the operating
context, implementing agency capacity, donor requirements, and other
factors. In preparing an M&E plan, it is important to identify these needs
and coordinate the methods, procedures, and tools used to meet them; this
conserves resources and streamlines M&E planning.

There is not a single, recognized industry standard for assessing the quality
of an M&E system. However, some key criteria are summarized below (IFAD
2002, pp. 4-20):

▪▪ Utility: The proposed M&E system will serve the practical information
needs of intended users.
▪▪ Feasibility: The methods, sequences, timing and processing
procedures proposed are realistic, prudent and cost-effective.
▪▪ Propriety: The M&E activities will be conducted legally, ethically and
with due regard for the welfare of those affected by its results.
▪▪ Accuracy: The M&E outputs will reveal and convey technically
adequate information.

Monitoring and Evaluation Planning • 4

 The Four Key Components
of an M&E System
The four key components discussed below form the foundation upon which
There are four key the M&E system is built. They play a critical role in M&E planning, answering
these four corresponding questions:
components that form
the foundation upon 1. What does the project want to change and how?
which the M&E system 2. What are the specific objectives to achieve this change?
is built. 3. What are the indicators and how will they measure this?
4. How will the data be collected and analyzed?

Causal Analysis Framework

A causal analysis framework seeks to specify the following:

1. The major problem and condition(s) that the project seeks to change
2. Factors that cause the condition(s)
3. Ways to influence the causal factors, based on hypotheses of the
relationships between the causes and likely solutions
4. Interventions to influence the causal factors
5. The expected changes or desired outcomes (see Table 1).

Causal analysis should be based on a careful study of local conditions and

available data as well as consultation with potential beneficiaries, program
implementers, other stakeholders, and technical experts. Such information
may be available in needs assessments, feasibility studies, participatory rapid
appraisals (PRAs), community mapping, and SWOT (strengths, weaknesses,
opportunities, threats) analysis. CARE outlines a holistic appraisal for
assessing the socioeconomic factors to identify target populations and
appropriate interventions (Caldwell 2002).

The assumptions underlying causal analysis can be assessed by involving

potential beneficiaries, program managers and implementers, other
stakeholders, and technical experts.

Monitoring and Evaluation Planning • 5

 Four Key Components of an M&E System

Table 1. Causal Analysis Framework

Causal Analysis Hypothesis Project Design
Development
Cause/Conditions IF mothers are aware of Interventions
Mothers do not know that the dangers of unclean Educate mothers about
unclean water will make water, the dangers of unclean
infants sick (knowledge). water

Mothers believe that AND that breastmilk is Educate mothers about

breastmilk alone does not nutritionally sufficient for the nutritional value of
satisfy infants younger infants younger than 6 breastmilk for infants
than 6 months (attitude). months, younger than 6 months

Mothers are giving THEN they will Desired Outcomes

breastmilk substitutes to breastfeed their infant Increased breastfeeding
infants younger than 6 exclusively to avoid of infants younger than 6
months (practice). exposure to unclean months
water,

Problem THEREBY contributing Reduced diarrhea among

High diarrhea rates to reductions in diarrhea infants younger than 6
among infants younger among infants younger months
than 6 months than 6 months,

Consequence THEREBY contributing Overall Goal

High rates of infant to reductions in infant Reduce infant mortality
mortality mortality

Many projects do not develop an explicit causal analysis framework.

Nevertheless, such a framework is helpful in clarifying key interventions
and identifying variables needed to assess the extent of project effects. For
example, the framework presented in Table 1 hypothesizes that mothers will
breastfeed their infants once they learn about the dangers of unclean water.
However, if mothers are not breastfeeding for other reasons, such as cultural
norms or working away from home, then different interventions are needed.
In effect, the M&E system tests the hypotheses to determine whether the
project’s interventions and outputs contributed to the desired outcomes.

The selection of problems to address and the appropriate interventions should

be grounded in research findings and program experience in similar settings.
Causal analysis is useful to examine cause and effect relationships and
identify community needs from which to formulate a working hypothesis.
Other forms of analysis include problem analysis, such as problem trees,
to isolate conditions and consequences that help identify objectives and
strategies (Stetson et al. 2004, p. 78), and theory of change analysis, which
uses backwards mapping to identify conditions required to bring about
desired long-term outcomes (Theory of Change 2008).

Monitoring and Evaluation Planning • 6

 Four Key Components of an M&E System

Logframe or Logical Framework

Tools & Resources A logframe or logical framework shows the conceptual foundation upon
Logframe Example which the project’s M&E system is built. Basically, the logframe is a matrix
that specifies what the project is intended to achieve (objectives) and how this
achievement will be measured (indicators). It is essential to understand the
differences between project inputs, outputs, outcomes, and impact, since the
indicators to be measured under the M&E system reflect this hierarchy. Table
2 defines the key terms and components of a classic 4 x 5 logframe matrix, and
Annex II provides an example of a logframe for outcome and output levels. It
is important to note that various organizations in the development community
use different formats and terms for the types of objectives in a logframe; Jim
Rugh (2008) developed a useful guide to decipher these terms used by major
development agencies.

A clear understanding of the logframe’s hierarchy of objectives is essential for

M&E planning. Ultimately, it will inform the key questions that will guide the
evaluation of project processes and impacts:

▪▪ Goal: To what extent has the project contributed towards its longer
term goals? Why or why not? What unanticipated positive or negative
consequences did the project have? Why did they arise?
▪▪ Outcomes: What changes have occurred as a result of the outputs
and to what extent are these likely to contribute towards the project
purpose and desired impact? Has the project achieved the changes for
which it can realistically be held accountable?
▪▪ Outputs: What direct tangible products or services has the project
delivered as a result of activities?
▪▪ Activities: Have planned activities been completed on time and within
the budget? What unplanned activities have been completed?
▪▪ Inputs: Are the resources being used efficiently?

Similarly, it is also important to understand the logframe’s hierarchy of

indicators. For instance, it is usually easier to measure lower-level indicators
such as the number of workshop participants, while the difficulty in precision
and measurement complexity increases when attempting to measure changes
in behavior. The higher levels of the indicator hierarchy require more analysis
and synthesis of different information types and sources. This affects the M&E
data collection methods and analysis, which has implications for staffing,
budgets, and timeframe.

Monitoring and Evaluation Planning • 7

 Four Key Components of an M&E System

Table 2. Logframe Definition Table

Project Indicators Means of Assumptions
Objectives Verification
Goal Impact Indicator Measurement External factors
Simple clear Quantitative method, data necessary to
statement of the or qualitative source, and sustain the long-
impact or results means to measure data collection term impact, but
to achieve by the achievement or to frequency for beyond the control
project reflect the changes stated indicator of the project
connected to
stated goal

Outcomes Outcome Measurement External

Set of beneficiary Indicator method, data conditions
and population- Quantitative source, and necessary if the
level changes or qualitative data collection outcomes are
needed to achieve means to measure frequency for to contribute to
the goal (usually achievement or to stated indicator achieving the goal
knowledge, reflect the changes
attitudes and connected to
practices, or KAP) stated outcomes

Outputs Output Indicator Measurement Factors out of the

Products or Quantitative method, data project’s control
services needed or qualitative source, and that could restrict
to achieve the means to measure data collection or prevent the
outcomes completion of frequency for outputs from
stated outputs stated indicator achieving the
(measures the outcomes
immediate
product of an
activity)

Activities Process Indicator Measurement Factors out of the

Regular efforts Quantitative method, data project’s control
needed to produce or qualitative source, and that could restrict
the outputs means to measure data collection or prevent the
completion of frequency for activities from
stated activities, stated indicator achieving the
i.e., attendance at outcomes
the activities

Inputs Input Indicator Measurement Factors out of the

Resources used Quantitative method, data project’s control
to implement or qualitative source, and that could restrict
activities means to measure data collection or prevent access
(financial, utilization of frequency for to the inputs
materials, human) stated inputs stated indicator
(resources used
for activities)
Source: Author based on an example from Caldwell (2002, p. 139).

Effective indicators are a critical logframe element. Technical expertise is

helpful, and before indicators are finalized, it is important to review them
with local staff to ensure that they are realistic and feasible and meet user
informational needs.

Monitoring and Evaluation Planning • 8

 Four Key Components of an M&E System

Consider the following questions when designing indicators:

▪▪ Are the indicators SMART (specific, measurable, achievable, relevant,

and time-bound)? Indicators should be easy to interpret and explain,
timely, cost-effective, and technically feasible. Each indicator should
have validity (be able to measure the intended concept accurately)
and reliability (yield the same data in repeated observations of a
variable).
▪▪ Are there international or industry standard indicators? For
example, indicators developed by UNAIDS, the UNDP Millennium
Development Goals, and the Demographic and Health Surveys have
been used and tested extensively.
▪▪ Are there indicators required by the donor, grant or program? This
can be especially important if the project-level indicator is expected to
roll up to a larger accountability framework at the program level.
▪▪ Are there secondary indicator sources? It may be cost-effective to
adopt indicators for which data have been or will be collected by a
government ministry, international agency, and so on.

Box 2. Indicator Traps

▪▪ Indicator overload. Indicators do not need to capture everything in a project, but
only what is necessary and sufficient for monitoring and evaluation.
▪▪ Output fixation. Counting myriad activities or outputs is useful for project
management but does not show the project’s impact. For measuring project
effects, it is preferable to select a few key output indicators and focus on outcome
and impact indicators whenever possible.
▪▪ Indicator imprecision. Indicators need to be specific so that they can be readily
measured. For example, it is better to ask how many children under age 5 slept
under an insecticide-treated bednet the previous night than to inquire generally
whether the household practices protective measures against malaria.
▪▪ Excessive complexity. Complex information can be time-consuming, expensive,
and difficult for local staff to understand, summarize, analyze, and work with.
Keep it simple, clear, and concise.

Decisions regarding indicators are linked to the overall research plan. The
type of data and information to be collected will depend on the research
question being addressed, the desired level of precision in measuring
project effects, and the project’s size and complexity. These issues need to be
considered when the logframe is being developed, since they are related to the
selection of interventions and project outputs, the proposed M&E budget, and
staffing levels.

It is important to note that there are other types of frameworks used to show
the relationships between project objectives and the indicators that will
demonstrate achievement or progress toward these objectives. This module
focuses on the logframe because it is widely used for development projects,
but it does have its limitations (see Box 3). Another framework used by the
U.S. Agency for International Development (USAID) and other donors is the
results framework, sometimes called a strategic framework. Using diagrams
to illustrate the steps or levels of results, the results framework emphasizes

Monitoring and Evaluation Planning • 9

 Four Key Components of an M&E System

the causal relationships that connect incremental achievement of each result to

Tools & Resources the comprehensive program impact.

Indicator Matrix Example

Box 3. Logframe Limitations
In M&E planning, it is important to be sensitive to the critique that logic models are
technocentric, with a cultural bias towards linear logic that can alienate rather than
foster local understanding, participation, and ownership. It is essential to consult
and involve local partners, especially managers, to enhance their understanding of
logframes.

The Indicator Matrix

An indicator matrix is a critical tool for planning and managing data
collection, analysis, and use. It expands the logframe to identify key
information requirements for each indicator and summarizes the key M&E
tasks for the project. While the names and formats of the indicator matrix
may vary, (e.g., M&E plan, indicator planning matrix, or data collection plan),
the overall function remains the same. Often, the project donor will have a
required format (see, for example, USAID 1996; IFRC 2007, p. 6; Stetson et al.
2004, p. 140; Barton 1997, p. 53; Caldwell 2002, p. 103; IFAD 2002, Annex C;
AusAID 2006, p. 6).

Annex III provides a sample format for an indicator matrix, with illustrative
rows for outcome and output indicators. The following are the major
components (column headings) of the indicator matrix:

1. Indicators: The indicators provide clear statements of the precise

information needed to assess whether proposed changes have
occurred. Indicators can be either quantitative (numeric) or qualitative
(descriptive observations). Typically the indicators in an indicator
matrix are taken directly from the logframe.
2. Indicator Definitions: Each indicator needs a detailed definition of
its key terms, including an explanation of specific aspects that will
be measured (such as who, what, and where the indicator applies).
The definition should explain precisely how the indicator will be
calculated, such as the numerator and denominator of a percent
measure. This column should also note if the indicator is to be
disaggregated by sex, age, ethnicity, or some other variable.
3. Methods/Sources: This column identifies sources of information
and data collection methods or tools, such as use of secondary
data, regular monitoring or periodic evaluation, baseline or endline
surveys, PRA, and focus group discussions. This column should also
indicate whether data collection tools (questionnaires, checklists)
are pre-existing or will need to be developed. Note that the logframe
column on “Means of Verification” may list a source or method, i.e.,
“household survey,” the M&E plan requires much more detail, since
the M&E work will be based on the specific methods noted.
4.

Monitoring and Evaluation Planning • 10

 Four Key Components of an M&E System

5. Frequency/Schedules: This column states how often the data for each
indicator will be collected, such as monthly, quarterly, or annually. It is
often useful to list the data collection timing or schedule, such as start-
up and end dates for collection or deadlines for tool development.
When planning for data collection timing, it is important to consider
factors such as seasonal variations, school schedules, holidays, and
religious observances (i.e., Ramadan).
6. Person(s) Responsible: This column lists the people responsible and
accountable for the data collection and analysis, i.e., community
volunteers, field staff, project managers, local partner/s, and external
consultants. In addition to specific people’s names, use the position
title to ensure clarity in case of personnel changes. This column is
useful in assessing and planning for capacity building for the M&E
system.
7. Data Analysis: This column describes the process for compiling
and analyzing the data to gauge whether the indicator has been
met or not. For example, survey data usually require statistical
analysis, while qualitative data may be reviewed by research staff or
community members.
8. Information Use: This column identifies the intended audience and
use of the information. For example, the findings could be used for
monitoring project implementation, evaluating the interventions,
planning future project work, or reporting to policy makers or donors.
This column should also state ways that the findings will be formatted
(e.g., tables, graphs, maps, histograms, and narrative reports) and
disseminated (e.g., Internet Web sites, briefings, community meetings,
listservs, and mass media).

The indicator matrix can be adapted to information requirements for project

management. For example, separate columns can be created to identify
data sources, collection methods and tools, information use and audience,
or person(s) responsible for data collection and analysis. It may also be
preferable to use separate matrices for M&E indicators.

It is critical that the indicator matrix be developed with the participation

of those who will be using it. Completing the matrix requires detailed
knowledge of the project and context provided by the local project team
and partners. Their involvement contributes to data quality because it
reinforces their understanding of what data they are to collect and how
they will collect them.

Data Collection and Analysis Plan

The data collection and analysis plan expands on the information provided
in the indicator matrix by describing in detail how data and information
will be defined, collected, organized, and analyzed. Typically, this plan
consists of a detailed narrative that explains how each type of data will be
collected along with all the steps needed to ensure quality data and sound
research practices. Key components of this plan include: the unit of analysis;
the link between indicators, variables and questionnaires; the sampling

Monitoring and Evaluation Planning • 11

 Four Key Components of an M&E System

frame and methodology; timing and mode of data collection; research staff
responsibilities; enumerator selection, training, and supervision; fieldwork
timing and logistics; checks for data quality; data entry and storage;
hypothesized relationships among the variables; and data analysis methods.
Special analyses, such as disaggregating data by gender, age, location and
socio-economic status, should also be described.

It is important to provide the rationale for the data collection and analysis
methods. This includes the triangulation of methods (quantitative and/
or qualitative) and sources to reduce bias and ensure data reliability and
completeness. It should also be informed by the standards that guide
good practice of project evaluation. There are many useful resources in
the evaluation community that identify key principles to ensure ethical,
accountable, and quality evaluations (for example, American Evaluation
Association [AEA] 2004, Australian Evaluation Society [AES] 2002, and
Development Assistance Committee [DAC] 2008).

The plan should also discuss the purpose of data collection and analysis in
terms of specific monitoring and evaluation functions. Some key functions
of monitoring include compliance, process, results, context, beneficiary, and
organizational monitoring. Typically, a project will use a combination of these
monitoring functions and design data collection and analysis accordingly. For
project assessments, the discussion should identify not only the methods used,
but the timing of the assessment event (i.e., baseline studies, annual reviews,
midterm and final evaluations), and the rationale for selecting evaluators with
specific skill sets and independence (i.e., internal versus external evaluators).

Major sources of data and information for project monitoring and

evaluation include:

▪▪ Secondary data. Useful information can be obtained from other

research, such as surveys and other studies previously conducted or
planned at a time consistent with the project’s M&E needs, in-depth
assessments, and project reports. Secondary data sources include
government planning departments, university or research centers,
international agencies, other projects/programs working in the area,
and financial institutions.
▪▪ Sample surveys. A survey based on a random sample taken from
the beneficiaries or target audience of the project is usually the best
source of data on project outcomes and effects. Although surveys are
laborious and costly, they provide more objective data than qualitative
methods. Many donors expect baseline and endline surveys to be
done if the project is large and alternative data are unavailable.
▪▪ Project output data. Most projects collect data on their various
activities, such as number of people served and number of items
distributed.
▪▪ Qualitative studies. Qualitative methods that are widely used in
project design and assessment are: participatory rapid appraisal,

Monitoring and Evaluation Planning • 12

 Four Key Components of an M&E System

mapping, key informant interviews, focus group discussions, and

observation.
Tools & Resources
▪▪ Checklists. A systematic review of specific project components can
References and Resources be useful in setting benchmark standards and establishing periodic
measures of improvement.
Data Collection Tools and
Techniques ▪▪ External assessments. Project implementers as well as donors often
hire outside experts to review or evaluate project outputs and
outcomes. Such assessments may be biased by brief exposure to
the project and over-reliance on key informants. Nevertheless, this
process is less costly and faster than conducting a representative
sample survey, and it can provide additional insight, technical
expertise, and a degree of objectivity that is more credible to
stakeholders.
▪▪ Participatory assessments. The use of beneficiaries in project review
or evaluation can be empowering, building local ownership, capacity,
and project sustainability. However, such assessments can be biased
by local politics or dominated by the more powerful voices in the
community. Also, training and managing local beneficiaries can
take time, money, and expertise, and it necessitates buy-in from
stakeholders. Nevertheless, participatory assessments may be
worthwhile as people are likely to accept, internalize, and act upon
findings and recommendations that they identify themselves.

See Annex IV for a more extensive list of data sources. Also, Annex I lists M&E
guides that describe the process of data collection and analysis.

Some practical considerations in planning for data collection include:

▪▪ Prepare data collection guidelines. This helps to ensure
standardization, consistency, and reliability over time and among
different people in the data collection process. Double-check that all
the data required for indicators are being captured through at least
one data source.
▪▪ Pretest data collection tools. Pretesting helps to detect problematic
questions or techniques, verify collection time, identify potential
ethical issues, and build the competence of data collectors.
▪▪ Train data collectors. Provide an overview of the data collection
system, data collection techniques, tools, ethics, and culturally
appropriate interpersonal communication skills. Give trainees
practical experience collecting data.
▪▪ Address ethical concerns. Identify and respond to any concerns
expressed by the target population. Ensure that the necessary
permission or authorization has been obtained from local authorities,
that local customs and attire are respected, and that confidentiality
and voluntary participation are maintained.

Monitoring and Evaluation Planning • 13

 Four Key Components of an M&E System

Box 4. Reducing Data Collection Costs

Data collection can be costly. One of the best ways to reduce data collection costs is
to reduce the amount of data collected (Bamberger et al. 2006). The following questions
can help simplify data collection and reduce costs:

▪▪ Is the information necessary and sufficient? Collect only what is necessary

for project management and evaluation. Limit information needs to the stated
objectives, indicators, and assumptions in the logframe.
▪▪ Are there reliable secondary data sources? This can save costs for primary data
collection.
▪▪ Is the sample size adequate but not excessive? Determine the sample size that
is necessary to estimate or detect change. Consider using stratified and cluster
samples.
▪▪ Can the data collection instruments be simplified? Eliminate extraneous
questions from questionnaires and checklists. In addition to saving time and cost,
this has the added benefit of reducing “survey fatigue” among respondents.

A data analysis plan should identify:

▪▪ When data analysis will occur. It is not an isolated event at the end of
data collection, but an ongoing task from project start. Data analysis
can be structured through meetings and other forums to coincide with
key project implementation and reporting benchmarks.
▪▪ To what extent analysis will be quantitative and/or qualitative, and
any specialized skills and equipment required for analysis.
▪▪ Who will do the analysis, i.e., external experts, project staff,
beneficiaries, and/or other stakeholders.
▪▪ If and how subsequent analysis will occur. Such analysis may be
needed to verify findings, to follow-up on research topics for project
extension and additional funding, or to inform future programming.

An important consideration in planning for data collection and analysis is to

identify any limitations, biases, and threats to the accuracy of the data and
analysis. Data distortion can occur due to limitations or errors in design,
sampling, field interviews, and data recording and analysis. It is best to
monitor the research process carefully and seek expert advice, when needed.

It is also important to carefully plan for the data management of the M&E
system. This includes the set of procedures, people, skills, and equipment
necessary to systematically store and manage M&E data. If this step is
not carefully planned, data can be lost or incorrectly recorded, which
compromises not only data quality and reliability, but also subsequent data
analysis and use. Poorly managed data waste time and resources.

Monitoring and Evaluation Planning • 14

 Other Aspects of M&E Planning

Information Reporting and Utilization

Reporting is closely
Reporting project achievements and evaluation findings serves many
related to M&E work, important functions, namely to:
since data are needed
▪▪ Advance learning among project staff as well as the larger
to support the major development community
findings and conclusions ▪▪ Improve the quality of the services provided
presented in a ▪▪ Inform stakeholders on the project benefits and engage them in work
project report. that furthers project goals
▪▪ Inform donors, policy makers and technical specialists of effective
interventions (and those that did not work as hoped)
▪▪ Develop a project model that can be replicated and scaled-up.

Reporting is closely related to M&E work, since data are needed to support
the major findings and conclusions presented in a project report. Often,
the focus and frequency of M&E processes are determined by reporting
requirements and schedules.

Practical considerations in information reporting and utilization planning

include:

▪▪ Design the M&E communication plan around the information

needs of the users. The content and format of data reports will
vary, depending on whether the reports are to be used to monitor
processes, conduct strategic planning, comply with requirements,
identify problems, justify a funding request, or conduct an impact
evaluation.
▪▪ Identify the frequency of data reporting needs. For example, project
managers may want to review M&E data frequently to assess project
progress and make decisions, whereas donors may need data only
once or twice a year to ensure accountability.
▪▪ Tailor reporting formats to the intended audience. Reporting may
entail different levels of complexity and technical language; the
report format and media should be tailored to specific audiences and
different methods used to solicit feedback.
▪▪ Identify appropriate outlets and media channels for communicating
M&E data. Consider both internal reporting, such as regular project
reports to management and progress reports to donors, as well as
external reporting, such as public forums, news releases, briefings,
and Internet Web sites.

Monitoring and Evaluation Planning • 15

 Other Aspects of M&E Planning

M&E Staffing and Capacity Building

Staffing is a special concern for M&E work because it demands special
training and a combination of research and project management skills. Also,
the effectiveness of M&E work often relies on assistance from staff and
volunteers who are not M&E experts. Thus, capacity building is a critical
aspect of implementing good M&E work.

Suggestions for ensuring adequate M&E support include the following:

▪▪ Identify the various tasks and related skills that are needed, such
as ensuring adequate data collection systems in the field, research
design, and data entry and analysis
▪▪ Assess the relevant skills of the project team, partner organizations,
and the community beneficiaries
▪▪ Specify to what extent local stakeholders will (or will not) participate
in the M&E process (see Table 3)
▪▪ Assign specific roles and responsibilities to team members and
designate an overall M&E manager
▪▪ Recruit consultants, students, and others to fill in the skill gaps and
special needs such as translation, statistical analysis, and cultural
knowledge
▪▪ Identify the topics for which formal training is needed and hold
training sessions
▪▪ Encourage staff to provide informal training through on-the-job
guidance and feedback, such as commenting on a report or showing
how to use computer software programs
▪▪ Give special attention to building local capacity in M&E.

Cultivating nascent M&E skills takes time and patience, but in the end the
contributions of various collaborators will enrich M&E work and lead to
greater acceptance of M&E’s role in project implementation.

Monitoring and Evaluation Planning • 16

 Other Aspects of M&E Planning

Table 3. Considering Participatory M&E

Potential Advantages Potential Disadvantages
▪▪ Empowers beneficiaries to analyze ▪▪ Requires more time and cost to
and act on their own situation (as train and manage local staff and
“active participants” rather than community members
“passive recipients”) ▪▪ Requires skilled facilitators to ensure
▪▪ Builds local capacity to manage, that everyone understands the process
own, and sustain the project. People and is equally involved
are likely to accept and internalize ▪▪ Can jeopardize the quality of collected
findings and recommendations that data due to local politics. Data
they provide. analysis and decision making can
▪▪ Builds collaboration and consensus be dominated by the more powerful
at different levels—between voices in the community (related to
beneficiaries, local staff and partners, gender, ethnic, or religious factors).
and senior management ▪▪ Demands the genuine commitment
▪▪ Reinforces beneficiary accountability, of local people and the support of
preventing one perspective from donors, since the project may not use
dominating the M&E process the traditional indicators or formats
▪▪ Saves money and time in data for reporting findings
collection compared with the cost of
using project staff or hiring outside
support
▪▪ Provides timely and relevant
information directly from the field
for management decision making to
execute corrective actions

Budgeting for M&E

A key function of planning for M&E is to estimate the costs, staffing, and other
resources needed for M&E work. It is important for M&E specialists to weigh
in on M&E budget needs at the project design stage so that funds are allocated
specifically to M&E and are available to implement key M&E tasks.

The following are suggestions for building a realistic budget:

▪▪ List all M&E tasks and overall responsibilities, analyze the necessary
items associated with each task, and determine their cost
▪▪ Budget for staffing, including full-time staff, external consultants,
capacity building/training, and other human resource expenses
▪▪ Ensure that the budget includes all capital expenses, including facility
costs, office equipment and supplies, travel and lodging, computer
hardware and software, and other expenses
▪▪ Determine whether all tasks are included in the overall project
budget, such as support for an information management system, field
transportation and vehicle maintenance, translation, and printing and
publishing of M&E documents/tools
▪▪ Review the donor’s requirements to determine whether there are any
extra items that need to be budgeted, or conversely, items such as an
external evaluation that will be funded directly by the donor

Monitoring and Evaluation Planning • 17

 Other Aspects of M&E Planning

▪▪ Allow for unexpected contingencies such as inflation, currency

devaluation, equipment theft, or the need for additional data
collection/analysis to verify findings.

A narrative justifying each line item can help guard against arbitrary budget
cuts. It may be necessary to clarify or justify expenses, such as wage rates
not normally paid to comparable positions, fees for consultants and external
experts, or the various steps in a survey that add up in cost (development
and testing the questionnaire, translation and back-translation, enumerator
training, enumerators’ and field supervisors’ daily rates, travel/lodging costs
for administering the survey, data analysis and write-up, and so on).

Program managers often ask what proportion of a project’s budget should be

allocated to M&E. There is no set formula; various donors and organizations
recommend that between 3 to10 percent of a project’s budget should be
allocated to M&E (Frankel and Gage 2007, p. 11). A general rule of thumb is
that the M&E budget should not be so small as to compromise the accuracy
and credibility of results, but neither should it divert project resources to the
extent that programming is impaired.

Monitoring and Evaluation Planning • 18

Annex I
References and Resources1

References
AEA (American Evaluation Association). 2004. American Evaluation Association Guiding Principles for
Evaluators. www.eval.org/Publications/GuidingPrinciplesPrintable.asp.

AES (Australian Evaluation Society). 2002. Australian Evaluation Society Guidelines for the Ethical Conduct of
Evaluations. http://www.aes.asn.au/.

AusAID. 2006. “M&E Framework Good Practice Guide” (Exposure Draft: March 17, 2006). Australian
Government AusAID. www.mande.co.uk/docs/MEF%20QAG_Guide%20(ver7)%201703016.pdf.

Bamberger, Michael, Jim Rugh, and Linda Mabry. 2006. RealWorld Evaluation: Working Under Budget, Time, Data,
and Political Constraints. Thousand Oaks, London, New Delhi: SAGE Publications.

Barton, Tom. 1997. “Guidelines to Monitoring and Evaluation: How Are We Doing?” CARE International,
Kampala, Uganda.
http://pqdl.care.org/pv_obj_cache/pv_obj_id_1DCDB23F514606B280C36E5E42B6EF31F9D70700.

Caldwell, Richard. 2002. Project Design Handbook. Atlanta: CARE International.

http://pqdl.care.org/practice/Design%20Monitoring%20and%20Evaluation/Forms/By%20Language.aspx.

DAC (Development Assistance Committee). 2008. “DAC Evaluation Quality Standards (draft).” DAC Evaluation
Network, Organization for Economic Co-operation and Development (OECD), Paris, France.
http://www.oecd.org/dataoecd/30/62/36596604.pdf.

Frankel, Nina and Anastasia Gage. 2007. “M&E Fundamentals: A Self-Guided Minicourse.” United States Agency
for International Development (USAID), Washington, DC.
http://www.cpc.unc.edu/measure/publications/pdf/ms-07-20.pdf.

IFAD (International Fund for Agricultural Development). 2002. “A Guide for Project M&E.” IFAD, Rome.
http://www.ifad.org/evaluation/guide/toc.htm.

IFRC (International Federation of Red Cross and Red Crescent Societies). 2007. “Monitoring and Evaluation in a
Nutshell.” IFRC, Geneva. http://participation.110mb.com/PCD/M%20and%20E%20guide%20final.pdf.

Rugh, Jim. 2008. “The Rosetta Stone of Logical Frameworks.” Compiled by Jim Rugh for CARE International and
InterAction’s Evaluation Interest Group. http://www.mande.co.uk/docs/Rosettastone.doc.

Stetson, Valerie, Guy Sharrock, and Susan Hahn. 2004. “ProPak: The CRS Project Package.” Catholic Relief
Services, Baltimore. http://crs.org/publications/list.cfm?sector=19.

1 Internet links have been provided where possible to download references and resources. If a link is no longer active, try a keyword
search on the publication title with an Internet search engine, e.g., Google.
Monitoring and Evaluation Planning • 19
 Annex I References and Resources

Theory of Change. 2008. A joint venture between ActKnowledge and the Aspen Institute Roundtable on
Community Change. http://www.theoryofchange.org/html/overview.html.

USAID (United States Agency for International Development). 1996. “Preparing a Performance Monitoring Plan.
Performance Monitoring and Evaluation TIPS: Number 7.” USAID, Washington, DC.
http://pdf.usaid.gov/pdf_docs/pnaby215.pdf

Resources
American Red Cross. 2006. “Integrated Planning Process: Project Design & Proposal Writing Guide.” American
Red Cross, Washington, DC. http://www.redcross.org/static/file_cont5976_lang0_2296.pdf.

Canadian International Development Agency. 2008. Useful M&E resources at

http://www.acdi-cida.gc.ca/CIDAWEB/acdicida.nsf/En/JUD-111795644-KJX.

DAC (Development Assistance Committee) Network on Development Evaluation. 2008. http://www.oecd.org/

document/35/0,3343,en_21571361_34047972_31779555_1_1_1_1,00.html. Organization for Economic Co-
operation and Development (OECD) M&E documents, publications, networks, and links, including criteria
and standards for evaluation.

Danida. 1996. “Logical Framework Approach: A Flexible Tool for Participatory Development.”
http://amg.um.dk/en/menu/TechnicalGuidelines/LogicalFrameworkApproach.

Department for International Development (DFID). 2006. “A Guide for DFID-contracted Research Programmes.”
http://www.dfid.gov.uk/research/me-guide-contracted-research.pdf.

IntraHealth International. 2008. Sample PI Monitoring and Evaluation Plan.

http://www.intrahealth.org/sst/tool5-1.html.

MandE News. http://mande.co.uk/. A comprehensive M&E resource site, with links to other M&E resources. For
example, multiple resources on logical frameworks and their pros and cons can be assessed at: http://portals.
wdi.wur.nl/ppme/index.php?Logical_Framework_Approach and http://www.mande.co.uk/logframe.htm.

MEASURE (Measure and Evaluation to Assess and Use Results Evaluation). 2008. http://www.cpc.unc.edu/
measure. Funded by USAID, the MEASURE framework offers M&E publications, tools, trainings, and other
resources.

Participatory Planning Monitoring & Evaluation (PPM&E) Resource Portal. 2008. Contains multiple resources for
M&E planning. http://portals.wdi.wur.nl/ppme/index.php?Home.

Performance Assessment Resource Centre (PARC). 2008. http://www.parcinfo.org/. Contains useful M&E
resources.

Resources for Methods in Evaluation and Social Research. 2008. http://gsociology.icaap.org/methods/. Lists useful
resources for program evaluation and social research methods.

Monitoring and Evaluation Planning • 20

 Annex I References and Resources

UNAIDS. 2002. Monitoring and Evaluation Operations Manual. Joint United Nations Programme on HIV/AIDS.
http://www1.worldbank.org/hiv_aids/docs/M&EManual.pdf.

UNICEF Evaluation. http://www.unicef.org/evaluation/. Provides useful M&E resources, as wells as links to other
M&E sites.

United Nations Development Programme (UNDP). 2008. http://www.undp.org/eo/. Useful M&E resources as
wells as links to other M&E sites.

United Nations Evaluation Group (UNEG). 2008. http://www.uneval.org/. M&E standards, resources, and events.

United Nations Population Fund (UNFPA). 2000. “The Programme Manager’s Planning, Monitoring and
Evaluation Toolkit: Planning and Managing an Evaluation.” UNFPA Office of Oversight and Evaluation.
http://www.unfpa.org/monitoring/toolkit.htm.

USAID (United States Agency for International Development). 2002. “Developing a Participatory Monitoring and
Evaluation Plan.” Produced for USAID by the Synergy Project. www.synergyaids.com/.

———. 2007. “Monitoring and Evaluation Systems Strengthening Tool.” USAID, Washington, DC.
http://www.pepfar.gov/documents/organization/79624.pdf.

USAID Center for Development and Evaluation Information. 1996. Evaluation Publications at http://www.usaid.
gov/pubs/usaid_eval. M&E resources from 1996, but still very relevant and useful, most notably the concise
series of TIPS documents. For example, “Preparing a Performance Monitoring Plan. Performance Monitoring
and Evaluation TIPS: Number 7.”

USAID/DCHA/FFP. 2005. P.L. 480 Title II Program Policies and Proposal Interim Guidelines. March 14, 2005,
Annex A, p. 9.

W. K. Kellogg Foundation. 2001. “Logic Model Development Guide.” W.K. Kellogg Foundation, Battle Creek,
Michigan. http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf.

World Bank, Independent Evaluation Group (IEG). 2008. International Program for Development Evaluation
Training. Course modules provide an overview of key M&E concepts and practices.
http://www.worldbank.org/oed/ipdet/modules.html.

Other Institutional Capacity-Building Modules

Produced by the American Red Cross/Catholic Relief Services (CRS) M&E Training Module Series. American Red
Cross/CRS, Washington, DC, and Baltimore. www.crs.org and www.redcross.org

Hagens, Clara, with assistance from Guy Sharrock. 2008. “Hiring M&E Staff.”

McMillan, Della E., and Alice Willard. 2008. “Preparing for an Evaluation.”

McMillan, Della E., Guy Sharrock, and Alice Willard. 2008. “Guidelines and Tools for the Preparation and Use of
Indicator Performance Tracking Tables (IPTT).”

Monitoring and Evaluation Planning • 21

Annex II
Logframe Example

Project Objectives Indicators Means of Verification Assumptions

Goal Impact Indicator G1 1. Household survey Security/political situation
Improve the health in target percent of children younger 2. Medical clinic records in Sri Lanka remains stable
communities in Matara than 5 who died from polio for country and allows for
District, Sri Lanka, by (child mortality rate) project implementation
reducing the incidence of
polio

Outcome 1 Outcome Indicator 1.a 1. Household survey Community acceptance of

Increased immunization of percent of children under 2. Vaccine records polio vaccine
children less than one-year 1 year who are fully
old in target communities immunized for polio
(immunization coverage)

Output 1.1 Output Indicator 1.1a. 1. Workshop attendance Community capacity to

Polio Immunization number of caretakers roster participate in project is
Awareness workshop (and participating in Polio 2. Focus group not compromised by other
people participation) Immunization Awareness development initiatives
workshops within the community,
natural disaster, and so on.

Activity A.1 Process Indicator A.1a Inventory of translated

Translation of polio number of polio booklets
immunization booklets immunization booklets
translated

Input I.1 Input Indicator I.1a Warehouse inventory for

Polio immunization number of polio booklets and printing
booklets, trainers, facilities, immunization booklets receipts
and so on printed

Monitoring and Evaluation Planning • 22

Annex III
Indicator Matrix Examples2
Indicators Indicator Methods/Sources Person/s Frequency/ Data Analysis Information Use
Definition Responsible Schedules
Example 1. Children refer 1. Endline ran- External Evalua- 1. Endline survey 1. Project manage- 1. Project
Outcome 1a. to age between domized house- tion Team depends on the ment team implementation
Percent of chil- 3 days and 1 hold survey project timeline during project and decision
dren younger year 2. Community 2. School Focus reflection meet- making with
than one-year 2. Fully im- focus group Group Discus- ing community
old who are munized for discussions sions (FGDs): 2. Post-project 2. Monitoring
fully immu- polio refers to teachers, meeting with process of
nized for polio 3. Community
getting polio key informant students, and implementing project with
(immunization immunization administration partners (Sri management of
coverage) interviews
vaccine accord- at the end of Lanka Red Sri Lankan Red
ing to MOH the project Cross Society) Cross Society
standards (1st 3. Beginning of facilitated by 3. Tsunami Recov-
dose at any data collection project man- ery Program
time after birth, according to the ager management
2nd dose at 1-2 project timeline
months later, 4. Impact evalu-
3rd dose at 6-12 4. Endline survey ation to justify
months after question- intervention
second vaccina- naire pending to Ministry of
tion) depends on the Health and
project timeline donors
3. Numerator:
number of fully
immunized
children in the
community
Denominator:
Total number
of children in
the community
per defined age
category

2 Note: The indicators in Annex III are illustrative and are not necessarily from the same project or objective.

Monitoring and Evaluation Planning • 23

 Annex III Indicator Matrix Example

Indicators Indicator Methods/Sources Person/s Frequency/ Data Analysis Information Use

Definition Responsible Schedules
Example 1. Caretak- Polio Immuniza- Education Field Attendance roster 1. Quarterly 1. Project imple-
Output II.a. ers refers to tion Workshop At- Officer (EFO): data collected at project report- mentation with
Number of community tendance Roster Priyantha Perera the workshop and ing and project community
caretakers beneficiaries reported quarterly reflection beneficiaries
participating in identified by meeting 2. Monitoring
Polio Immuni- Local Govern- 2. Project manage- process of
zation Aware- ment Agent ment team community
ness workshops (Grama Nila- during quar- outreach train-
dari) and who terly reflection ing for project
are participat- meeting with manage-
ing in project ment with Sri
activities Lankan Red
2. Polio Immuni- Cross Society
zation Aware- 3. Tsunami Recov-
ness Workshop ery Program
refers to a management
one-day train-
ing, which is 4. Impact evalu-
designed to ation to justify
convey knowl- intervention
edge on polio to Ministry of
immunization Health and
according to donors
Ministry of
Health recog-
nized standard
curriculum
3. Numerator:
number of ben-
eficiaries who
participate and
complete one-
day workshop

Example 1. “Schools” refers 1. Pre-arranged School Field Offi- 1. Checklist data 1. Post-drill 1. Project imple-
Outcome 2a. to K-12 in Mat- site visits dur- cer (SFO): Shantha collected quar- meeting with mentation with
Percent of ara District ing disaster Mande terly School Disaster School Disaster
target schools 2. Criteria of drill 2. FGDs: teachers, Committee, Committees
that success- “success”: drill 2. Complete disas- students, and facilitated by 2. Monitoring
fully conduct unannounced ter drill check- administra- SFO process of
a minimum of through early list and entered tion every six 2. Project manage- school outreach
one disaster warning sys- into quarterly months ment team training for
drill per quarter tem; response project report during quar- project with
3. Begin data
time under 20 (QPR) collection on terly reflection management
minutes, school 3. School focus 4/15/06 meeting with Sri Lankan
members report group discus- Red Cross
to designated 4. Scenario check- Society
sions (teachers, list completed
area per the students, ad- 3. Tsunami Recov-
School Crisis by 3/8/06
ministration) ery Program
Response Plan management
3. Numerator: 4. Impact evalu-
number of ation to justify
schools with intervention
successful to Ministry
scenario per of Education,
quarter. Ministry of
Denominator: Disaster Relief,
total number donors
of targeted
schools

Monitoring and Evaluation Planning • 24

Annex IV
Data Collection Tools and Techniques3
Case study: A detailed descriptive narrative of individuals, communities, organizations, events, program, or time
periods. They are particularly useful in evaluating complex situations and exploring qualitative impact.

Checklist: A list of items used for validating or inspecting that procedures/steps have been followed, or the
presence of examined behaviors.

Closed-ended (structured) interview: A technique for interviewing that uses carefully organized questions that
only allow a limited range of answers, such as “yes/no,” or expressed by a rating/number on a scale. Replies can
easily be numerically coded for statistical analysis.

Community interviews/meeting: A form of public meeting open to all community members. Interaction is
between the participants and the interviewer, who presides over the meeting and asks questions following a
prepared interview guide.

Direct observation: A record of what observers see and hear at a specified site, using a detailed observation
form. Observation may be of physical surroundings, activities, or processes. Observation is a good technique for
collecting data on behavior patterns and physical conditions.

Focus group discussion: Focused discussion with a small group (usually 8 to 12 people) of participants to record
attitudes, perceptions, and beliefs pertinent to the issues being examined. A moderator introduces the topic and
uses a prepared interview guide to lead the discussion and elicit discussion, opinions, and reactions.

Key informant interview: An interview with a person having special information about a particular topic. These
interviews are generally conducted in an open-ended or semi-structured fashion.

Laboratory testing: Precise measurement of specific objective phenomenon, for example, infant weight or water
quality test.

Mini-survey: Data collected from interviews with 25 to 50 individuals, usually selected using non-probability
sampling techniques. Structured questionnaires with a limited number of closed-ended questions are used to
generate quantitative data that can be collected and analyzed quickly.

Most significant change (MSC): A participatory monitoring technique based on stories about important or
significant changes, rather than indicators. They give a rich picture of the impact of development work and
provide the basis for dialogue over key objectives and the value of development programs.

Open-ended (semi-structured) interview: A technique for questioning that allows the interviewer to probe and
follow up topics of interest in depth (rather than just “yes/no” questions).

3 Note: This list is not exhaustive, as tools and techniques are emerging and evolving in the M&E field.

Monitoring and Evaluation Planning • 25

 Annex IV Data Collection Tools and Techniques

Participant observation: A technique first used by anthropologists; it requires the researcher to spend considerable
time with the group being studied (days) and to interact with them as a participant in their community. This
method gathers insights that might otherwise be overlooked, but is time-consuming.

Participatory rapid (or rural) appraisal (PRA): This uses community engagement techniques to understand
community views on a particular issue. It is usually done quickly and intensively – over a 2 to 3-week period.
Methods include interviews, focus groups, and community mapping.

Questionnaire: A data collection instrument containing a set of questions organized in a systematic way, as well as
a set of instructions to the enumerator/interviewer about how to ask the questions (typically used in a survey).

Rapid appraisal (or assessment): A quick cost-effective technique to gather data systematically for decision-
making, using qualitative and quantitative methods, such as site visits, observations, and sample surveys.
This technique shares many of the characteristics of participatory appraisal (such as triangulation and multi-
disciplinary teams) and recognizes that indigenous knowledge is a critical consideration for decision-making.

Self-administered survey: Written surveys completed by the respondent, either in a group setting or in a separate
location. Respondents must be literate (for example, it can be used to survey teacher opinions).

Statistical data review: A review of population censuses, research studies, and other sources of statistical data.

Survey: Systematic collection of information from a defined population, usually by means of interviews or
questionnaires administered to a sample of units in the population (e.g., person, beneficiaries, and adults).

Visual techniques: Participants develop maps, diagrams, calendars, timelines, and other visual displays to
examine the study topics. Participants can be prompted to construct visual responses to questions posed by the
interviewers, for example, by constructing a map of their local area. This technique is especially effective where
verbal methods can be problematic due to low literate or mixed language target populations, or in situations
where the desired information is not easily expressed in either words or numbers.

Written document review: A review of documents (secondary data) such as project records and reports,
administrative databases, training materials, correspondence, legislation, and policy documents.

Monitoring and Evaluation Planning • 26