Project

Monitoring and Evaluation

 Chapter Two:
Developing Indicators
 Contents

 Definition: Indicators
 Guidelines for Developing Indicators
 Setting Priorities for Monitoring and Evaluation
 Selecting which results to monitor and evaluate
 Identifying which assumptions to monitor and evaluate
INTRODUCTION
 Introduction
 Monitoring – Routine tracking of data to learn to what extent
implementation is occurring and progress is being made, e.g. tracking
inputs and outputs. Monitoring data indicates:
 How programs are conducted: Is the intervention on track or on
budget? (inputs & activities)
 Program’s level of performance: Is it reaching the desired
number of households? (outputs)

 Evaluation – Systematic collection of information to learn why

and/or to what extent a program has been successful
/unsuccessful. Ascertains whether the inputs and outputs
have produced the desired results – that is, evaluating their
effectiveness

Evaluation data indicates:

• whether programs achieved what they were intended to achieve
(outcomes & impact)
Definition: Indicators
 What is Indicator?
• Indicator is a measure used to demonstrate
 Change in a situation of a project or program, or
 Change in the progress in project/program
 Change in results of project/program,
 Change an activity of project/program.
• Indicators enable project managers to track project progress, to
demonstrate results and, if necessary, to take corrective action to improve
project management
• Indicator is used to assess the state of a project/program by defining its
characteristics or variables, and then tracking changes in those
characteristics over time or between groups.
• Indicator is a quantitative metric that provides information to monitor
performance, measure achievement and determine accountability
• Indicator is the basis of any effective monitoring and evaluation
system

Indicators has a specific, observable and measurable
characteristic
that can be used to show changes or progress.
 What is Indicator?...

Indicator should be focused, clear and specific. The change
measured by the indicator should represent progress that the
program hopes to make.

Indicator should be defined in precise, unambiguous terms that
describe clearly and exactly what is being measured.

Indicators which do not specify a particular level of achievement
-- the words like ―improved”, “increased”, or “decreased‖
are
not belong in an indicator.

Indicator should give a relatively good idea of the data required
and the population among whom the indicator is measured.

Indicator should already be established during the project design
phase, as part of the project proposal.

 What is Indicator?
• An indicator is a quantitative or qualitative variable that
allows changes produced by an intervention relative to what
was planned to be measured.
• It provides a reasonably simple and reliable basis for
assessing achievement, change or performance.

• An indicator is preferably numerical and can be measured

over time to show changes.

• Indicator has the following essential components.

1. What is to be measured? What is going to change?
(E.g., participants reporting higher school attendance of
girls in a village)
 Why is it helpful to specify indicators?

• Objectives are abstract concepts, and it is thus difficult to measure

performance against them. Indicators are ways of quantifying objectives or
sub-objectives.
• The most important role of indicator sets may be in framing the issues
and defining the problems, rather than suggesting the solutions.
Indicators can help to clarify and quantify policy objectives and strategies,
but there is a danger in excessive use of targets
 Essential components of an Indicator?
Indicator has the following essential components.
1. What is to be measured? What is going to change?
(E.g. participants reporting higher school attendance of girls in
a village)
2. Unit of measurement to be used
(to describe the change, e.g., percentage)
3. Pre-program status or baseline
(e.g., 40 per cent in 2007)
4. Size, magnitude or dimension of intended change
(e.g., 75 per cent in 2008)
5. Quality or standard of the change to be achieved
(e.g., improvement such that girls obtain higher
grades)
6. Target populations(s)
(e.g., girls vulnerable to trafficking from villages in southern
district)
7. Time frame (e.g., January 2008 to January 2009
 Essential Components of Indicator
The essential components of an indicator are:
1. Title: A brief heading that captures the focus of the indicator.
2. Definition: A clear and concise description of the indicator.
3. Purpose: The reason that the indicator exists; i.e. what it is for
4. Rationale.: underlying principle(s) that justify the development and
The
deployment of the indicator; i.e. why the indicator is needed and useful.
5. Method of measuring : The logical and specific sequence of operations used
to measure the indicator; e.g. data collection tools, sampling frame and quality
assurance.
6. Numerator : The top number of a common fraction, which number
indicates the of parts from the whole that are included in the calculation.
7. Denominator: The bottom number of a common fraction, which indicates
the number of parts in the whole.
8. Calculation : The specific steps in the process to determine the indicator value
9. Data Collection Method : The general approaches (e.g. surveys, records, models,
estimates) used to collect data.
10. Data collection tools : The specific tools used to collect data.
11. Data collection frequency: The intervals at which data are collected;
e.g.
 Essential Components of Indicator(Cont…)

12. The relevant subgroups that collected data can be

disaggregation : in order to more precisely understand and analyze the
separated into
findings. Common subgroups include sex, age and risk population.

13. Guidelines to interpret and use data: Recommendations on how best to

evaluate and apply the findings; e.g. outlining what it means if the indicator
shows an increase or a decrease in a particular measure.
14. Strengths and weakness: A brief summary of what the indicator does well
and not so well.

15. Challenges: Potential obstacles or problems that may have an impact on

the use of an indicator or on the accuracy/validity of its findings.
 Characteristics of Good Indicators

Valid: accurate measure of a behavior, practice, task that is the
expected output or outcome of the intervention

Reliable: consistently measurable over time, in the same
way by different observers

Precise: operationally defined in clear terms

Measurable: quantifiable using available tools and methods

Timely: provides a measurement at time intervals relevant and
appropriate in terms of program goals and activities

Programmatically important: linked to the program or
to
achieving the program objectives (Gage and Dunn, 2009)
 Characteristics of Good Indicators
 Indicator also should be ― CREAM‖ or
―SMART‖
C= Clear S= Specific
R= Relevant M= Measurable
E= Economic A=
A= Adequate R= Relevant
Attainable
M= Monitorable T= time
bounded
 Clear = Precise and unambiguous
Relevant = Appropriate to the subject at hand
Economic = Available at a reasonable cost
Adequate = Provide a sufficient basis to assess
performance Monitorable= Amenable to independent
validation
 If any one of these five criteria are not met, formal performance indicators
will suffer and be less useful
 Characteristics of Good Indicators
• Performance indicators should be relevant to the desired
outcome, and not affected by other issues tangential to the
outcome.
• The economic cost of setting indicators should be considered.
This means that indicators should be set with an understanding
of the likely expense of collecting and analyzing the data
• Indicators ought to be adequate. They should not be too
indirect, too much of a proxy, or so abstract that assessing
performance becomes complicated and problematic.
• Indicators should be monitorable, meaning that they can be
independently validated or verified, which is another
argument in favor of starting with quantitative indicators as
opposed to qualitative ones. Indicators should be reliable and
valid to ensure that what is being measured at one time is what
is also measured at a later time— and that what is measured is
actually what is intended
 The Need of Developing Indicators
The importance of developing indicators are:
1. Indicators provide a means to track progress
• In order to track the way in which a Project/program evolves
and its progress towards reaching certain goals. you need to
be able to measure this change over time.
2. Indicators provide data that can be measured to show changes
in relevant project/program areas.
3. Indicators provide signal to the weak performance of a
project
4. Indicator make it easier to report results to donors of others
5. Indicators are standardized measures that allow
for
comparisons over time, over different geographic
areas and/or across programs.
 The Need of Developing Indicators
6. Indicators describe successful results of the program through
the establishment of targets.
7. Indicators help in outlining or describing each result
(outcomes/ outputs, effects and Impact/goal) by making
statements more clear and precise.
8. Indicators help to verify outcomes in an "objective" manner
so as to arrive at an agreement on the progress as
demonstrated by the evidence.
9. Indicators constitute the foundation for program M &E.. The
progress towards achieving anticipated results can be
measured at the outcomes/outputs and effect levels, at which
the programme is more "accountable".
10. Indicators constitute the foundation for making informed
decisions in the case of corrective measures.
 How many indicators are enough?

Some guidelines to follow when selecting indicators:
o
At least one or two indicators per result (ideally, from different
sources)
o
At least one indicator for every core activity (e.g.
training, airing of TV spot)
o
No more than 8-10 indicators per area of significant program
focus
o
Use a mix of data collection strategies and sources

Indicators are required for all Levels of M&E Systems. Setting
indicators to measure progress in inputs, activities, outputs,
outcomes, and goals(Impact) is important in providing necessary
feedback to the management system.
 Steps in Developing Indicator

Step 1: Identify What to Measure

( Identification of Indicator)
Step 2: Use the SMART Process to Develop High Quality
Indicators. ( Indicator development and Evaluation: the process of
evaluation based on four criteria in turn: ‗validity‘, ‗statistical power‘,
‗technical specification’ and ‘fairness’.)
Step 3: Establish a Reference Point

Step 4: Set Targets

Step 5: Determine the Frequency of Data Collection.

 Step 1: Identify What to Measure
 The first step to create project/program indicators for M & E is to
determine which characteristics of the Project/program are most
important to track.

 A project/program will use many indicators to assess different

types and levels of change that result from the intervention, like
changes in certain health knowledge, attitudes, and behaviors
among the priority audience(s).

 Referring to the program‘s logic model can help to identify key

program areas that need to be included in monitoring indicators.
Step 2: Use SMART Process to Develop High-Quality Indicators
 One way to develop good indicators is to use the SMART
criteria, where
1. S=Specific: Indicator should accurately describe what is intended
to be measured, and should not include multiple measurements in
one indicator.
Outcome Indicator
Children treated for 1. Increased utilization of clinics Which indicator is
malaria 2. Increased use of malaria drugs more specific?

2. Measurable: be clear about how it will be measured. Regardless of

who uses the indicator, consistent results should be obtained and
tracked under the same conditions.
i. % of health centers without stock of drugs x , y, & z for Which indicator is
more than a week at a time measurable
ii. % of health centers with availability of drugs
Step 2: Use SMART Process to Develop High-Quality Indicators…
3. Attainable: Collecting data for the indicator should be
simple, straightforward, and cost-effective.
i. HIV prevalence aiming 15-24 year old pregnant women Which indicator
is attainable
ii. HIV prevalent among the total population

4. Relevant: The indicator should be closely connected with each

respective input, output or outcome.(Logically and closely linked to a
program‘s efforts)

i. Life expectancy Which indicator

is relevant
ii. % of children fully immunized at year

5. Time-bound: The indicator should include a specific time frame.

 Step 3: Establish a Reference Point
 To show change or progress in a project/program, a reference
point ( baseline) must be established.

 A reference point is a point before, during, or at the end of a

program where indicators are used to establish the state of the
program in order to provide a point of comparison as the
program progresses.
 The reference point is often chosen before or at the start of a
program to assess the progress of the program over time. At
the same time, implementation timelines do not always allow
for baseline data to be collected. In these cases reference
points can be set up at other times in the program.
 Step 4: Set Targets
 Target:
 define as the path and end destination of what a program hopes to
achieve;
 is pre-established goals that are set for the program;
 is a number or percentage which will measure success. Once the
reference point is established, determine what changes should be
seen in the program‘s indicators that would reflect progress
towards success.
 When establishing targets, consider:
1. Baseline data or reference point: This sets a certain point in
time in the program from which to observe change over time.
2. Stakeholder’s expectations: Understanding the expectations of
key stakeholders and partners can help set
reasonable expectations for what can be achieved.
 Step 4: Set Targets (Cont…)

3. Recent research findings:


Do a literature search, if literature is available, for the latest
findings about local conditions and the program sector,

4. Accomplishments of similar programs:

 Identify relevant information on similar programs that have
been implemented under comparable conditions. Those with
a reputation for high performance can often provide critical
input on setting targets.
 Step 5: Determine the Frequency of Data Collection
• As a last step, consider how often data should be collected in order
to properly track the program‘s progress. These designated points
in time are usually referred to as benchmarks.
Benchmarks are designated points in time in which data are
collected to track the program’s progress
• Ideally, at least one round of data collection should occur between
the reference point and the end of the program.
• If the data are collected at the midpoint of the program, it is called
a midline.
• If data are collected at the end of the program, it is
called an endline.
• The frequency of collecting data is mostly dependent on the cost
and length of the program .
Step 5: Determine the Frequency of Data Collection…
 Quantitative and Qualitative
indicators
 Quantitative indicators:
 measures change in numeric terms such as number,
percentage, frequency, ratio, proportion, etc.
 should be reported in terms of a specific number (number,
mean, or median) or percentage. ―Percent's can also be
expressed in a variety of ways, e.g., percent that fell into a
particular outcome category . . . percent that fell above or
below some targeted value . . . and percent that fell into
particular outcome intervals . . . ‖ (Hatry 1999, p. 63).
– Percentage of girls and boys attending primary school.
– Female and male unemployment rates in rural areas.
 Quantitative and Qualitative indicators
 Qualitative indicators:
 give information on judgements, opinions, perceptions, or attitudes
of people and groups. It can be expressed in terms of satisfaction,
perception of change, applicability, etc.
– Satisfaction of water service beneficiaries.
– Community interests in project activities
 Qualitative indicators: provide insights into changes in institutional
processes, attitudes, beliefs, motives and behaviors of individuals‖.
 A qualitative indicator might measure perception, such as the level of
empowerment that local government officials feel to adequately do their
jobs.
 Qualitative indicators might also include a description of a behavior, such
as the level of mastery of a newly learned skill.
 Although there is a role for qualitative data, it is more time consuming to
collect, measure, and distill, especially in the early stages.
 Furthermore, qualitative indicators are harder to verify because they often
involve subjective judgments about circumstances at a given time.
 Advantage and Disadvantages of using pre-designed Indicators
Advantage:
• They can be aggregated across similar projects,
programs, and policies.
• They reduce costs of building multiple unique
measurement systems.
• They make possible greater harmonization of donor requirements.

Disadvantage:
• They often do not address country specific goals.
• They are often viewed as imposed, as coming from the top down.
• They do not promote key stakeholder participation and ownership.
• They can lead to the adoption of multiple competing indicators.
 Types of indicators

• Related to the level of assessment, Indicators can be

grouped in different tape
• These are :
1. Input Indicator
2. Activity Indicator,
3. Output Indicator,
4. Outcome Indicator and
5. impact Indicator.
 Types of Indicators…
1. Input Indicator:
 Related to a resource used in a project. It refers to what you need to implement the
program.
 These include: financial (money/funds,) human, trained personnel/staff curricula,
materials, other resources and supplies, other contributions (volunteers)
 Developing input indicators help to monitor the availability of essential resources
2. Activity Indicators:
 is about what the project/program does, or what the program
provides to accomplish its objectives
Example: Provide training workshops for personnel and site managers : Teach classes,
Develop curriculum, supervise.
 are essential in order to understand the extent to which a project was
delivered as planned, and to highlight obstacles to implementation.
 are most valuable when you are able to connect a given set of activities to a
particular output or outcome.
 should include three essential elements;
• who conducted the activity,
• what they did, and
• where were they working
 Types of Indicators…
3. Output indicators
• Output Indicator is what the program delivers. It is about what the
project produces. It is the immediate results of program/project activities.
• It measure the most immediate or delivered outputs of a development intervention
(during and at the end of implementation).
• It highlights the physical quantities of goods produced or services supplied through
the development intervention.

• Include Immediate tangible and intangible products that result from

project activities. For example: In a training projects
• the number of counseling sessions completed,, the number of
people reached , the number of materials distributed, training and
technical assistance provided; the developed standards and
legislative documents; Investing in buildings and infrastructure;
and hiring staff required to implement a project, etc.
 When combined with measures of inputs and activities, output indicators
can provide measures of economy and efficiency, describing the
relationship between investments in a project and products.
 Types of Indicators…
4. Outcome indicator:
 Outcome indicator is about what the project achieves
 It is when the results of the program or changes that occur both
immediately or some time after activities are completed. These
include changes in knowledge, behavior , attitudes, skills , access to
services, policies and environmental condition.
 The intermediate changes (short and medium) that a program
effects on target audiences or populations,
 Well-designed outcome indicators are important mechanisms for
ensuring transparency and accountability, describing the return on
investments and the benefits that a project delivers.
 Outcome indicators should measure improvements because if
intervention that result from these activities and outputs
 help in measuring the short and medium term effects (or
outcomes) resulting from beneficiary use of outputs/outcomes
produced through the development intervention
 Types of Indicators…
Outcome Indicators…
• Effective outcome indicators typically combine quantitative and qualitative
measures, describing the number of people benefitting from a project and the
nature of those benefits.
• For example: outcome indicators for a crime reduction project may include
 changes in the number of people experiencing violent crime (quantitative
indicator) alongside
 changes in perceptions of public safety (a qualitative indicator).
• Generally, outcome indicators highlight a change in behaviour, attitude,
practices or an improvement in skills and capacities.
• They can also be used to measure a change in preferences or the satisfaction of
beneficiaries with regard to the quality of received goods and services.
• Comparison of Output and outcome indicators:
• Output indicator : help in measuring immediate deliverables or products
of the intervention
• Outcome Indicators: help in measuring short or medium term
(intermediate) effects that result from the project interventions such as
change in attitudes, knowhow, skills, etc.
 Types of Indicators…
5. Impact Indicator:
 is about how the project contributes to higher-level strategic goals
 describe progress made towards higher-level goals.
 Is the longer range, cumulative effect of programs over time on what they
ultimately aim to change.
 Is long term effect of the interventions. Possible examples:
• Health and longevity improved (nutrition programs), Violence against
children decreases (parenting programs, etc.), Decreases in HIV
mortality and morbidity (HIV programming), Improved economy
(education programs/savings and loans groups)\
 are similar to statements of purpose, describing those objectives that are shared
with other development partners and national government agencies, such as
reducing poverty, increasing access to justice, improving the accountability
of national institutions. Or Often, the effect will be on a population-level
health outcome, such as a change in HIV infection, morbidity and
mortality.
• Impacts are rarely, if ever, attributable to a single program, but a program may,
with other programs, contribute to impacts on a population
 Types of Indicators…
 Comparison b/n Impact and Outcome Indicators:
• Impact indicators: typically represent long-term and high-
level goals that are beyond the immediate control of an
individual project.
• Outcome Indicators:
 are measure a short or medium term effects of the
intervention of a project.
 are directly linked to your project outputs. Because
outcome indicators define the criteria for assessing
your project is successful,
whether the should be and achievable
resources.
realistic given your capacity and
 should only state what will be measured, rather
than baseline data or target figures.
 Types of Indicators

Point of What is
Indicators
measurement measured

Outputs Effort Implementation of activities

Use of outputs and sustained

Outcomes Effectiveness
production of benefits

Difference from the original

Impact Change
problem situation
Types of Indicators
ORT= Oral rehydration therapy
 Examples of result Chains
Impact
Projects Activities Outputs Outcomes /long-term
outcome/
• • Teachers • •
Teachers using new Increase Increased
training, methods completion test score
Education • Textboo • Use of rate •
k textbooks • Increase Increase
printed, attendance labor
delivered productivity

• Doctors • New • Increased •

hired Doctors use of Improved
• Birth practicing health Maternal
Health attendants • Attendants clinics for mortality
trained applying
methods deliveries
Question: Identify the sequence of Input, Activities, Output and
Outcome?

1. Information is available for parents about the importance

of beast feeding
2. Children in community healthier
3. fewer children are having diarrheal diseases
4. Mothers breast feeding rather than using formula
5. New funds available to implement a health project to
reduce
child malnutrition rates
6. Design information campaigns on the importance of
breast feeding
 Answer: Identify the sequence of Input, activities,
output and outcome
5. New funds available to implement a safety net project
to reduce child malnutrition rates ( Input)
6. Design information campaigns on the importance of breast
feeding ( Activity)
1. Information is available for parents about the importance
of breast feeding (output)
4. Mothers breastfeeding rather than using formula
( Outcome)
3. Fewer children are having diarrheal diseases ( Outcome)
2. Children in community healthier (Higher level outcome =
Impact)
• Construction of hydro Dam project
1. What is the input used ?

2. What are the different activities done in constructing?

3. What is the output?

4. What is the outcome ?

5. What id then impact?

 Example: How to Designing indicators: 7 stages
Objective of the intervention: Agricultural food production increased
1. Identify the indicator: Small farmers increase their wheat production
2. Set the quantity: Increase in production by 50%
3. Specify the group: 10,000 small farmers (owning 3 ha or less)
4. Specify the quality: Use of new varieties of bread-making quality
5. Set the time frame: Between October 2010 and October 2012
6. Specify the location: Province X

 Construct the Objectively Verifiable Indicator: 10,000 small

farmers (owning 3 ha or less) of province X, increased their wheat
production by 50% between October 2010 and October 2012, using new
varieties of bread-making quality.
Example of Millennium Development Goal, Targets, and
Indicators
• Goal: Eradicate extreme poverty and hunger
• Target l. Halve, between 1990 and 2015, the proportion
of people whose income is less than US$1 a day
• Indicator 1. Proportion of population below US$1 per day
• Indicator 2. Poverty gap ratio (incidence × depth of poverty)
• Indicator 3. Share of poorest quintile in national consumption

• Target 2. Halve, between 1990 and 2015, the proportion

of people who suffer from hunger
• Indicator 4. Prevalence of underweight children (under 5 years
of age)
• Indicator 5. Proportion of population below minimum level of
dietary energy consumption
Examples that should give you some idea of the kinds of
indicators you can use (especially if you want to measure
impact):
Economic development indicators:
• Average annual household income (#)
• Employment, by age group (%)
• Earned income levels (#)
• Per capita income (#)
• People living below the poverty line (%)
Political/organisational development indicators:
• Number of NGOs/CSOs (#)
• Participation levels in organised sports (# or %)
• Number of youth groups (#)
• Participation in youth groups (# or %)
• Number of groups for the elderly (#)
• Structure of political leadership, by age and gender (%)
• Participation in elections, by age and gender (%)
• Number of public meetings held (#)
• Participation in public meetings, by age and gender (# or
%)
Social development indicators:
• Death rate (%)
• Life expectancy at birth (#)
• Infant mortality rates (%)
• Causes of death (% or #)
• Number of doctors/nurses per capita (#)
• Number of hospital beds per capita (#)
• Literacy rates, by age and gender (%)
• Student/teacher ratios (#:1)
• Number of suicides (#)
• Cause of accidents (% or #)
• Number of homeless (#)
• Number of violent crimes (#)
• Birth rate (%)
• Fertility rate (%)
• Rate of HIV infection (%)
• Rate of AIDS-related deaths (%)
• Church participation by age and gender (%)
 Guidelines for
Developing Indicators
 Guidelines for Developing Indicators

The guidelines are structured into five sections which represent

the five main stages in the development and reporting of
indicators:
Establishing the purpose of the indicators
Designing the conceptual framework
Selecting and designing the indicators
Interpreting and reporting the indicators
Maintaining and reviewing the indicators
 1. Establishing the purpose of the indicators
• In establishing the purpose of the indicator, it requires to
• define critically the indicator
• identify clearly the target audience
• determine the scope of the indicator
• keeping the project focused.
• Important questions to consider at this stage include:
Who will be responsible for the final selection and publication of
the indicators?
How will key stakeholders be involved?
Will an expert group be established to provide specialist
advice?
Will public consultation be undertaken?
How will the indicators be sustained and funded over time?
2. Designing the Conceptual framework for indicators
• In the development of a set of indicators, conceptual framework
is very important b/s
• It provides a formal way of thinking about the topic area. It
is a valuable tool for building a coherent set of
indicator
• It helps to ensure the selection of relevant and
balanced
indicators.
• It provides a useful device for organizing and reporting on
indicator in structured and meaningful way
• The absence of a framework can result in the generation of an
eclectic mix of indicators, with no clear rationale for their
selection.
 3. Selecting the indicators
• The process of selecting indicators is generally an iterative
(procedural that involves repetition of steps) that undertaken in
consultation with interested stakeholders.
• Care is needed in selecting indicators which resonate with
the
target audience and yet which are technically sound.
• Selection criteria should be used as a tool to evaluate the
proposed indicators during the selection phase to ensure they are
relevant, analytically sound and measurable.
• Experts should be consulted as they will have a good sense of the
data issues and the scientific rigor regarding the proposed
indicators.
 Set of criteria for indicator selection
 Valid and meaningful – an indicator should adequately reflect the
phenomenon it is intended to measure and should be appropriate to the
needs of the user.
 Sensitive and specific to the underlying phenomenon – sensitivity
relates to how significantly an indicator varies according to changes in
the underlying phenomenon.
 Grounded in research – awareness of the key influences and factors
affecting outcomes.
 Statistically sound – indicator measurement needs to be
methodologically sound and fit for the purpose to which it is being
applied.
 Intelligible and easily interpreted – indicators should be sufficiently
simple to be interpreted in practice and intuitive in the sense that it is
obvious what the indicator is measuring.
 Relate where appropriate to other indicators – a single indicator
often tends to show part of a phenomenon and is best interpreted
alongside other similar indicators.
 Set of criteria for indicator selection (Cont…)
 Allow international comparison – indicators need to reflect New Zealand-
specific goals, but where possible should also be consistent with those used in
international indicator programs so that comparisons can be made
 Ability to be disaggregated over time – indicators should be able to be
broken down into population sub-groups or areas of particular interest,
such as ethnic groups or regional areas.
 Consistency over time – the usefulness of the indicators is directly
related to the ability to track trends over time, so as far as possible
indicators should be consistent.
 Timeliness – there should be minimal time lag between the collection
and reporting of data to ensure that indicators are reporting current rather
than historical information
 Linked to policy or emerging issues – indicators should be selected to
reflect important issues as closely as possible. Where there is an
emerging issue, indicators should be developed to monitor it.
 Compel interest and excite - the indicator should resonate with the
intended
 4. Interpreting and Reporting Indicators
• The interpretation and reporting of indicators is a critical stage in
the development process, as it bridges the gap between
measurement and understanding.
• In deciding how to report the indicators, the target audience should
be kept in mind. A mix of graphs and commentary is generally
more effective for a public audience than large amounts of
commentary. In some instances, producing a summary report and a
larger technical report may be the best approach for reaching
different audiences.
• The presentation of the indicators should be objective and policy-
neutral. Contextual information, such as changes in key
demographic variables should be provided where appropriate to
assist readers to interpret the indicator.
 Steps in indicator selection that help to ensure that
indicators are appropriate for their intended purpose.
 4. Interpreting and Reporting Indicators …
• In addition to commenting about each indicator, it is often
useful to discuss the overall picture based on all the indicators.
It is also informative to discuss linkages between the
indicators.
• All aspects of the indicators should be readily transparent in
the reporting of indicators. To ensure that this is the case,
detailed meta data describing each indicator and how it was
designed should be made available.
 5. Maintaining and reviewing the indicators
• Indicators should be subject to ongoing assessment. They
should be open to challenge, discussion and modification to
reflect changing objectives, the emergence of new issues and
improvement in measurement techniques and data availability.
• It is useful to have a formal mechanism or process for
reviewing the indicators that will allow for the creation of new
indicators and the revision of existing indicators.
• This should involve open consultation with stakeholders,
including technical and subject-matter experts, data providers,
the target audience for the indicators, other interested groups
and the community. There are a variety of ways that this can
be achieved, such as through public meetings, focus groups
and written submissions.
 Process of Developing Indicators

Step 1. Define
indicator purpose

Step 2:
Step 5:
Designing the
Marinating
Conceptual
and reviewing
framework
the Indicator

Step 4:
Interpreting and Step 3 Selecting
Reporting the Indicator
indicators
 Process of Developing Indicators
Developing indicators to measure progress is a useful and
necessary step:
 During the planning phase: indicators can help to define the
vision and goals for the community.
 During the acting phase: the indicators identify the data that
should be gathered.
 During the evaluating phase: indicators are used to assess
progress and determine what new programs or projects to
undertake to ensure a sustainable future for the community‘s
forest resources.
 Plan

Develop

Indicator

Evaluatio Act
n

Developing Indicators as Part of Plan-Act-Evaluate Process

 Setting Priorities for
Monitoring and Evaluation
 Key principles when deciding on M&E priorities
Key principles to bear in mind when deciding on M&E priorities
 :
areAvailability the human resources and human It is
essential
Capacityto: have enough staff, who are properly trained, with dedicated
time to fulfill M&E responsibilities.
 Support M&E activities that build one M&E Develop
system .
M&E partnerships and collaborative networks to foster coordination at
all levels to minimize fragmentation and duplication of effort.
 Data should be collected with the intention of being used. Data
collection needs to address key questions for national program
management and improvement:
(1) Are we doing the right things?
(2) Are we doing them right?
(3)Are we doing them on a large enough scale to make a
difference?
 Key principles when deciding on M&E priorities…
 Not everyone working within the M&E system needs to be
involved in all aspects of M&E:
• B/s of Scarcity Resources: financial, human, time , etc.
• As a result
• Conduct basic program input and output M &E for
the purposes of good program management and for
selecting a few indicators to report to key stakeholders to
whom the program is accountable.
• Conduct some basic process evaluations (such as
implementation assessments, quality assessments, basic
operations research, case studies, cost analyses).
 Process of Monitoring and Evaluation Priority Setting

• The process of priority setting starts by addressing three basic

considerations in general, they are:

1. Links with Planning,

2. Participation, and

3. Availability of Inputs
 Process of M & E Priority Setting…
1. Links with Planning:
• The design and implementation of a priority setting
depends on the type of plan that it supports.
• Priorities need to be established b/s planning always
confronts a scarcity of human and financial resources.
Therefore M & E priority setting depends on the type of
plan. For example:
• Priority setting exercise in national plan is part of
planning for long-term research system
development; it should not be conducted more
frequently than every 10 years.
• Priority setting for an institute is part of strategic or
corporate planning exercises and should be done
every 5 to 10 years.
• Priority setting to support planning activities at
program level is conducted for every 3 to 5 years.
 Process of M & E Priority Setting…
2. Participation:
• A participatory approach to leads
decisions and priority setting to better
implementation. enhances their of
• The main interested partieschances
should be represented successful
in the priority
setting exercise.
• Wide stakeholder and client representation is very important but
not easy to organize. For some client groups, such as resource-
poor farmers, it is a particular challenge to identify persons who
can validly represent the group's research needs.
 Process of M & E Priority Setting…
3. Time and Information on inputs resources:
 A crucial question for the design of an appropriate priority setting process is:
• how much time and money (for materials, information and purchased) to
invest. The actual ability to find the desired time, money and information.
• When setting monitoring priorities, it is always important to consider: it is
manageable given the available human and financial resources.
• The cost of monitoring depends on how data collection is organized (for
example, through group meetings, workshops, case studies, field visits etc.)
and can be as high or low as the organization wants.
• The M&E staff confirms with the users that the priorities are realistic by
asking,
- Who will be responsible for collecting the data?
- How often will it be collected?
- What methods are best suited to collecting the data?”, etc.
 Once the links with planning, the participating groups and the available inputs
are established, the priority setting exercise can be implemented. This is done
through four Steps. The capacity to design and implement priority setting
exercises needs to be developed like that any other area of research. The steps
are as follows
 Selecting which results
to monitor and evaluate
Evaluation Questions and the logframe
 The Uses of Monitoring and Evaluation Findings
Monitoring and evaluation reports can play many different
roles, and the information produced can be put to very different
•uses:
Todemonstrate accountability delivering on political
promises
— made to citizenry and other stakeholders
• To convince —using evidence from findings
• To educate —reporting findings to help organizational learning
• To explore and investigate —seeing what works, what does not,
and why
• To document —recording and creating an institutional memory
• To involve —engaging stakeholders through a participatory process
• Togain support —demonstrating results to help gain
support among stakeholders
• To promote understanding — results to
reporting
understanding of projects, programs, enhance
and reports serve many purposes. policies. Evaluation
 The Power of Measuring Results
•If you do not measure results, you cannot tell success
from failure.
• If you cannot see success, you cannot reward it.
•If you cannot reward success, you are probably
rewarding failure.
• If you cannot see success, you cannot learn
from it.
• If you cannot recognize failure, you cannot
correct it.
•If you can demonstrate results, you can win public
support
Key Features of Implementation Monitoring vs Results Monitoring
Elements of Implementation Monitoring
(traditionally used for projects)
• Description of the problem or situation
before the intervention
• Benchmarks for activities and immediate outputs
• Data collection on inputs, activities, and immediate outputs
• Systematic reporting on provision of inputs
• Systematic reporting on production of outputs
• Directly linked to a discrete intervention (or
series of
interventions)
• Designed to provide information on administrative,
implementation, and management issues as opposed to
broader development effectiveness issues.
Key Features of Implementation Monitoring vs Results Monitoring
Elements of Results Monitoring
(used for a range of interventions and strategies)
• Baseline data to describe the problem or situation before the
intervention
• Indicators for outcomes
• Data collection on outputs and how and
whetherthey contribute toward achievement of outcomes
• More focus on perceptions of change among stakeholders
• Systemic reporting with more qualitative and
quantitative on the progress toward outcomes
information
• Done in conjunction with strategic partners
• Captures information on success or of partnership
failure strategy in achieving desired outcomes