Unit Six

Monitoring and Evaluation

Basics of Project Management: Monitoring and Evaluation

What is Evaluation?

• Evaluation is a periodic assessment of an ongoing or completed

project - its design, implementation and results.

• Evaluation involves identifying and reflecting upon the

effects of what has been done, and judging its worth.

• Evaluation is a systematic and objective assessment to

determine the elements of project success or failure -
performance, effectiveness and impacts of a project. 2
 What is Evaluation?

• It thus focuses on making judgments on the performance, value,

outputs, success and impacts of a project.
• Evaluation can be done during project implementation, some years or
several years after the completion of the project depending on its
purpose.

• The ultimate objective of evaluation is to determine the;

• relevance and fulfillment of project objectives,
• efficiency,
• effectiveness,
• impact and
• sustainability of the project.
3
What is Evaluation?

 Relevance/Appropriateness : concerns whether the

rationale behind a project is in line with the priorities of
the stakeholders and the society in question.

• Objectives compared with available resources

• Comparison of current needs with the original needs
• Appropriateness of intervention, solutions/technologies
compared with target group needs
4
What is Evaluation?

 Effectiveness: concerns the extent to which the purpose

has been achieved, or can be expected to be achieved.

• Original objectives compared with outcomes (what

was desired and what was achieved)
• Outcomes compared with needs
• Current outcomes compared with past outcomes
• Outcomes compared with standards
5
• Comparison between the target groups within the
What is Evaluation?

 Efficiency: Is a measure of the “productivity” of the

implementation process (efficient use of financial,
human and material resources).
• Current costs compared with past costs (materials,
people, tools, technology, and processes)
• Costs compared with similar interventions
elsewhere (benchmarking)
• Extent of implementation compared with targets
6
What is Evaluation?

 Sustainability: concerns with what happens after the project

has been completed. The sustainability of any project will
depend on whether the positive impact justifies the
investments.

 Impact: the extent of socio-economic, technological,

environmental, and psychological changes attributable to the
project’s interventions. It includes both positive and negative
consequences, whether these are foreseen and expected or not.
7
 Purpose of Evaluation

• To judge the value of project outputs and impacts, and provide a clear
picture of the extent to which the intended objectives of the project
have been realized.

• To identify the constraints or bottlenecks that hinder the project in

achieving its objectives.

• To provide solutions by taking into consideration the benefits and

costs that accrue to the intended beneficiaries of the project.

• To draw lessons from the project implementation experience and use

of the same for future projects.
8
 Question
1. Which of the following is not the purpose of
undertaking project evaluation?
A. To learn from project implementation
B. To identify bottlenecks against project success
C. Identify the value of project output and input
D. A and B
E. A, B and C
 Types of Evaluation

• There are different types of project evaluation, which can be categorized

in a variety of ways.

• The categories and types of project evaluation are not mutually exclusive
and are often used in combination.

• Ultimately, the approach and method used in any project evaluation is

determined by the target audience and purpose of the evaluation.

• It is best to involve key stakeholders in the evaluation process as much as

possible.

5/25/2015 10
 Types of Evaluation

Evaluations can be categorized in a variety of ways.

• According to evaluation timing

• According to who conduct the evaluation

• According to evaluation methodology

11
 Types of Evaluation

• According to evaluation timing

 Ex-ante/Pre-implementation Evaluation – occur at the point of

project planning

 Ongoing/Formative Evaluation – occur during project implementation

to improve project performance and assess compliance

 Midterm Evaluation - are formative in purpose and occur midway

through the project implementation

12
 Types of Evaluation

 Terminal/Summative Evaluation – occur at the end of project

implementation to assess effectiveness and impact.

 Final Evaluation - are summative in purpose and are conducted (often

by external bodies) at the completion of project implementation to
assess how well the project achieved its intended objectives.

 Ex-post Project Evaluation/Final Post Evaluation - are conducted

some time after the completion of the project implementation to
assess long-term impact and sustainability.

13
 Types of Evaluation

• According to who conduct the evaluation

 Internal or Self-evaluation – are conducted by those responsible for

implementing a project.

 External or Independent Evaluation - are conducted by evaluator(s) outside of

the implementing team, lending it a degree of objectivity and often technical
expertise.

 Participatory Evaluation - are conducted with the beneficiaries and other key
stakeholders, which can be empowering - building their capacity, ownership and
support.

 Joint Evaluation - are conducted collaboratively by more than one implementing

partner and can help to build consensus and credibility at different levels. 14
 Types of Evaluation

• According to evaluation methodology

 Real-Time Evaluation – are undertaken during project implementation to provide

immediate feedback to improve ongoing implementation.

 Meta Evaluation - are used to assess the evaluation process itself. For instance, check
compliance with evaluation policy and good practices; assess how well evaluations are
disseminated and utilized for organizational learning and change.

 Thematic Evaluation - focus on one theme, typically across a number of projects, or the
whole organization.

 Impact Evaluation - focus on the effect of a project, rather than on its management and
delivery.

 Cluster/Sector Evaluation - focus on a set of related project activities, typically across15

sites and implemented by multiple organizational units.
 Questions
1. On the basis of who conduct the evaluation
projects are also classified as;
A. Final evaluation
B. Terminal/summative evaluation
C. Thematic evaluation
D. Participatory evaluation
 Basis of Evaluation

 The main basis for evaluation is the objectives and goals of the project.

 Hence, the objectives and goals of the project must be stated in clear
and measurable terms.

 There must be indicators in the project document to measure the

impact of the project.

 A good impact evaluation result can be obtained if there are all the
necessary types of indicators that relate to the project.

 Evaluation is best executed when good tools for gathering information

are prepared and used. 17
 Participants of Evaluation

 like monitoring, it is essential that all stakeholders be

involved in impact evaluation.

 Unlike monitoring, however, the key role in impact evaluation

will have to be played by external evaluators for the purpose
of avoiding bias, conflicts of interest, and for ensuring
objectivity and transparency.

18
 Evaluation Framework

 Steps in designing an evaluation framework include the

following:
• Review project objectives
• Convert project objectives into evaluation objectives
• Identify and list down key factors
• Select the key factors that should be used for developing
the evaluation tools

19
 Evaluation Framework

• Determine indicators for key factors

• Utilize the indicators in formulating key questions
• Check the relevance of the key questions to the
information needs
• Pre-test evaluation tools
• Orient the evaluation team on the tools before putting
the tools to use

20
 Discussion
• Discuss about the types of evaluation
 Planning an Evaluation System

Some of the COMMON steps in planning and conducting an

evaluation system for a project include:

 Determining the Purpose and Type of Evaluation

 Selecting Appropriate Evaluation Methods
 Collecting and Analysing Information
 Reporting Findings 23
 Planning an Evaluation System

 Determining the Purpose and Type of Evaluation

• Determine the purpose/goals of the evaluation.

• Identify evaluation objectives
• Decide on the type of evaluation.
• Decide on who conducts evaluation (evaluation team)
• Review existing information in program documents including
monitoring information.
• List the relevant information sources including location
• Describe the users of the evaluation.
24
• Assess your own strengths and limitations.
 Planning an Evaluation System

 Selecting Appropriate Evaluation Methods

• Decide on the appropriate evaluation design

• Identify measurement standards
• Identify measurement indicators
• Develop data collection instruments.
• Formulate evaluation questions and sub-questions
• Develop an evaluation schedule
25
• Develop a budget for the evaluation.
 Planning an Evaluation System

 Collecting and Analysing Information

• Pre-test data collection instruments.

• Make amendment to the instruments.
• Undertake data collection activities.
• Analyse data.
• Interpret the data.
26
 Planning an Evaluation System

 Reporting Findings

• Write the evaluation report.

• Decide on the method of sharing the evaluation results and on
communication strategies.
• Share the draft report with stakeholders and revise as needed.
• Disseminate evaluation report.

27
 Planning an Evaluation System

 Implementing Evaluation Recommendations

• Develop a new/revised implementation plan in partnership

with stakeholders.
• Monitor the implementation of evaluation recommendations
and report regularly on the implementation progress.
• Plan the next evaluation.
28
 Project Monitoring & Evaluation

 M&E differ and are distinct concepts. Yet they are closely related or
interactive processes.

 Both M&E gather and analyze data & information in order to examine
the progress, effectiveness, outcomes & impacts of projects.

 So, they are processes that gather and analyze information to be used
for the effective management of projects.

 Information generated through M&E provide project staff with a

clearer basis for decision-making
29
 Project Monitoring & Evaluation

 Nevertheless, M&E have their own focuses and strategies,

 Monitoring is a continuous or periodic review of project implementation

focusing on inputs, activities, work schedules, outputs, which is used as
an input for evaluation function.

 Monitoring is more descriptive in nature (quantitative), which provides

the data on which the evaluation of a project is based.

 Generally, through routine tracking of project progress, monitoring can

provide both quantitative and qualitative data useful for the evaluation
function.
30
 Project Monitoring & Evaluation

 Relationship between M&E

Evaluation is a process that
assesses an achievement against
preset criteria. Has a variety of
purposes, and follow distinct
methodologies (process, outcome,
Monitoring is a planned,
performance, etc).
systematic
process of observation
that closely follows a
course of activities, and
compares what is happening
with what is expected
31
to happen
 Project Monitoring & Evaluation

Complementary features of M&E

Monitoring Evaluation
 Implementation oriented  Policy oriented
 Track results  Explain results
 Assess immediate results  Assess attribute
 Focuses on timelines  Focus on rigor/objectivities
 Emphasize on multi-level results  Emphasize final results
 Inform budgeting  Inform broad resource allocation
 Essential for project implementation and  Essential for strategy development
improvement
 Can use disaggregated data  May use aggregated data, etc

32
 Project Monitoring & Evaluation
Distinguishing features of M&E
Features Monitoring Evaluation
 Objectives/basic  Improving efficiency and adjusting work  Improve effectiveness, impact, future
purpose plan programming.
 Focus  Determine project inputs, activities,  Determine effectiveness, relevance,
outputs, process outcomes, work plans effects and impacts, sustainability
 Main action  Keeping track/oversight  Assessment

 Information sources  Routine systems, field visits, Same plus (+)

stakeholder meetings, output reports,  Surveys (pre-post project)
rapid assessments  Special studies
 Undertaken  Project/program managers  External evaluators
by/Implementers  Supervisors  Beneficiaries
 Beneficiaries  Project/program managers
 Financiers  Supervisors
 Other Stakeholders  Financiers

33
 Project Monitoring & Evaluation

Distinguishing features of M&E

Features Monitoring Evaluation

 Data to gather  Primarily quantitative data  primarily qualitative data
 Tools for data  Generally short monitoring formats  Generally long questionnaires or
collection interview
 Time focus and  Present and details  Past-Future and on the big picture
attention level
 Frequency of data  More frequent, regular and ongoing  Less frequent and periodic/episodic
collection
 Use of data  Mainly for decision making during  Primarily for planning projects
project implementation
 Questions asked  What is working/not working?  What results occurred?
 What are the deviations?  Who benefits?
 What needs to be improved?  With what resources?
 How can it be improved?

34
 Project Monitoring & Evaluation

 In conclusion, the main difference between M&E is their

timing and focus of assessment.

• Monitoring is ongoing and tends to focus on what is

happening.

• Monitoring data is typically used for ongoing project

implementation, tracking outputs, budgets, compliance
with procedures

35
 Project Monitoring & Evaluation

• On the other hand, evaluations are less frequent and

conducted at specific point in time to assess larger changes or
outcomes, such as the impact and relevance of the project.

• Recognizing their differences, it is also important to

remember that both monitoring and evaluation are very
much inter-linked

 Monitoring typically provides data/information for evaluation

to assess any difference made towards the overall objective
or change in a project.

36
 Question
1. Which of the following is not true about
monitoring and evaluation?
A. Gather, and analyze data and information
B. Both are identical concepts and used
interchangeably
C. Use the information for decision making
D. Use information for effective management
 Implications of M&E for Project Success

 Importance of M&E

In summary, timely and reliable (accurate and evidence based) M&E is

important for various reasons:

 Support project implementation – guiding and supporting the decision making process,
which can improve the overall performance and quality of ongoing and future projects.

 Contribute to organizational learning and knowledge sharing – sharing experiences and

lessons learned, which can improve organizational learning and capacity development.

 Uphold accountability and compliance at all levels – performing project activities as

agreed upon and in compliance with established standards.
38
 Implications of M&E for Project Success

 Importance of M&E

 Provide opportunities for stakeholders feedback - introducing openness &

building greater transparency in what the project does to the beneficiaries.

 Improve the relevance, performance, and success of a project by enforcing

responsibility and accountability in the use of project resources.

 Promote the culture of project work accomplishments and achievements,

which helps in building staff morale and fostering organizational improvement
and change.
39
 Implications of M&E for Project Success

 Factors Affecting the Quality of M&E

Factors that affect the quality of M&E among others include:

 Spatial – tendency to determine project sites that provide comfort

and ease.
 Project – Bias toward project sites where contacts and information is
readily available and may have been assessed before by many others.
 Season - conducting assessments during periods of pleasant weather,
or cutting off or neglecting areas due to bad weather conditions.
40
 Implications of M&E for Project Success

 Factors Affecting the Quality of M&E

 Diplomatic - selecting projects as a selling point or for

diplomatic reasons.
 Professional - missing linkages between processes due to
narrow focus or specialization of the Assessor
 Person – the tendency to select key informants those who
are in a high position and have the ability to communicate.
Partiality to the opinions of elite key informants.
41
 Implications of M&E for Project Success

 Factors Affecting the Quality of M&E

 Conflict - Assessors go only to relatively safe areas for security reasons.

 Political - Informants present information that is skewed toward their
political agenda; assessors look for information that fits their political
agenda.
 Cultural - Incorrect assumptions are based on one’s own cultural norms;
assessors do not understand the cultural practices of the affected
populations.
 Class/ethnic - Needs and resources of different groups are not included in
the assessment.

42
 Implications of M&E for Project Success

 Factors Affecting the Quality of M&E

 Interviewer or investigator - Tendency to concentrate on information

that confirms preconceived notions and hypotheses, causing one to seek
consistency too early and overlook evidence inconsistent with earlier
findings.
 Key informant - Biases of key informants carried into assessment results.
 Gender - Male monitors may only speak to men; women may be
omitted.

43
 Implications of M&E for Project Success

 Factors Affecting the Quality of M&E

 Time of day or schedule bias - The assessment is conducted at a

time of day when certain segments of the population may be
over- or under-represented.
 Sampling - Respondents are not representative of the population,
and so forth.

44
 Questions
1. One dose not affect the quality of monitoring
and evaluation
A. Time
B. Culture
C. Key informants
D. Professionals
E. None
Key Steps in Project Monitoring and Evaluation (M&E)

 M&E Process Steps

• Your ability to monitor and evaluate progress on your project,

calculate your variance from plan, and predict the future depends
upon a number of key process elements (IFRC, 2011).
 Identify the purpose and scope of M&E system
 Establish a baseline of measurement
 Plan for methods of data collection & management
 Plan for data collection & analysis (processes, tools, and methods)
 Plan for Information reporting and utilization
 Plan for M&E human resources and capacity building
 Prepare the M&E budget

46
Key Steps in Project Monitoring and Evaluation (M&E)

 M&E Process Steps

D: Plan for data
A: Identify the B: Establish a baseline C: Plan for methods collection and
purpose and scope of of measurement of data collection and analysis
M&E system management

E: Plan for G: Prepare the M&E

F: Plan for HR and budget
information reporting
capacity building
and utilization

47
 Discussion
Discuss the difference between monitoring and
evaluation