CHAPTER FIVE: PROJECT IMPLEMENTATION,

TERMINATION AND EVALUATION

1.1. Implementation Phase

1.1.1. Process of Implementation
1.1.2. Project Control
1.1.3. Change and Control Management
1.2. Project Termination
1.2.1. Closeout Task: Activities
1.2.2. Turnover to Operation
1.2.3. Administrative Closure
1.2.4. The Final Report
1.3. Project Evaluation and Types of Evaluation
1.3.1. What is Project Evaluation?
1.3.2. Relationship of Monitoring to Evaluation
1.3.3. The Purpose and Use of Evaluation
1.3.4. Different Kinds of Evaluation
1.3.5. Evaluation Criteria
1.3.6. The Steps in Doing an Evaluation
 Summary
 Check in Progress 5

Objectives
After completing this chapter you should be able to:
 Understand the reasons for implementing controls over a project;
 Determine the appropriate corrective actions to restore a project to its
planned schedule;
 Understand the steps needed to effectively close a project;
 Identify a component of project documentation;
 Outline the components of the final project report.
 Measure and analyze variance from the project plan;
  Distinguish between monitoring and control; and
 Describe the purpose of project evaluation;
 Conduct an on going or a terminal project evaluation studies; and
 Distinguish between evaluation and monitoring.
 Identify the steps involved in evaluation as well as some relevant tools
and techniques;
Introduction

The project implementation phase, which is the heart of project success, is going to be
seen in detail as follow. This is where the project manager organizes the project’s
activities and resources with in specified time frame and prepares the project for
successful implementation. Thorough planning allows the community and the Volunteer
to feel that they have given their project an excellent chance of being successful. The next
important step in the process is monitoring the progress of the project to identify
unanticipated problems and plan effective action steps to solve them. During this session,
the participants will examine the tasks involved in monitoring projects, and identify items
to monitor.

Project termination (sometimes called project closeout) is the last phase in the project
lifecycle. Closeout begins when the user accepts the project deliverables and the project
oversight authority concludes that the project has met the goals established. The report
provides a historical summary of the projects deliverables and baseline activities over the
course of the project. Additionally, the project closeout report identifies variances from
the baseline plan, lessons learned, and disposition of project resources.

5.1 Implementation Phase

The project implementation phase is the part of the project lifecycle where the tasks that
build the deliverables are executed. The project implementation phase begins when the
project plan is approved and the resources necessary for executing the starting task are
assembled. Project execution should be in accordance with the approved project plan.
Project implementation consists of the following list of processes:

- Execution,
- Measuring project progress,
- Reporting project status, and
- Exercising management controls.
- User Acceptance

The project team executes the tasks as mapped out in the project plan. Controlling project
execution requires the measurement of project performance, monitoring project risk, and
controlling change to the project baseline. The manager is focused on observing and
analyzing the work underway. Controls outlined in the project plan keep the project on
schedule, in scope, and within budget. During this phase, the processes of executing,
controlling, and planning are continuous interactive activities.

Figure 5.1 depicts these activities. This phase ends when the product, good, or service
developed has met the user acceptance criteria established in the performance.

Figure 5.1: Project Implementation Phase Processes

1.a.1 Process of Implementation

 I. Executing the Project

Execution is the act of carrying out planned activities. The execution of the project plan is
simply the act of performing task and activities that result in the production of the project
deliverables. Task and activities performed must be completed effectively and efficiently.

The project plan serves as a road map and a common frame of reference for all members
of the project team. The project plan is therefore, the foundation for successful delivery
of projects. In a perfect world, plans are executed precisely as written. In reality, no plan
is ever performed with such precision. Plans are forward looking documents that cannot
anticipate all eventualities.

During execution, the project team must continuously monitor its performance in relation
to the baseline project plan. By measuring and evaluating the actual execution of project
activities against the baseline plan, the project team and stakeholders can gauge the
progress of the project.

Moving from planning into execution can be a major obstacle in successful project
delivery. A project kick-off meeting can facilitate the transition from planning activities
and tasks to executing them. A kick-off meeting enhances execution by focusing the team
on the project and by defining a starting point for beginning project execution.
Additionally, it is a mile-stone when all resources needed to begin execution are
assembled and available to the team.

The kick-off meeting provides an opportunity for communication and establishing the
commitment of the team and stakeholders to the success of the project. The focus of the
meeting is communications, identification of team members and stakeholders, reviewing
the project scope and business objectives, identifying the challenges, and identifying the
next step in getting the project underway.

Generally, a project kick-off meeting will enable the Project Manager to:

 Confirm key resources are ready to start to deliver project tasks;

  Ensure ground rules are established for the team;
 Review project statements and objectives with those who will work on the
project and test their understanding and agreement on the goals of the
project; and
 Review the project plan to clarify initial assignments, performance
expectations and completion dates.

II. Measuring the Project

Performance Measuring can provide assurance that the project is progressing as planned
or reveal the need to intervene and take action to ensure the achievement of the desired
business objectives.
The execution of project task and activities occur in a cycle where:

 The task is executed;

 Execution is measured;
 The results are reported; and
 Management controls needed are applied.

Performance measuring involves the collecting, analyzing, and reporting project

performance information to provide the project team and stakeholders with information
on the status of project execution.
 Figure 5.2: Project Monitoring Cycle

Various metrics can be gathered to monitor project progress. Common areas to monitor
typically include:
 Project schedule;
 Work effort;
 Costs;
 Issues resolution; and
 Changes to the project.
Other metrics may be requested and defined by project or organizational management. Some
common metrics, which may be utilized during project execution, are provided below.

 Project Schedule Deviation

Project schedule may include all tasks and estimated work hours for the entire project or
it may represent a “phased” or “iterative” approach to the project. In the latter case, the
schedule will reflect the tasks and efforts of only a portion of the entire project. Reporting
for the purposes of management should be against the detailed portion of the project
schedule. Monitoring the critical path is essential.

All schedule changes must therefore be analyzed for impact to the project’s critical path
since such changes will result in deviation from the project schedule. Monitoring of the
planned versus actual starts and completions provides a gap analysis and leads to
identification of overall trends. Status of tasks is reported as:

Not Started - 0 %
Started/In Process - 1-99 %
Completed - 100%

Metrics To Capture – For the reporting period and for plan to date:
 Number of “Planned Starts”
 Number of “Planned Completed”
 Number of “Actual Starts”
 Number of “Actual Completed”
 Number of tasks reporting > 84% completed
 Total number of project tasks
 Number and percent of mile-stones tasks outstanding

Total number of project milestones – Completed = Total Outstanding

Total Outstanding x 100% = % Outstanding

Total number of tasks

 Work Effort

Monitoring work effort is essential for evaluating whether the project is executing within
budget or not. This information is used to project the cost of labor for the project. The
number of hours, preferably at the task level, needs to be captured and tracked.

Metrics To Capture – For the reporting period and for planned to date:

 Number of “Planned Hours”

 Number of “Actual Hours”
Metric Calculations – For the reporting period and for planned to date:

 Deviation from plan

(Planned – Actual = Deviation from Planned Hours)
 Percent Deviation from plan
(Deviation from Plan/Planned Hours x 100 = % of Effort Deviation)

 Cost

The budget plan developed during planning represents the basis for measurement of
deviation during execution. Measuring cost requires the support of the financial and
procurement support business units.

Metrics to Capture – Costs for the reporting period and for planned to date for all labour,
materials and other cost items:

Metric Calculations – For the reporting period and for planned to date:

 Difference between actual expenditures and planned budget for each metric;
 Increase or decrease to total project budget cost;
 Percentage deviation from spending plan for the period measured; and
 (Spending Plan – Actual Cost)/Actual Cost x 100 = % Deviation from
Spending Plan.

 Project Issues

One indicator of project health is the number of open issues and their impact on the
project. Proactive issue management aims to track and analyze all issues, specifically
focusing on those that have remained unresolved.
Metrics To Capture – For the reporting period and for planned to date:

 Number of new issues;

 Number of closed issues; and
 Number of outstanding issues.

Metric Calculations – For the reporting period and for planned to date:

 Total Number of issues =Number of outstanding issues + Number of

closed issues.
 Percentage of issues closed = Issues Closed .
Total Issues

Project Change Requests - There will always be changes to a project. The challenge is
to identify and manage them.

The Change and Configuration Management Plan provide a process and guidance for
managing change during project execution. A change management log and change
request documents are used as tools to monitor, track and approve request to change
items under change control or configuration management.

III.Project Reporting

A standard requirement of all projects is to provide information to both executive

management and the project team members on the status of the project. Although the
frequency of the reports may sometimes vary, the frequency should correspond with
information requirements identified in the project Communications Plan. Often status
reports are prepared for executive or team meetings.
The project status report is a means of communicating regularly the ongoing progress and
status of a project. The overall project status is communicated to all team members using
the project status report.
The information shared in the Status Report should be in a consistent format throughout
the project. The types of reports a particular project uses may vary in detail and metrics
required but the basic format remains consistent across all projects.

IV. Modify Project (Apply Management Control)

No matter how well defined projects are, situations will arise that require changes to be
made to the project plans. They may be imposed by senior management, by changes in
the business environment, or the changing preferences of a client.

Any change in the project’s objectives may require a fine-tuning of the project’s work
breakdown structure, the resource requirements, responsibility assignment matrix, or
project plan. Many of these changes are caught in the planning phase, where the team
seeks out problems and opportunities. Modifying the project can range in complexity
from shifts in the bars of a Gantt chart through reformulation of the WBS, responsibility
assignment matrix and project plan. In each modification, project managers must balance
the time expenditure against the need to revise whatever thinking underlies the project.
The project manager’s decision making skills are critical when modifying the project.

Change Control is a critical element of the Implementation Phase. Despite efforts to

ensure the plan is implemented with minimum difficulties or deviations from scheduled.

Work/resources during the Definition Phase and the Planning Phase, changes will
inevitably occur. The Project Manager has to deal with these and still ensure that the
objectives of the project are met to achieve success. Changes can be internal (to the
organization) or external (in the business and economic environment) and may range in
complexity from a change to a schedule to a change in technology, equipment or
resources. These more extensive changes have to be analyzed carefully, and rational
decisions made based on available information. Changes to scope must be progressed
through a formal Project Change Control.

V. User Acceptance
Acceptance criteria for project deliverables establishes in advance an agreed upon
standard of performance or capability that the user will accept in a specific deliverable.
The Performance Plan developed in the Project Planning Phase articulates the project
deliverables and acceptance criteria. Acceptance criteria then become the fundamental
guideline for the design team to build a solution that the user will find acceptable.

The execution phase ends when the user has agreed to accept the deliverable(s) in the
state that they exist. The acceptance criterion is the standard that the user uses to judge if
each deliverable is satisfactory. In some cases, the deliverable may not meet all
acceptance criteria but, from an overall view, the deliverable will meet the requirements
of the user. The user must authenticate acceptance of each deliverable. The user will also
identify any issues that remain outstanding and the agreed to plan for resolution of any
outstanding issues.

1.a.2 Project Control

Control of the project is exercised through formal and informal processes exercised by
the project manager, project team, and stakeholders. The process of conducting reviews
and monitoring reports exert a degree of control over the project. This discussion will,
however, focus on the formal processes of control established by the project plan.
 Project Plan

The main part of project control documentation is the project plan. The project plan fixes
the project schedule, tasks, and resources. The plan also establishes the procedures to
manage quality, risk, communications, and change. Use of the plan to guide execution of
project task exerts a great degree of control in the process of developing the deliverables.
As mentioned previously, change is inevitable. Issues and unforeseen risk events must be
addressed and resulting changes must be managed. Control over unplanned events is
exerted through issue management and formal change management processes.

 Issues Management Process

The purpose of the issue management process is to provide a mechanism for organizing,
maintaining, and tracking the resolution of issues that cannot be resolved at the individual
level. The approach consists of issue control mechanisms and a defined process that
enables the project team to identify, address, and prioritize problems and issues.

The Issue Log and Issue Management Document are used to track, document and resolve
issues that are identified during project execution. Initially, issues may be identified in a
project status report or in an Issue Management Document.
 The Issue Log is a master record of issues that are identified and is used to
track progress toward resolution.

The Issue Management Document is a means of reporting issues and is

used by the project team to document the issue, assess the impact of the
issue, make recommendations, and identify the resources needed to resolve
the issues. Managers use the Issue Management Document to assign
responsibility and to document decisions on actions directed to resolve the
issue.

Any project team member, customer, stakeholder, or contractor can submit an issue. This
must be done in writing, either in a Project Status Report or in an Issue Management
Document. If the issue is identified in a Project Status Report, the person making the
report prepares an Issue Management Document.

 Issue management is divided into three phases.

 The first phase occurs when a person identifying an issue reports the issue and
completes the first section of the Issue Management Document. The Issue
Management Document provides a format to identify the issue document, the
date the issue is submitted, and identify the person reporting the issue.

It also requires the submitter to provide:

- Date to which resolution is required;

- A proposed assignee to resolve the issue;
- An issue description; and
- An impact statement.
  The identifier of the issue should attach any supporting documentation that
will clarify the issue, such as reports, correspondence, test results, error messages,
or other pertinent data. When completed the Issue Management Document is
submitted to the project manager. The project manager must then assign someone
to investigate the issue and develop a recommendation for resolving the issue.
The issue resolution task is an action item reported in the project status reports.
The project manager completes:

 The second phase of issue management is determining a resolution for the issue.
Identifying alternatives for resolving the issue and making recommendations on
what actions should be taken. The person assigned to investigate alternatives and
make a recommendation.

Completes the following information:

- Discussion of alternatives;
- Recommendation; and
- Estimate of additional resources.

 The third and final phase is the management decision.

Management reviews the recommendation provided and decides to accept the
recommendation made, modify the recommendation, reject the recommendations,
defer a decision, or return the document for additional information.

Management decisions can lead to change management actions, which are then handled
through the change and control management process.

1.a.3 Change and Control Management

Any change to the configuration of a deliverable or to the baseline elements of the project
plan must be managed with deliberate precision. The Change and control Management
Plan establishes the processes used to manage and control change. Additionally, the
specific items that will be controlled through the change and control management process
are listed in the Change and Control Management Plan.

Activities involved in change and configuration management include controlling changes

to the scope and the schedule.

 Scope Change Control - Uncontrolled expansion of the project scope often results in
project failure. A scope change usually requires additional project funds, resources,
and time. The project manager and team must monitor changes to the scope baseline
and recognize when a formal scope change should be made.
 Schedule Control - Schedule issues come from a variety of sources. Variation from
the project schedule must be investigated and the cause determined as soon as
possible.

When the reason for a schedule problem is discovered, a plan must be developed to
correct the problem, as quickly as possible, with the least impact to the project.
 Cost Control - Projects fail to control cost, or go over budget, for many reasons.
Failure to control cost is often a result of

- Incremental changes,
- Unplanned risk mitigation, or
- Inaccurate budget planning.

The Project Management Body of Knowledge describes cost control as being concerned
with the following:
- Influencing the factors that create changes to the Project Budget Estimate to
ensure that the changes are beneficial
- Determining that the project budget estimates have changed
- Managing the actual changes when and as they occur

Cost control includes the following:

- Monitoring expenditures to detect variances from the project spend plan
 - Executing the change control plan to prevent incorrect, inappropriate, or
unauthorized changes from being made to the Project Budget
- Recording authorized changes accurately in the Project Budget Plan

 Quality Control - Quality control involves monitoring project deliverables and

performance goals to ensure that the project delivers the required results established
in the project performance plan. Quality control is performed throughout project
planning, execution, and closure.

Project performance measures include both product results such as deliverables and
management results such as cost and schedule performance. This will be discussed in the
next section.

5.2 Project Termination

As it is a must to all things, termination comes to every project. In this Section, we will
discuss the major activities performed in terminating a project. Project termination
(sometimes called project closeout) is the last phase in the project lifecycle. Closeout
begins when the user accepts the project deliverables and the project oversight authority
concludes that the project has met the goals established.

5.2.1 Closeout Task: Activities

Closeout begins when the user accepts the project deliverables and the project oversight
authority concludes that the project has meet the goals established. The major focus of
project closeout is administrative closure and logistics as well as undertaking post
implementation evaluation/review.

Project closeout includes the following key elements:

 Turn over of project deliverables to operations;

  Redistributing resources—staff, facilities, equipment, and automated
systems;
 Closing out financial accounts;
 Completing, collecting, and archiving project records;
 Documenting the successes of the project;
 Documenting lessons learned; and
 Conduct Post Implementation Review.

Next, we will see the major activities performed in closing out a project. For your
convenience all activities listed above are grouped into three sections. Even though post
project evaluation is one of the activities performed, because of its importance, I put it in
a separate section (see Section 1 in the coming Unit).

5.2.2 Turnover to Operation

The most important aspect of project closeout is the physical turnover of control of the
product, good, or service delivered by the project. All project deliverables will need to be
maintained and supported after the project team disbands. An operational unit of the
organization (for which the deliverable is developed) assumes responsibility for the
support of the deliverable. Procedures for this turnover and acceptance by the operational
unit must be determined. Turnover and acceptance activities include but are not limited to
knowledge transfer, documentation transfer, and physical transfer of the deliverable. A
formal acknowledgement of receipt (acceptance) of the project deliverable is executed by
the operations and project managers.

5.2.3 Administrative Closure

Administrative closure involves the preparation of administrative documentation,

collection of project documentation, disposition of project documents, and logistics
activities that ensure that the project resources are redistributed. Administrative closure
includes, but is not limited to, task such as:
 Archiving project information,
  Personnel reassignment;
 Facilities turnover (or closure),
 Financial account closure, and
 Contract closure.

The purpose of Administrative Closeout is to perform all administrative tasks required to

bring the project to an official close.
a) Archiving project information
Throughout the course of the project, the Project Manager maintains a project repository.
As the project progressed, the purpose of the repository was to create a central point of
reference for all project materials to be used by anyone involved in the project. Once the
project comes to an official close, the repository provides an audit trail documenting the
history and evolution of the project.
During Project Closeout, the Project Manager should examine the repository to ensure
that all relevant project-related material, documents produced, decisions made, issues
raised and correspondence exchanged have been captured. In addition, the Post-
Implementation Report and the final project closeout report should be included.

Archived Project Repository is a collection of all project related materials, documents

produced, decisions made, issues raised and correspondence exchanged, providing the
history and evolution of the project.
There are at least five reasons why we need to do documentation or maintain repository:

1) Reference for future changes in deliverables. Even though the project

work is complete, there will be further changes that warrant follow up projects.
By using the deliverables, the customer will identify improvement opportunities,
further to be added, and functions to be modified. The documentation of the
project just completed is foundation for the follow-up projects.
2) Historical record for estimating duration and cost in the future projects,
activities and tasks. Completed projects are a terrific source of information for
future projects, but only if data and other documentation from them is archived so
 that it can be retrieved and used. Estimated and actual duration and cost for each
activity on completed projects are particularly valuable for estimating these
variables of future project.
3) Training resource for new project managers. History is a great teacher, and
nowhere is that more significant than no completed projects. Such items as how
the work breakdown structure architecture was determined, how change requests
were analyzed and reached on decisions, problem identification, analysis and
resolution situations, and a variety of other experiences are in valuable lessons for
a newly appointed manager.
4) Input for further training and development of the project team. As a
reference, project documentation can help the project team deal with situations
that arise in the current project. How a similar problem or change request was
handled in the past is an excellent example.
5) Input for performance evaluation by the functional managers of the project
team members. In many organizations, project documentation can be used as an
input to the performance evaluations of the project managers and team members.
Care must be exercised in the use of such information, however. There will be
cases where a project was doomed to fail even though the team members’
performance may have been exemplary. The reverse is also likely. The project
was destined to be a success even though the team members’ performance may
have been less than expected.

b) Personnel Reassignment
If personnel have been committed to the project full-time, it is important to get the people
back into the available resource pool as quickly as possible. This will ensure that the staff
stays busy and that other projects within the organization do not fall short of resources. In
some cases, employee performance reports or other documentation must be prepared for
personnel assigned to the project manager. In matrix organizations, the project manager
should communicate to the functional manager information about the performance of the
employee. The project manager should also make recommendations for recognition of
performance as the case may warrant. Before any employee is officially transferred, the
project manager or his representative must ensure that all project materials and properties
are turned over by the employee. The project manager must also ensure that each
employee’s project hours have been accounted for and charged to the project.

c) Facilities Turnover (or closure)

If the project team has occupied agency facilities for a long period of time during the
project, it is a good idea to let the controlling facilities personnel know that the space
used for the project will become available again. Be sure to check facilities guidance
documentation to determine whether changes made to the project team area (structure,
equipment, or technical modifications) are the responsibility of the project team after the
project is complete. Returning the facility and equipment to its original state could add
unanticipated cost and manpower to a project.

d) Financial Account Closure

Financial closure is the process of completing and terminating the financial and
budgetary aspects of the project. Financial closure includes both (external) contract
closure and (internal) project account closure. All expenditures must be accounted for
and reconciled with the project account. When financial closure is completed, all
expenditures made during the project have been paid as agreed to in purchase orders,
contracts, or inter-agency agreements.

e) Contract Closure
Contract closure is the process of terminating contracts with external organizations or
businesses. These contracts may be vehicles for providing technical support, consulting,
or any number of services supplied during the project that the agency decided not to
perform with internal resources. Contracts can be brought to closure for a variety of
reasons, including contract completion, early termination, or failure to perform.
Contract closure is a typical but important part of project management. It is a simple
process, but close attention should be paid so that no room is left for liability of the
agency.
In order to close a contract it is important to collect all of the pertinent documentation
for review. This will include all of the original contracts and supporting documentation
such as schedules, contract changes, and performance reports. This documentation
needs to be reviewed thoroughly to ensure there are no unrealized contract issues that
could result in legal liability. A thorough review of the procurement and contracting
documents must include contract milestones, services provided or deliverables and
documentation delivered.

To formally close a contract, the agency provides the contracted company or organization
with a formal written notice stating the completion of the contract and reason for
termination. Standard verbiage for acceptance and closure is usually found in the original
contract itself.

It is also a good idea to keep a complete set of contractual records for the project in
a safe and accessible place in case they need to be referenced at any point in the
future.

5.2.4 The Final Report

The final report (sometimes called a project closeout report) documents the completion of
closeout tasks and project performance. The report provides a historical summary of the
projects deliverables and baseline activities over the course of the project. Additionally,
the project closeout report identifies variances from the baseline plan, lessons learned,
and disposition of project resources. The project closeout report is intended to provide a
concise evaluation of the project. The final project report acts as the memory or history of
the Project. It is the file that others can check to study the progress and impediments of
the project.

Many formats can be used for a final report, but the content should include comments
relative to the following points.

1. Overall success of the project. Taking into account all the measures of success
that we considered, can we consider this project to have been success?
 2. Organization of the project. Hindsight is always perfect, but now that we are
finished with the project did we organize it in the best way possible? If not, what
might that organization have looked like?
3. Technique used to get the result. By way of summary list, what specific things
did you do that helped to get the results?
4. Project strengths and weaknesses. Again by way of a summary list, what
features, practices, and processes did we use that provided to be strengths and /
or weaknesses? Do you have any advice to pass on to future project teams
regarding these strengths/ weaknesses?
5. Project team recommendations. Through out the life of the project there will
have been a number of insights and suggestions. This is the place to record them
for posterity.

Project Closeout may be perceived as the least important of all of the project phases, but
its value to future projects cannot be underestimated. The knowledge gathered, the
expertise developed, the lessons learned, the practices perfected will remain locked
temporarily in a few people's heads unless the Post-Implementation Review is conducted
promptly, documented thoroughly, and (most importantly) its results are disseminated
appropriately throughout the Performing Organization

You have now completed all five phases of the project management life cycle. We can
only hope that the practical tools and techniques we have shared will provide a lasting
and a valuable store of resources for you to use as you grow in this exciting profession.

1.3 Project Evaluation and Types of Evaluation

This Part is intended to help you to understand the underlying principles of evaluation, in
order to be clearer about its uses and limitations. The purpose of this chapter is to provide
grounding in evaluation and to discuss the kinds of information evaluation can provide.
We start with the assumption that the term “evaluation” describes different models or
data collection strategies to gather information at different stages in the life of a project.
 1.3.1 What is Project Evaluation?

The notion of evaluation has been around a long time-in fact, the Chinese had a large
functional evaluation system in place for their civil servants as long ago as 2000 B.C. Not
only does the idea of evaluation have a long history, but it also has varied definitions.

Evaluation means different things to different people and takes place in different
contexts. Thus, evaluation can be synonymous with tests, descriptions, documentation, or
management. Many definitions have been developed, but a comprehensive definition for
evaluation is presented as follows:

This definition centers on the goal of using evaluation for a purpose. Evaluations should
be conducted for action-related reasons, and the information provided should facilitate
deciding a course of action.

Evaluation is a learning and management tool: an assessment of what has taken place in
order to improve future work. Measuring, analyzing and interpreting change helps people
to determine how far objectives have been achieved and whether the initial assumptions
about what would happen were right; and to make judgments about the effectiveness,
efficiency, impact and sustainability of the work.

Evaluation is judging, appraising, or determining the worth, value, or quality of a project,

whether it is proposed, on going, or completed. This is done in terms of its relevance,
effectiveness, efficiency, and impact. (Relevance refers to the appropriateness and
importance of goals and objectives in relation to assessed needs. Effectiveness refers to
the degree to which goals have been achieved. Efficiency refers to the cost-effectiveness
of activities. And impact refers to the broad, long-term effects of project.)

1.3.2 Relationship of Monitoring to Evaluation

Monitoring is different from evaluation. Monitoring measures whether the project is on

track; evaluation questions whether it is on the right track. Monitoring is concerned
mostly with project activities, and concentrates on the short-term performance compared
with the project plans. Evaluation looks more at the overall project purpose/objectives,
and examines longer-term effects of the project. Monitoring is a continuous process,
while evaluation is a periodic event.

A distinction is usually made between monitoring and evaluation.

 Monitoring is a continuous, methodical process of data collection and

information gathering throughout the life of a project. The information collected
can be used for regular evaluation of progress, so that adjustments can be made
while the work is going on. Monitoring is also used to mean the systematic
'tracking' of a particular condition, or set of conditions to identify trends.

 Evaluation is a learning and management tool: an assessment of what has taken

place in order to improve future work. Measuring, analyzing and interpreting
change helps people to determine how far objectives have been achieved and
whether the initial assumptions about what would happen were right; and to make
judgments about the effectiveness, efficiency, impact and sustainability of the
work.

 In monitoring, information for 'tracking' progress against previously agreed plans

and 'milestones' is routinely gathered. The changes that are resulting from project
activities can be identified: both the effects and the impact. If there are
discrepancies between actual and planned progress, corrective action can be taken.
This can include changing the overall purpose and plan of the activity. Monitoring
can also mean keeping a check on the use of resources. Questions for later
evaluation can be identified during monitoring.

 Evaluation uses information gathered during regular monitoring, but may need
other information as well. It often uses 'baseline information': information
collected at the very beginning of a project, against which progress can be
measured. Evaluation happens at set times in the life of a project. Evaluation
 looks at the relevance, effectiveness and impact of a project, with the aim of
improving an existing project or influencing future projects.

The relationship of monitoring to evaluation can be illustrated in the chart below.

Evaluation
Monitoring Information Information
from from
Recording (Data) Monitoring Other source

Analysis (Information) Analysis

Storage Recommendation
Reporting

Corrective Affirmation or Modification

Action at the in Objectives, Resources,
Operational Level and Processes

Figure 5.3: Relationship of Monitoring to Evaluation

Using the above chart, it is easy to see the relationship of monitoring to evaluation.

 Monitoring is the recording, analysis, and reporting which transforms data into
information. This must then be communicated in a timely way to managers. The
information must also be stored and retrievable for evaluation later on.
 Evaluation analyzes the information from monitoring and other sources and
formulates conclusions and recommendations. These recommendations can be used at
different levels of management to affirm or modify objectives (relevance), resources
(performance), and processes (quality).
 Monitoring is done during implementation and is somewhat continuous. Evaluations
occur periodically, before or after projects are completed. Evaluation of projects is
done at the end of the project cycle, as terminal evaluation is done just before the end
of the project or as ex-post evaluation when the project has been terminated some
time ago.

1.3.3 The Purpose and Use of Evaluation

There are many different reasons why evaluations are carried out. Some good reasons
are:
- To measure progress and effectiveness;
- To look at costs and efficient use of resources;
- To find out if it is necessary to change the way things are being done; and
- To learn from what has happened in order to make plans for the future.

These are all constructive reasons, within the control of those organizing their work.
There are other, less good, reasons why evaluations are carried out. An evaluation might
be demanded by a funding agency, who are wondering whether to go on supporting a
project; or there might be a statutory requirement for evaluation from a government
department. Evaluations are sometimes done as part of a research project as a way of
testing out new techniques for gathering information; or sometimes because people who
raise funds for an organization need something to put in their publicity material

From the above group of reasons for doing evaluation, four main uses can be identified:

i. To improve performance
This is 'formative' evaluation, helping to 'form' or shape work while it is still going on.
This kind of evaluation is useful for those directly involved in, or in charge of work. It
can identify problems, and things that are working well and can be built on. It can also
check whether or not objectives have evolved, and be a way of keeping the different
people involved in the project informed about progress or the need for change.
ii. To make choices and decisions
This is 'summative' evaluation, a summing-up of a project to make a judgment of how
effective it has been in achieving results. Information from such evaluations can be used
to compare different ways of doing things, to help people make choices between types of
development action. This kind of evaluation can be used by funders to decide on whether
to continue support or not.

iii. To learn lessons

The prime purpose of an evaluation may be learning, so that the results can be shared
within a project, between projects, between organizations and so on.

iv. To increase accountability

Evaluation can improve accountability at the management level by reporting on the
management of inputs, results, etc, and by reporting to stakeholders in the form of impact
assessment. Evaluations may be a condition of receiving funds, as donors want to be sure
that their funds are being used effectively. They may also want to find out if there are
alternative ways of doing the work. Evaluations to promote accountability of donors and
implementing agencies to the women and men they are supporting are rare. Improving
the way evaluations are set up can encourage greater two-way accountability.

1.3.4 Different Kinds of Evaluation

There are five different types of project evaluation. These are:

i. Appraisal or ex-ante evaluation

Information collected before a project starts, or in the very early stages, helps to define
what is to be done, and provides a baseline from which to measure change.
ii. On-going evaluation

Monitoring indicates whether activities are being carried out as planned and what
changes are happening as a result. Monitoring should be accompanied by on-going
evaluation, which analyses the information in order to improve performance during a
project. If a project has been conducting regular on-going evaluation, and this is well
documented, more formal evaluations may be unnecessary.
iii. Mid-term evaluation

Midterm evaluation is carried out while the project is under way; in order to check
weather the project is functioning properly. The mid-term review will verify the
relevance, sustainability and efficiency of the project and will recommend relevant
adjustments.
iv. Termination evaluation (Post Implementation Evaluation)

Is conducted at the formal end of the project with emphasis on documenting what the
project has achieved in comparison with the stated objectives and targets.

This will happen at the end of a project, in order to learn lessons about how the project
has been implemented and the results. A final report of a project written by a project
manager can be evaluative, comparing objectives with what was achieved. The aim of the
end-of-project evaluation is to assess what lasting impact the project is likely to have.
The focus of the end-of-project evaluation is on the effectiveness of the project and the
impact of the project. Furthermore, the end-of-project evaluation will assess the
sustainability of the project.

v. Ex-post evaluations

These happen some time after a project has finished. They look at impact and
sustainability. They also consider broader 'policy' issues. This kind of evaluation is rare.

1.3.5 Evaluation Criteria

Three sets of questions should be built into any evaluation:

 What changes have taken place, and are these changes the ones, which the
original plan hoped for? Evaluations should give an account of what the
project has achieved, or not achieved, and compare this with expectations.
  What were the reasons for the success or failure? It is important to know why
things happened as they did, and analyze the factors, which influenced the
way the project progressed.
 What actions should now be taken? Evaluators should suggest courses of
action, in the light of answers to the first two questions.

Therefore, evaluation should be based on predefined criteria in order to obtain a precise

analysis. The criteria that are commonly used as a focus for shaping evaluation questions
are:

I. Effectiveness: how far is the project achieving objectives?

For example, the project might be concerned with the training of primary health care
workers to improve their technical skills, or disseminating information about sustainable
agricultural practices. Achievements at this level are project outputs, or what was done.
The inputs are the human, financial and material resources that were provided, to achieve
the objectives.
 II. Efficiency: what is the cost of achieving the objectives?

A project may be very 'effective' in working towards its objectives, but it may be doing so
at very high cost (both socially and economically), which is neither or sustainable. There
may be ways of achieving the same things more cheaply. This may involve looking at
how things are organized, what type of technology is being employed, as well as financial
management.

III. Relevance: is the project relevant?

A project may or may not prove to be appropriate to the needs of the people it is designed
to help. There might be other problems that should take priority. The overall approach
and strategy of the project should be consistent with the problem and intended effects.

IV. Impact: what are the effects of the project?

The impact of a project is the social, economic, technical, environmental and other effects
on individual and communities directly or indirectly involved in the project.

The impact on individuals may be different. Part of the impact may have been changes in
the institutions involved. The number of people affected should be estimated.

Impacts can be intended and unintended, positive and negative, immediate or long-term.
They can operate at the micro level (for example, at household level) or macro level (they
may affect a whole sector).

Sometimes a distinction is made between shorter-term results (outcomes) and longer-term

results (impacts). For example, outcomes could be; a change in the way people do things
as a result of the project, e.g. domestic servants may no longer be prepared to work for
less than minimum wages. However, if the overall impact hoped for is an improvement in
standard of living, and basic health, through improved income and conditions, it could be
very difficult to prove the causality between the intervention of a small project and what
actually happens. So many other external influences can have an effect.

V. Sustainability: will project activities and benefits continue after

external support is withdrawn?

Many types of work cannot financially be well sustaining. The issue is how far the people
directly involved can take charge them selves of finding the resources necessary. There
are two key aspects of sustainability for social development projects: social/institutional
and economic.

Another aspect of sustainability is the effect the project has on the environment and
natural resources.

VI. Progress: is the project achieving the original objectives, or have these
changed?

An evaluation can also question the objectives and design of the project itself. It may be
concluded that a project is progressing very well, even though it is far from meeting the
original objectives. These may have been too ambitious, or irrelevant. The evaluation
may also look at who was involved in setting objectives: was it only project leaders or
were staff and beneficiaries also involved?

1.3.6 The Steps in Doing an Evaluation

Whatever the purpose of an evaluation, or the methodology chosen, there are seven
distinct stapes in the process. All the seven phases are critical for provision of useful
information. If the information gathered is not perceived as valuable or useful (the wrong
questions were asked) or the information is not credible or feasible (the wrong techniques
were used), or the report is presented too late or is written inappropriately, then the
evaluation will not contribute to the decision making process.

Step 1: What/Who is it for?

Detailed planning should start well in advance. Check that the plans for evaluation made
at the assessment stage of the project are still valid; it may be that the passage of time and
the development of the project, or the quality of monitoring, make the original plans
redundant.

Many different people will use the findings of the evaluation exercises. It is therefore
necessary to determine at the beginning how the results will be used. This will help to
determine the objectives of the exercise, what information is needed, what approach
should be used, how the information should be gathered, what degree of accuracy is
needed, and how and to whom the results will be presented.

An evaluation is necessary to:

 Assess the objectives and their relevance

 Assess the progress in achieving the objectives
 Evaluate the impact of the project
 Assess the long-term sustainability of the project
 Provide a basis for further decision-making
 Influence donors, governments, and other actors
 Look at external factors that may affect the project

An internal evaluation is used within the organization to bring about improvements in

project implementation. An external evaluation is undertaken by outside evaluators, often
at the request of donor agencies, because they are usually considered to be reliable,
objective, and unbiased.

Step 2: Who will do it?

As noted above in Step 1, internal evaluation is used within the organization to bring
about improvements in project implementation and thus project staff can conduct it
while external an outside consultant may conduct evaluations.
Step 3: Preparation
In this step, the ground preparations are made for the monitoring or evaluation exercise.
This includes finalizing the objectives of the exercise, looking at information already
gathered, and deciding on data collection and analysis methods.

 Objectives

For any monitoring or evaluation exercise, objectives should be set depending on the
overall purpose of the exercise, and for whom it is being conducted. The objectives
should always be SMART.

Objectives for an evaluation include assessments of the following:

 Is the project progressing toward its objectives?

 What has the impact been?
 Who has benefited, and how?
 Are the objectives of the project still relevant?
 Is the project cost-effective?
 Is the project sustainable?

Objectives also include recommendations, such as:

 How the project can be improved

 How the goal and objectives can be changed

 Key questions

Next, key questions to be answered during the exercise should be developed. Key
questions should focus the exercise on the objectives of the project or the part of the
project being analyzed.

Key questions must always be realistic. Therefore, when formulating the questions for the
exercise, it is important to take into account the resources available to answer the
question, the importance to the project of answering the question, and what is already
known about the issue.

Indicators are integral to the formulation of key questions and how they will be
answered. Indicators, which should be developed during the project planning stage, are
very useful for the evaluation of the project. These indicators can measure project
processes, progress made in achieving the objectives and the project’s impact.

 Background information

Background information about the project is useful when planning evaluation exercise.
Worthwhile information to examine includes: a) details about the issue, such as
information about the local context, services and resources that already exist, and how the
issue affects the target group; and b) details about the project, such as its history, its
progress and process, and its impact thus far.

Developing means for obtaining information should include brainstorming about:

 What information is needed?

 Where can the information be found?
 What methods should be used for collection?
 What information is available from previous monitoring or evaluation?
 What information can be gathered from a field visit?
 How will the information be analyzed?
 What skills and technical expertise are needed to gather and analyze the
information?
 What sort of communication skills and language abilities are needed for
interviewing people?
 What equipment will be needed, and how will it be obtained?

 Drawing up the terms of reference (TOR)

Many evaluations suffer from the over-wide scope of the exercise. The terms of reference
(TOR) become a shopping list, rather than a guide to focus the work.

The TOR set out the formal agreements about the evaluation, its scope, purpose, and the
methods to be used, and outline the specific tasks of the evaluation team leader. Those
managing the evaluation process are responsible for drawing up the TOR.

Good TOR pave the way for a good evaluation, acting as a point of reference throughout.
They should be drawn up well in advance of the date of the evaluation, in order to allow
adequate time for planning, selecting and employing evaluators, sorting out the logistics,
and briefing everyone involved. The TOR should reflect both the needs of staff and
others involved to learn from their experience, and the need of the organizations to
improve performance and accountability. Evaluators may help to draw up the TOR and
should be asked to review and comment on them before beginning the evaluation. A good
evaluator is likely to raise questions about the initial TOR.

Sample Contents of the TOR

Terms of reference should cover:

1. Background: purpose and objectives of activity, work, project or

program to be evaluated.
2. Objectives: major issues to be addressed, what the evaluation is
expected to find out, the questions to be answered.
3. Methods: visits, review of documentary material, data collection,
interviews, workshops.
4. Timetable: schedule for the major activities (e.g. pre-visits, field
work, writing, feedback) of the evaluation and its completion date.
5. Products: the products required from the evaluation exercise, (e.g.
report, workshop), who is responsible for producing them, who will present them,
who the reports are for. The length, format and language of the main report and
executive summary should be indicated. The team leader is usually the person
 responsible for the completion of the formal report. The process of follow-up should
be noted.
6. The evaluation team: the person specifications (mandatory and
desired) of each team member, the number of team members, the ideal combination
of skills and experience at team level (including language requirements, gender
balance, and understanding of gender issues).
7. Budget and logistics: details of the main expenses (e.g. salaries,
expenses, travel, lodging, communications). Financial reporting requirements (e.g.
reimbursement for actual, or perdiems). Logistical support being offered (e.g.
vehicles, office space, computer facilities, secretarial help), and how, where and by
whom this will be made available.
8. Use of information: extent of confidentiality, ownership of the
report.
9. Terms of reference: for evaluation leader the team leader of the
evaluation must be given an individual TOR outlining his/her specific tasks and
responsibilities, particularly any writing and managerial tasks.

Step 4: Collecting information

Collecting information is central to the monitoring and evaluation processes. It includes

analyzing existing information, conducting field visits, interviewing people, organizing
meetings, and ensuring all key stakeholders are included.

It should be noted that there is no single correct way of conducting evaluation exercise.
Ideally, use more than one method for collecting data. Two or three methods that
complement each other can provide the most complete picture possible.

Step 5: Analyzing the information

It is very important that the collected data is accurately analyzed. Analysis may include
entering data, use of computer projects, recording information, summarizing findings,
and translating documents. The analysis should be unbiased, and should take into
consideration different points of view. The project planning must be done correctly, as
the indicators and tools used during the planning phase will be used again for evaluation.

Step 6: Report preparation, presentation and feedback

I. Drawing conclusions and recommendations

Conclusions can be drawn as a result of the monitoring and evaluation exercises. These
conclusions, which are always based on the exercise results, will reflect the information
gathered and the course of future directions to take. Conclusions can also reflect differing
opinions, if applicable. It is often useful to use case studies or quotes to illustrate points.

Recommendations are then made based on the conclusions.

Recommendations should:
 Propose a course of action
 Demonstrate how, when, and by whom the actions should be taken
 Mention the inputs and resources required for the action
 Discuss obstacles that may be faced in implementation
 Propose follow-up and monitoring systems necessary for ensuring that action
is taken

II. Report preparation and presentation

It is best to decide on your report format as early as possible. A mixture of print and
visual reports can clarify your messages. The type of report depends on whom it is for.
That is the report should be written with the main users in mind: who needs what
information in what form?

There is no set rule about the length of a report, but it should not be so long that nobody
would read it! Supporting information should be in the form of accompanying
documents, or appendices, rather than overloading the main report.

Sample evaluation report format

Here is an outline of what a report should cover:

Front cover
 Name of organization, name of project, location
Summary page
 Name of organization, project, and location
 Who carried out the evaluation
 Purpose of the exercise
 Dates it was undertaken
 Date of report completion
 Acknowledgments of those who contributed
Table of contents
Executive summary

 A brief summary of the report, including the goal and objectives of the
exercise, who it was for, and how it was undertaken. In addition, conclusions
and recommendations should be listed.
Background information
 Information about the local context (political, economic, social) as well as
project history, objectives, etc.
Purpose and methodology of the evaluation
 Objectives and key questions
 Data collection methods chosen
 Data collection process, including who was interviewed, where, the duration,
etc.
 Any data collection constraints and how they were overcome
Results
 What the findings were (categorized into subheadings)
 Utilization of case studies and quotes
Conclusions
 Based on the results, draw main conclusions which reflect the original
objectives and key questions
Recommendations

 What should be done and how it should be undertaken

Appendices (if applicable)

 Data collection methodologies used (e.g. questionnaire, interview format,

etc.)
 Actual data collected
 Terms of reference for evaluators

III.Dissemination and feedback

After the report is complete, it should be disseminated to the people for whom it was
written (depending on if it is internal or external) and also to all who participated in the
process. It is usually best to write a draft report, receive feedback, and then finalize it.
The report should then be stored so that it will be available to people currently working
on the project, as well as for future projects and evaluation exercises.

Step 7: Follow-up

Follow-ups to recommendations are integral to any evaluation exercise. Thus, when

planning an exercise, it is crucial to allow time and resources to be spent on following up
the recommended changes and actions to be taken. These actions are extremely important
for project improvement. Furthermore, the information must be shared with all concerned
parties, so that they are kept informed on your project’s process and impact, and can
make any necessary adjustments to their own projects. Results should be shared with all
stakeholders.

Summary

The Execution Phase of the project is when the benefits of the investment of time spent
on feasibility and detailed planning become apparent and the true value of the plan is
seen. During this phase, the Project Manager and Project Team monitor progress against
time, cost and quality targets in an efficient way. By having the key information
available, the ability to respond quickly to changes or problems should become evident.
Information sharing is improved by use of a common process. Based on sound planning,
additional benefits will include more effective monitoring and control, improved change
control and better risk management.

The Closeout Phase of the project marks the actual completion of the project. The
benefits and results are systematically evaluated. The time and effort invested by the
Project Manager and team are recognized and rewarded. The learning gained by
completing the project is captured for reference to help with the planning of future
projects.

There are several phases of conducting and implementing an evaluation. No one stage is
more important than the rest. And, as can be seen from the discussion of the role of the
stakeholders in both the first step—developing questions—and the last—provision of
information—the groundwork laid at the earliest stages can have important implications
for the success of the evaluation in the long run.

Evaluation isn’t easy, but there also is very little mystery about the steps that need to be
taken and the activities that need to be carried out. While there certainly are technical
skills needed to do an evaluation that is helpful and credible—and that is why trained
evaluators are important—there is also a lot of “common sense” involved. Sound advice
is to blend these two factors—technical skills and common sense. In the best evaluations,
both of these inevitably exist.

The next chapter considers practical considerations when planning a specific evaluation
exercise. There is advice on drawing up terms of reference, and ensuring better feedback
and follow up, including suggestions on presenting evaluation reports. The final part of
the chapter covers particular points that evaluators need to consider.