Project Monitoring, Evaluation and Closure
Project Monitoring, Evaluation and Closure
Project monitoring is defined as a process which is performed to track the progress of project execution
so that potential problems can be identified well in time for the taking of the corrective actions for the
purpose of controlling the execution of the project. It is a continuing function during project
implementation which provides management of an ongoing development intervention with indications
of the extent of progress and achievement of objectives and progress in the use of allocated funds. It
provides information and ensures the use of such information by management to assess project effects
– both intentional and unintentional – and their impact. It aims at determining whether or not the
intended project goals and objectives are being on the track.
Project monitoring can also be defined as the ongoing process by which management gets regular feedback on the
progress being made towards achieving the goals and objectives of the project. It focuses on reviewing of progress
against achieving of goals. In other words, monitoring is not only concerned with the taking of the actions but is
also concerned with making the progress towards achievement of the results. In the more limited approach,
monitoring can focus on tracking project with regards to the use of the resources. In the broader approach,
monitoring also involves tracking strategies and actions being taken by management, and figuring out what new
strategies and actions need to be taken to ensure progress towards the project objectives.
Objectives of Monitoring
In defining the term monitoring, one needs to be exposed to a number of concepts associated
therewith. Monitoring is the continuous assessment of a programme or project in relation to the agreed
implementation schedule. It is also a good management tool which should, if used properly, provide
continuous feedback on the project implementation as well assist in the identification of potential
successes and constraints to facilitate timely decisions. Unfortunately, in many projects, the role of this
is barely understood and therefore negatively impacts on the projects. Monitoring is not only concerned
with the transformation of inputs into outputs, but can also take the following forms:
Physical and financial monitoring :Measuring progress of project or programme activities against
established schedules and indicators of success.
Process monitoring: Identifying factors accounting for progress of activities or success of output
production.
Impact monitoring: Measuring the initial responses and reactions to project activities and their
immediate short and long term effects.
Projects are monitored so as to:
One needs to recognize the role played by the various stakeholders in monitoring. These players include
the financiers, implementing agencies, project teams, interested groups such as churches,
environmentalists, etc. It should further be recognized that, to be an effective management tool,
monitoring should be regular but should take into account the risks inherent in the project/programme
and its implementation.
1. To determine whether recipient is carrying out its development project, and its individual activities,
as described in the application for assistance and the agreement.
2. To determine whether the activities are carried out in a timely manner, in accordance with the
schedule included in the Agreement.
3. To determine whether charging costs to the project are eligible under applicable laws, regulations,
and are reasonable in light of the services or products delivered.
4. To determine whether conducting activities are adequate to control over program and financial
performance, and in a way that minimizes opportunities for waste, mismanagement, fraud, and abuse.
5. To assess if the program has continuing capacity to carry out the approved project, as well as other
grants for which it may apply.
6. To identify potential problem areas and to assist them in complying with applicable laws and
regulations.
7. To assist in resolving compliance problems through discussion, negotiation, and the provision of
technical assistance and training.
8. To provide adequate follow-up measures to ensure that performance and compliance deficiencies
are corrected and not repeated.
10. To determine if any conflicts of interest exist in the operation of the program,
11. To ensure that required records are maintained to demonstrate compliance with applicable
regulations.
Steps of Monitoring
Step 1. Define the desired goal of the project
One of the most important assessment questions to ask is, “What best practices efforts are building
on”? “What is to expect to accomplish through this project? Another way to phrase this is: What
challenges/opportunity does the project address (e.g., lack of coordination for health services, poor
nutrition, and dental health)? Often projects are established to meet one or more specific goals and
build on evidence from promising efforts. These goals and efforts are often described in the original
project plans. Goals do not need to be measurable, but they must be consistent with the overall purpose
and intent of what you want to accomplish.
A prerequisite for evaluation is the development of a project plan with measurable objectives that are
logically related to one another and to the goals and interventions defined in the project proposal. All
project objectives should specify what is to be done and by when. It is imperative that project staff
clarify the project’s objectives for each goal. You and your project staff must review your project‘s
objectives, and if necessary, you may have to rewrite some of the objectives to reflect what you can
measure. If your preproject planning was done thoroughly, then, it should be relatively easy to
determine program goals, objectives and activities.
Attainable – Given Socio-Cultural, Political and Technical Reality and Capacity of Your System
Once all aspects of the objectives have been determined, the target value must be defined and
documented. The target value allows the indicators or terms to be measured, interpreted and analyzed.
Without an expected target level for your measure it will be very difficult to interpret and analyze your
indicator data. In other words did the plan do well or poorly on this measure? Was the measure
accomplished as expected or not? The target value adds specificity to the project expectation or
outcome statement(s). Remember that Quantitatively the target value is the level of change you would
like to obtain for all persons described in your target population. Quantitatively, it is that level of
information needed or deemed acceptable to help you/your team make decisions about your targeted
actions . You will need to identify a value or set of information as the baseline from which to develop a
meaningful analysis of the measurement of your defined target.
The types of information needed will be guided by the objective you assess. For example, when the
objective refers to what you plan to do, you must collect information on the types of services, activities,
or educational/training products that are developed and implemented; who received them; and their
duration and intensity.
Outcomes and indicators are often confused as one and the same, when they are actually distinct
concepts as has been described above. Each project is unique and is aimed at achieving a range of
different results (outcomes).
A Formula is a method used to provide a result (in the form of a fraction or summary statement) that
specify the extent to which an objective or sub-objective is achieved. To calculate a formula you must
have a baseline (starting figure), numerator (top figure) and denominator (bottom figure). For purposes
of the Modified Achievement Index the denominator is the target value or point of observation (in the
case of qualitative objectives) and the numerator is targeted activity or type of change (quantitative or
qualitative) that is be achieved (see the example in the Modified Achievement Index)
Step 6. Define steps/activities (interventions) that you believe are effective toward achieving the
objectives
A critical next step after the indicator(s), measure(s), numerator(s) and denominator(s) have been
defined and documented is to define and clarify what it will take to get the work done. What is to be
done depends on what defines the desired result or type of objective.
Steps/activities/tasks are actions that are necessary to accomplish the desired result. Modified
Achievement Index for example, they are descriptions of the details or logistics. Depending on time and
resources, these details can be intricate or simple.
However, they must be logically linked to the indicators and measure and the desired results. For
example, steps/activities/tasks for establishing an advisory group must be specific to achieving this
objective; so using the advisory group to assist with the development of a needs assessment form would
not fit, the latter would occur after the group has been formed and would be a part of a separate
objective or sub-objective.
An Intervention is any planned effort designed to produce intended changes in a target population. If
the objective is focused (also) on changing something (e.g., attitude, behaviors, system operation, etc.)
than it becomes necessary to identify, define and document what type(s) of intervention will be needed.
Step 7. Identify the source (s) of the data for each measure
There are four basic ways to collect assessment data: document review, observation, interview and
surveys. Sources of Data may include claims data, client service database, client Intake profile database,
local, state and national economic characteristic database, population-based data Interview documents
or notes, meeting documents or notes, client personal logs etc.
Documentation of the result comes from the product of the Formula to Measure Success. As discussed
earlier that formula provides a fraction or summary statement that indicates the degree to which the
objective or sub-objective has been achieved within a certain time frame (e.g., by the end of six months
or one year).
The result, effect or impact (expected or unexpected, intended or unintended) that can be reasonable
attributed to the activities or actions of a program. These can be intermittent (short-term/ongoing) or
ultimate (long-term or end-of-program) outcomes.
METHODS OF MONITORING
Performance indicators.
This identifies objectives and expected causal links and risks along the results chain. It
is a vehicle for engaging partners and can help improve programme design.
Theory-based monitoring
These are quick, cheap ways of providing decision-makers with views and feedback
from beneficiaries and stakeholders. They include interviewing, focus groups and field
observation.
These trace the flow of public funds and assess whether resources reach the intended
recipients. They can help diagnose service-delivery problems and improve
accountability.
These tools assess whether the cost of an activity is justified by its impact. Cost-
benefit measures inputs and outputs in monetary terms, whereas cost-effectiveness
looks at outputs in non-monetary terms.
Impact Monitoring.
Purpose of Evaluation
Evaluating a project means performing a rigorous analysis of completed goals, objectives and
activities to determine whether the project has produced planned results, delivered expected
benefits, and made desired change. The project evaluation process involves an analysis of
different components or indicators that characterize the project’s progress towards the
achievement of its goals and objectives.
Project evaluation is a systematic and objective assessment of an ongoing or completed project. The aim
is to determine the relevance and level of achievement of project objectives, development
effectiveness, efficiency, impact and sustainability.
Stages in Evaluation
Step 1: Define your stakeholders
Your stakeholders are supporters, implementers, recipients, and decision-makers related to your program.
Getting them involved early on will help you get different perspectives on the program and establish common
expectations. This helps to clarify goals and objectives of the program you’ll evaluate, so everyone understands
its purpose.
Articulating the answers to those questions will not only help with accountability and quality improvement, but it
will also help you promote the program to its beneficiaries.
As you begin formulating your evaluation, think about the specific purpose of the evaluation—what questions
are you trying to answer? How will the information be used? What information-gathering methods are best
suited for collecting what our organization needs to know?
Qualitative data offers descriptive information that may capture experience, behavior, opinion, value, feeling,
knowledge, sensory response, or observable phenomena. Three commonly used methods used for gathering
qualitative evaluation data are: key informant interviews, focus groups, and participant observation.
Quantitative methods refer to information that may be measured by numbers or tallies. Methods for collecting
quantitative data include counting systems, surveys, and questionnaires.
Types of Evaluation
The best development project will conduct different types of evaluations, constantly looking to
streamline their project or program at different stages and using different metrics.
Formative Evaluation
(also known as ‘evaluability assessment’)
Formative evaluation is used before program design or implementation. It generates data on the
need for the program and develops the baseline for subsequent monitoring. It also identifies areas
of improvement and can give insights on what the program’s priorities should be. This helps project
managers determine their areas of concern and focus, and increases awareness of your program
among the target population prior to launch.
Conduct sample surveys and focus group discussions among the target population focused on
whether they are likely to need, understand, and accept program elements.
Process Evaluation
(also known as ‘program monitoring’)
Process evaluation occurs once program implementation has begun, and it measures how effective
your program’s procedures are. The data it generates is useful in identifying inefficiencies and
streamlining processes, and portrays the program’s status to external parties.
Conduct a review of internal reports and a survey of program managers and a sample of the target
population. The aim should be to measure the number of participants, how long they have to wait
to receive benefits, and what their experience has been.
Outcome Evaluation
(also known as ‘objective-based evaluation’)
A randomized controlled trial, comparing the status of beneficiaries before and during the
program or comparing beneficiaries to similar people outside of the program. This can be done
through a survey or a focus group discussion.
Economic Evaluation
(also known as ‘cost analysis’, ‘cost-effectiveness evaluation’, ‘cost-benefit analysis’, and ‘cost-utility
analysis’)
Economic evaluation is used during the program’s implementation and looks to measure the
benefits of the programs against the costs. Doing so generates useful quantitative data that
measures the efficiency of the program. This data is like an audit, and provides useful information
to sponsors and backers who often want to see what benefits their money would bring to
beneficiaries.
Impact Evaluation
Impact evaluation studies the entire program from beginning to end (or at whatever stage the
program is at), and looks to quantify whether or not it has been successful. Focused on the long-
term impact, impact evaluation is useful for measuring sustained changes brought about by the
program or making policy changes or modifications to the program.
A macroscopic review of the program, coupled with an extensive survey of program participants, to
determine the effort involved and the impact achieved. Insights from program officers and
suggestions from program participants are also useful, and a control group of non-participants for
comparison is helpful.
Summative Evaluation
Summative evaluation is conducted after the program’s completion or at the end of a program
cycle. It generates data about how well the project delivered benefits to the target population. It is
useful for program administrators to justify the project, show what they have achieved, and lobby
for project continuation or expansion.
Conduct a review of internal reports and a survey for program managers and target populations.
The aim should be to measure the change that the project has brought about and compare the
change to the costs.
Goals-Based Evaluation
(also known as ‘objectively set evaluation)
Goals-based evaluation is usually done towards the end of the program or at previously agreed-
upon intervals. Development programs often set ‘SMART’ targets — Specific, Measurable,
Attainable, Relevant, and Timely — and goals-based evaluation measures progress towards these
targets. The evaluation is useful in presenting reports to program administrators and backers, as it
provides them the information that was agreed upon at the start of the program.
This depends entirely on the goals that were agreed upon. Usually, goals-based evaluation would
involve some survey of the participants to measure impact, as well as a review of input costs and
efficiency.
Development programs with effective monitoring and evaluation frameworks use different types of
evaluation at different points of time. Some programs might even run two different types of
evaluation at the same time for entirely different purposes. No matter what types of evaluation you
use, we hope you find this blog useful in making your project and program more successful and
efficient.
Evaluations can be defined in terms of different modalities of UNDP support,such as project and
programme, and also different levels or frameworks of results such as outcome, UNDAF and themes.
Evaluations can also be defined by when they are carried out: Ex-ante evaluation is a forward-looking
assessment of the likely future effects of new initiatives and support such as policies, programmes and
strategies. It takes place prior to the implementation of an initiative. Midterm evaluation generally has
a formative nature as it is undertaken around the middle period of implementation of the
initiative.Formative evaluation intendsto improve performance, most often conducted during the
implementation phase of projects or programmes. Final or terminal evaluations normally serve the
purpose of a summative evaluation since they are undertaken towards the end of the implementation
phase of projects or programmers. Summative evaluation is conducted at the end of an initiative (or a
phase of that initiative) to determine the extent to which anticipated outcomes were produced. It is
intended to provide information about the worth of the programme. Ex-post evaluation is a type of
summative evaluation of an initiative after it has been completed; usually conducted two years or more
after completion. Its purpose is to study how well the initiative (programme or project) served its
aims,to assess sustainability of results and impacts and to draw conclusions for similar initiatives in the
future. Evaluations defined by the modality of development initiatives or level of results can be further
defined by the timing. For example, a programme unit may undertake a final project evaluation or a
midterm UNDAF evaluation
The process of forecasting or approximating the time and cost of completing project deliverables.
The task of balancing expectations of stakeholders and need for control while the project is
implemented.
Beyond the broad classifications of direct and indirect costs, project expenses fall into more
specific categories. Common types of expenses include:
Labor: The cost of human effort expended towards project objectives.
Materials: The cost of resources needed to create products.
Equipment: The cost of buying and maintaining equipment used in project work.
Services: The cost of external work that a company seeks for any given project (vendors,
contractors, etc.).
Software: Non-physical computer resources.
Hardware: Physical computer resources.
Facilities: The cost of renting or using specialized equipment, services, or locations.
Contingency costs: Costs added to the project budget to address specific risks.
Analogous estimating: Like expert judgment, analogous estimating — also called top-down estimating
or historical costing — relies on historical project data to form estimates for new projects. Analogous
estimating draws from a purpose-built archive of historical project data, often specific to an
organization. If an organization repeatedly performs similar projects, it becomes easier to draw parallels
between project deliverables and their associated costs, and to adjust these according to the scale and
complexity of a project.
Analogous estimating can be quite accurate if used to form estimates for similar projects and if experts
can precisely assess the factors affecting costs. For example, a similar project conducted three years ago
might be used as the basis for a new project cost estimate. Adjust the estimate upward for inflation,
downward for the amount of resources required, and upward again for the project’s level of difficulty.
These adjustments are typically stated as percentage changes — a new project might require 10 percent
more preparation time and 15 percent more on resources.
Bottom-up estimating: Also called analytical estimating, this is the most accurate estimating technique -
if a complete work breakdown structure is available. A work breakdown structure divides project
deliverables into a series of work packages (each work package comprised of a series of tasks). The
project team estimates the cost of completing each task, and eventually creates a cost estimate for the
entire project by totaling the costs of all its constituent tasks and work packages — hence the name
bottom-up. Bottom-up estimates can draw from the knowledge of experienced project teams, who are
better equipped to provide task cost estimates.
While deterministic estimating techniques such as bottom-up estimating are undoubtedly the most
accurate, they can also be time-consuming, especially in large and complex projects with numerous
work breakdown structure components. It is not unusual for definitive estimates to also use techniques
such as stochastic, parametric, and expert-judgment-based estimating (if these have proved suitably
accurate in early estimates). That said, bottom-up estimating is also the most versatile estimating
technique and you can use it for many types of projects.
Parametric estimating: For projects that involve similar tasks with high degrees of repeatability, use a
parametric estimating technique to create highly accurate estimates using unit costs. To use parametric
estimating, first divide a project into units of work. Then, you must determine the cost per unit, and
then multiply the number of units by the cost per unit to estimate the total cost. These units might be
the length in feet of pipeline to be laid, or the area in square yards of ceiling to be painted. As long as
the cost per unit is accurate, estimators determine quite precise and accurate estimates.
It’s called the triple constraint for a reason. If you’re pulling one constraint lever, it will directly
affect the other two constraints. For example, if you decide to extend the scope and build 20
webpages instead of 10, you’ll need more time and money to achieve that adjusted goal.
The triple constraint is well-known, so it’s already been discussed a lot. But even if this type of
constraint is balanced, issues can occur. So I started wondering why.
Most project management books will tell you this: If you make sure that the triple constraint is
covered, you won’t have any problems. But you actually need to optimize further constraints as
well.
Quality
While this constraint is quite similar to scope, it’s slightly different. Scope defines the exact desired
outcome. For example, your project’s scope may involve creating 10 webpages. Quality
concentrates on the attributes of each of these webpages. It can be defined by answering this
question: “How closely does the outcome match the expectations?”
In our example, the quality doesn’t define the number of webpages, but there could be a quality
tolerance regarding the number of words. Perhaps you’ve requested 1,000 words, and you have a
quality tolerance of +/- 100 words. So if a webpage contains 900 words, you’d approve it. And if
another webpage only contains 850 words, you’d reject it.
Quality interrelates with the other constraints. Let’s assume you’re running out of time and need to
meet a certain due date. You could possibly meet the deadline by enlarging the quality tolerance
and decreasing the number of words to 800.
Risk
Managing risks is an important task for project managers. But what does this term actually mean?
When you estimate probability, a risk will have a certain impact on your proect. Perhaps you’re
creating a wireframe for your website, and you decide to skip the client-review step because you’re
running late. If so, there’s a risk that the client will reject your final webpages.
Of course, you can control risk to a certain extent. For instance, you could decide to avoid the risk
and insist on the review step. But this decision would affect your timeline and your related costs,
since the client would review each of your designs, and a project manager needs to oversee this
process.
Resources
Resources are strongly connected to the project cost. The amount of money that’s available for
achieving the desired outcome will restrict the use and acquisition of resources, which creates a
separate constraint.
Sometimes, even an infinite amount of money couldn’t allow you to acquire the specific resources
you need. For example, it’s going to take longer than you expected to receive a physical resource in
the project (such as a chip), which will cause you to miss the deadline. If this resource is essential to
the project, you’ll have to sacrifice making the deadline, because no reasonable amount of money
could reduce the delivery time.
Sustainability
The sustainability of a project can play a major role in the long-term strategy of a company, and can
often affect a project’s success. There are three parts of sustainability: social, environmental, and
economic. Even if the first two ones don’t apply to your project, the economic component shouldn’t
be neglected by project managers.
Basically, managing a project’s economic sustainability refers to the way you handle its possible
impact on the future of the the organization behind it. For example, if you’re managing an
automotive production line, you could use cheap resources to build some parts of the cars, in order
to save costs. But you’ll also be sacrificing sustainability, since cheaper parts tend to bite the dust
more frequently than high-quality pieces.
Coping with organizational structure and limitations can be difficult. You’re often not able to break
through the organization’s existing patterns, make the deadline, and stay within the budget.
Methodology
Each of these methodologies manifest various limitations. If you use Scrum, you’ll need to organize
daily meetings and get plenty of reviews, so you’ll have to acquire decent resources to cover these
efforts. But if you use Agile, you’ll be more likely to deliver excellent quality.
However, with Agile, it’s harder to estimate the needed time upfront, which will increase the risk of
missing your deadline. Hence, to choose the right methodology, it’s important to assess the project’s
situation.
Analyze the different dimensions of the project (novelty, technology, complexity, and pace).
Select a suitable methodology for project management.
Customer Satisfaction
Customer satisfaction measures how much you’re meeting the customer’s expectations. If your
team handed over excellent deliverables on time and within the budget, your client will probably be
happy.
But, if your cost turned out to be higher than expected, it would force you to make a tradeoff.
Perhaps you’re sacrificing customer satisfaction, since the product isn’t meeting your client’s
expectations anymore. In the end, customer satisfaction will highly affect the project’s success.
Conclusion
It’s crucial to know the different constraints of a project. The only way to control them is to
understand them. So when making important decisions, don’t just consider time, scope, and budget,
but also factor in quality, risk, resources, sustainability, organization, methodology and customer
satisfaction. Balancing all the constraints within project management requires robust tools for
tracking and reporting.
The close of the project is the final phase of your job, it’s the last turn of the project life cycle, and
like any other aspect of a project, it requires a process. The following are five steps you should take
to make sure you’ve dotted all the I’s and crossed all the T’s, as well as taken full advantage of the
experience.
Gather the core team to invite feedback about what worked, and what didn’t. Encourage honesty.
By documenting the mistakes and the successes of the project, you’re building a catalog that offers
historic data. You can go back and look over the information for precedents when planning for new
projects. Projects are never standalone things, but part of a continuum, where the specifics might
vary, but the general methods usually remain the same. There’s a wealth of knowledge produced
after any project closes.
2. Complete Paperwork
As noted, projects generate reams of documents. These documents are going to have to get sign off
and approval from stakeholders. Everything needs attention, and must be signed for, which is the
legal proof that in fact these documents have concluded. That includes closes all contracts you
might have made with internal partners or vendors or any other resources you contracted with.
This includes addressing all outstanding payments. You want to make sure that all invoices,
commissions, fees, bonus, what have you, are paid. Complete all the costs involved with the project.
It’s not done, if it’s not paid for.
3. Release Resources
You assemble a team for the project, and now you must cut them loose. It’s a formal process, and a
crucial one, which frees them for the next project. Each team is brought together for the mix of skills
and experience they bring to a project. The project determines the team members you’ll want to
work with, and each project is going to be a little bit different, which will be reflected in the team
hired to execute it.
This is true for internal as well as external resources. The external ones might be more obvious, as
you contracted with them, and that contract is going to have a duration. When it’s over, make sure
they’re all paid in full so they can sign off and leave. But internal resources remain, so you have to
remind yourself that their time on the project is also limited, and you might be blocking other
team’s projects if you don’t release your resources once the project is done.
4. Archive Documents
There are lessons to be learned from old projects, which is why you meet with your team regularly
during the project and look back on the process afterwards. However, if you don’t have an archive
in which to pull the old records, then whatever knowledge you gain is lost because of poor
organization and management. You worked hard to have great project documentation, don’t lose it.
Before you close a project, archive all the documents and any notes and data that could prove
useful. Even if you never access it, there’s a need to keep a paper trail of the work done on any
project for other people in the organization. This might include legal teams, or HR teams, or even
your successor. You never know when someone might have to go back and respond to a question or
want to learn how an old issue was resolved. Consider it like putting away provisions for the
winter.
5. Celebrate Success
If it sounds silly to you, then you’re not doing your job. There’s nothing silly about rewarding your
team to acknowledge a job well done. It creates closure, which is what this part of the project is all
about, but it also plants a seed that will bloom in later projects when you work with members of the
old team.
That’s because when you note a job well done you’re building morale. It makes team members feel
better. You might have been a hard task master in the project, but you give them their due for a job
well done. That creates loyalty, and they’re going to work even harder for you the next time. And
there will be a next time, because a happy team is a team that you retain. Why would you want to
close a project and lose the very resources that made it a success?
Many seem to think that project closure is not an important process in project management ,
but that’s not true. Closing a project is as important as other processes in project
management . Until and unless your project has been closed with the planned procedures, it
officially provides no value to the organization.