7 Steps For Setting-Up A Monitoring & Evaluation System

Designing a Monitoring and Evaluation system, or M&E system, is a complex task that usually
involves staff from different units. This article describes the development of such a system in 7
steps (1). Each step is linked with key questions, which are intended to stimulate a discussion of
the current state of the M&E system in a project or in an organization. Therefore, the 7 steps do
not represent a strict chronological sequence for the development of an M&E system. All steps
should be considered from the beginning:
 Step 1: Define the purpose and scope of the M&E system
 Step 2: Agree on outcomes and objectives - Theory of change (including indicators)
 Step 3: Plan data collection and analysis (including development of tools)
 Step 4: Plan the organization of the data
 Step 5: Plan the information flow and reporting requirements (how and for whom?)
 Step 6: Plan reflection processes and events
 Step 7: Plan the necessary resources and skills

Step 1: Define the purpose and scope of the M&E system

It is crucial to define the scope of the M&E system at the very beginning. A question that will
likely need to be answered is whether the system should be impact-oriented and whether we even
want to monitor higher-level impacts, or whether the project team is satisfied with simply
recording the proper implementation of activities and their results. Both can make sense and be
correct, depending on the circumstances and what is needed to be able to improve the project in
the best possible way. Of course, high-level results-oriented monitoring is usually preferable, but
it could also fail due to the available capabilities. Moreover, it should be clear from the
beginning who will continue to work with the M&E findings later on.

A challenge in this first step is engaging staff and convincing them that the additional time and
effort to set-up an M&E system is worthwhile in order to improve project steering and thus the
quality of project or program results. There are many and varied activities that can be carried out
to this end. For some project/program teams, a workshop or a presentation may be helpful to
convince of the usefulness of monitoring; in other cases, various face-to-face discussions may be
more appropriate. The approach must ultimately be decided by the person responsible for M&E
and depends on the resources available and the key people involved. Prior consultation on these
can be useful.

Step 2: Agree on outcomes and objectives - Theory of change

(including indicators)
A Theory of Change (ToC) is a description of how and why activities are expected to lead to
short, medium, and long term outcomes over a period of time. This is more than just identifying
outcomes and objectives. In fact, a ToC is a set of impact assumptions or hypotheses that can be
described as a visual diagram, a narrative, or both. Once there is a ToC or something similar,
staff will know better which M&E data to collect.

ToCs are not necessarily complex but they do provide a way to summarize the complexity of a
situation and bring clarity to it. At best, this allows a wide range of stakeholders to come to a
shared understanding of why and how activities will lead to desired results. Further information
and examples of ToC are provided by Culligan & Sherriff in "A Guide to the MEAL DPro".

It is helpful to involve a variety of stakeholders when developing the ToC – this could include
staff, beneficiaries, partners, funders and even other experts who are familiar with the technical
theme. The development process, and the thinking involved, are often as important as the
diagram or narrative produced. However, if this seems too time-consuming, a common, good
practice is to produce a first draft, which can be then discussed with other key stakeholders. The
result of this work should be a complete but not over-complicated description of the activities
and its results, with prioritized outcomes for measurement and SMART indicators (2) to collect
data against them.

Of course, ToCs need regular review because as context and needs change so do they. But high
level outcomes and impacts are usually valid for some years.

It’s worth noting that an important critique of ToC is that it neglects social realities and possible
negative project effects and that it might narrow the view on planned project/program goals.

Step 3: Plan data collection and analysis (including development

of tools)
For this step, it is recommended to create an M&E work plan, an M&E matrix or a combination
of both. Depending on the needs of a program or project, the design of such a document may
vary greatly. However, to ensure that the M&E activities are implemented, it is advisable to
determine in such a matrix clear responsibilities with timelines or frequencies of data
collection. Evaluation Toolbox provides a template that can be customized according to these
needs

The methods to collect the data depend on the information needed. For example, if quantitative
information on jobs created is needed, then a survey with a standardized questionnaire may be
useful whereas if information on the reasons for behavioral changes of supported groups is
required, qualitative interviews or a combination of interviews and a standardized survey may be
more useful. Data collection tools (e.g. interview guidelines and questionnaires) should be pre-
tested before they are actually used. Important guidance on how to develop such tools can also
be found in the aforementioned Evaluation Toolbox and at the websites of INTRAC. Some of the
staff involved probably need to have analytical skills (e.g. statistical skills in the case of analysis
of questionnaires). If such skills are missing or there is no time to develop the required tools or to
analyze the collected data, hiring an external M&E expert could be considered.

For some programs or projects, there might be an M&E officer who coordinates such M&E
activities. If this is not the case, then it will be necessary that the staff coordinate the activities
among themselves. It will then be even more important to assign clear responsibilities within the
team.

Step 4: Plan the organization of the data

To use the collected data, the information needs to be stored and shared with the people
involved, regardless of their location. One can store data physically or digitally using an
information system. This means that M&E systems and data management go together. The data
management system should be designed according to the needs, size and complexity of the
project or program. Staff engaged in M&E activities, may need to liaise with the IT support of
their organization. In any case, it is important to label and organize items in storage clearly
(chronological, by location, by content or any other category considered useful).

Good data management includes storing data securely to avoid unauthorized access, theft, or
unintentional destruction of data and to comply with any legal requirements, such as data
protection legislation. This often involves IT protection methods, such as passwords, firewalls
and virus checks. But it might also simply mean having a lock on a filing cabinet. The global
collaboration organization BetterEvaluation synthesizing advice from the UK Data Archive (3)
recommends: not to store digital data on externally networked servers or computers; the
installation of firewalls and security systems to protect against malware and viruses; the
existence of password protected computer systems; the encryption of sensitive materials (even
when transferring data by email); the signing of non-disclosure agreements. If it is important for
a survey to include personal data, such as address or name, it is essential to obtain the permission
of the respondent beforehand.

When using external software for data management needs, the terms of use, data protection and
confidentiality, and the servers location should be checked.

Data management is linked with data quality assurance too: It is important to avoid gathering
data of low quality and to ensure that data is "cleaned" of any errors. The collected data may be
the basis for further decisions. Data quality methods may include the use of multiple data
sources, such as triangulation of data and interviewer training and supervision. It should be
clarified among the staff who is responsible for data quality assurance and how.

Step 5: Plan the information flow and reporting requirements

(how and for whom?)
To be useful, information gained through M&E needs to be communicated to different
stakeholders. Most likely, there are certain reporting requirements set by donors. However, it is
good practice to disseminate and to discuss findings among other stakeholder groups so that
learning from M&E has a wider reach. At the very least, the M&E results should also be
discussed with the supported communities and groups.

There are many ways to communicate M&E information with stakeholder groups. The best
communication method will depend on the audience and how the information will be used. For
example, project managers may require much more detailed information on the progress made;
program directors may require regular, short summary reports across different projects and
programs, with aggregated tables and statistics; policy-makers might benefit from a short brief
summarizing the main issues, and making recommendations for change; a member of the public
that supports an organization through donations might prefer to see a story of change, a
photograph or a short video that enables to connect with beneficiaries on an emotional level.

Sometimes, especially when communicating information to partners or supported groups, it is

useful to discuss communication methods with the audience beforehand. This is a core element
of participatory M&E. Consideration should also be given to how information can be
communicated to people with audio or visual disabilities, or whether stakeholders are able to
access the venues for meetings. When communicating information to illiterate or semi-literate
people, presenting information in written form is of little use.

It is also important to know when information needs to be communicated. For example, if

decision-making meetings occur on a quarterly basis then it is important to communicate M&E
findings before those meetings are held. Similarly, when seeking to influence a government
policy, it is important to supply information at the right time so that it has the maximum chance
of achieving its purpose.

The use of communication strategies or dissemination plans will facilitate the organization of the
information flow. The key point is to be very clear about who needs what M&E information,
when and where. Narratives (formal reports, case studies, newsletters, press releases, policy
briefs) are the most common way of communicating M&E findings. Other means of
communication are through photographs, videos, pictures and cartoons. The big advantage of the
latter mentioned communication channels is that they can communicate information from
supported communities and groups directly to different audiences, without being filtered through
a report. In addition, M&E findings can be communicated verbally in meetings and workshops,
through feedback sessions and even through informal conversations. Speaking directly to a target
audience allows messages to be tailored to the individual or group, and allows for some
discussion of findings as well.

Also, more artistic and traditional methods of communication such as poems, drama, mime and
song can be used to share M&E information with others. Using such activities can help prevent
M&E becoming a sterile exercise, and can foster a broader understanding and discussions about
change.

Recent technological advances offer another way of communicating M&E information. Websites
and social media sites, podcasting, and webinars have made it much easier to present and
communicate information in new and innovative ways. Communication via mobile phones and
tablets offer further opportunities in the communication of M&E information, although to date,
this has mainly been used for data collection (for example surveying through text messages)
rather than for communication of M&E findings (4).

The dissemination plan below, taken from the website of the Technical Centre for Agricultural
and Rural Cooperation provides a good example of how such a plan could look like. Other
examples are offered by the websites of BetterEvaluation and MEAL DPro initiative.

Dissemination Plan

Audience Purpose Message Products and Timeline

channels

Who do you want For each target For each target For each target For each
to reach, audience: audience: audience: product/
who needs to learn what is the what are the lessons what are the channel:
about your purpose for that you want to best ways to when do you
experience? sharing with share with them? reach them? plan to share?
them? Which steps
need to be
taken?

1.
Dissemination Plan

2.

Etc…

Source: Website of the CTA. Technical Centre for Agricultural and Rural Cooperation, licensed
under a Creative Commons Attribution-Share Alike 3.0 Unported License

In some organizations, there is a communications officer responsible for external

communications and organizing the flow of information; in others, especially smaller
organizations, this task is the responsibility of an M&E officer or the project team itself.

Step 6: Plan reflection processes and events

This step goes a bit further and wants to not only communicate results but discuss M&E findings
with stakeholders so that everyone learns from each other. Again, the discussion and learning
formats can vary widely. These could include, workshops, exchange visits, seminar, conferences,
After-Action Reviews (AAR) (5) to name a few. However, learning does not happen in one
sitting. It is important that moments of reflection take place regularly throughout the life of the
project or program. The incorporation of learning events in the project/program cycle is key. In
this regard, annual or bi-annual reviews are critical learning opportunities to reach conclusions
about achievement and failures. The optimal sequence of learning events follows reporting lines
of decision making. It should be ensured that the right people are involved in such reviews.
Therefore, it may sometimes be important to include decision makers in the reviews (so that they
learn at the same time as their staff and therefore make appropriate decisions), but it can be a
challenge to ensure that this does not affect the openness of the conversations. It could be very
helpful if staff are trained in facilitating intentional group learning processes.

Regular team meetings are another important opportunity for reflection. Team members may
include project staff, implementing partners, and primary stakeholder representatives – this
depends on how the project is structured. Weekly meetings are common but if other stakeholders
are involved this may be needed less frequently. In each project context, there are usually forums
where implementing partners interact with each other. These events offer another chance for
reflection.

It is recommended to assign roles and responsibilities for leading the learning events. In addition,
the learning and resulting conclusions for further actions should be documented well, with a
focus on documenting “action needed”, “person responsible for implementation”, “deadline”,
and “persons responsible for follow-up”. Such documentation could be tabular (see the example
below), but any other form is fine as long as it records the most important items.

Documenting learning and conclusions

Weakness Improved action Person(s) responsible Timeline Unit/person responsible

identified suggested for action for follow-ups

Source: own composition

Lastly, it is worth mentioning here that learning in the context of M&E is about having a culture
that encourages intentional reflection and processes that support this culture. All teams learn as
they implement project activities. But to take advantage of this learning and consistently translate
it into improved practice, learning must be planned and managed.

Step 7: Plan the necessary resources and skills

It is good to start planning the M&E budget already in the project/program design phase so that
adequate funds are allocated and later available for M&E activities. There is no standard formula
to determine the budget for a project/program’s M&E system. An industry standard is that
between 3 and 10 percent of a project/program’s budget should be allocated to M&E (6). A
planning table for key M&E activities can be useful in this regard. It is particularly important to
budget for any expensive items, such as baseline surveys and evaluations.

Moreover, an effective M&E system requires capable people. Therefore, when defining roles and
responsibilities for M&E, specific consideration should be given to the M&E qualifications and
expectations, including the approximate proportion of time for each person to support the
system. A first step in planning for M&E human resources is to determine the available M&E
experience within the project/program team, partner organizations, target communities and any
other potential participants in the M&E system. This will inform the need for capacity building
or outside expertise. For long-term and larger projects/programs, it may be useful to create an
M&E training schedule. Ideally, data collection, analysis and M&E training involves the people
to whom these processes and decisions most relate (7).

One key planning consideration is who will have the overall responsibility for the M&E system.
It is important to clearly identify who will coordinate all these M&E activities and to whom
others will turn to for M&E guidance. The responsible person or team should supervise the M&E
functions, and have an overview of any problems that might arise.
This article was written with the intention to support especially smaller organizations in their
M&E activities. Hopefully the information has been helpful and of practical use in setting up
M&E systems. The author welcomes suggestions, additions and comments.

Susanne Neymeyer is an independent M&E consultant with more than 15 years experience in the
field of development cooperation and humanitarian aid. Her academic background is in social
work, adult education and evaluation. Susanne started working as an independent M&E
consultant in 2009. Since then, she has evaluated and supported a wide range of development
and humanitarian projects and programs all over the world. Before she became an international
consultant, she worked for various international development and humanitarian organizations as
project coordinator and manager.

Footnotes, references and further reading

 (1): In other guidance notes the development of an M&E-System is described in 6 or 10
steps. I have a preference for 7 steps, but a representation of the process with fewer or
more steps is of course just as good.
 (2): SMART indicators are Specific, Measurable, Achievable, Relevant and Time-Bound.
More information about SMART indicators and how to design them is provided by
the INGO People in Need (PIN)
 (3): Van den Eynden, V., Corti, L., Woolard, M., Bishop, L. and Horton, L. (2011).
Managing and sharing data: Best practice for researchers. UK Data Archive, University
of Essex: Essex.
 (4): More information on innovative tools can be found at Glenn O’Neil. (2017) A Guide:
Integrating Communication in Evaluation
 (5): An After-Action Review (AAR) is a simple process used by a team to capture the
lessons learned from past successes and failures with the goal of improving future
performance. More information on AAR and other knowledge sharing methods provides
the Knowledge Sharing Toolkit developed among others by the Food and Agriculture
Organization of the United Nations (FAO) and the United Nations Children's Fund
(UNICEF).
 (6): IFRC. (2011). Project/programme monitoring and evaluation (M&E) guide
 (7): Ultimately, the degree of participation of supported communities will vary according
to the project/program and context. Some examples of M&E participation include among
other activities: vulnerability capacity assessments; involvement of local representatives
in the project/program design and the identification of indicators; participatory
monitoring where elected community representatives reporting on key monitoring
indicators; sharing monitoring and evaluation findings with community members for
participatory analysis and identification of recommendations.