Professional Development

Seminars & Evaluation

Practice Exchange Sessions
13 - 15 May 2019, United Nations Office at Nairobi (UNON), Nairobi, Kenya
Professional Development Seminars

Seminar 1
Big data for evaluation: How to walk the walk
Organiser – Adan Ruiz Villalba (WIPO)
Facilitator – Claudia Isabel Marques de Abreu Lopes, Research Fellow,
UN University, International Institute for Global Health

Modality - Lectures and group work Time - Monday, 10:45am Location - Conference Room 10

In this seminar we will discuss how big data can be used for evaluation, providing some case studies for
discussion and guidelines to design an evaluation using big data. The participants will be exposed to critical
discussions about big data (related to measurement and ethics) and to new methodologies to use big data
as complementary to other data sources for evaluation.

Seminar 2
Evaluation Theories and Approaches:
Applications in evaluation practice
Organiser – Alena Lappo (FAO)
Facilitator – Daniela Schroeter, Western Michigan University
Collaborator – Julia Engelhardt (WIPO)

Modality - Small group activities, case studies, and open discussions Time - Monday, 10:45am Location - Training Room 4

This workshop presents an introduction to historical and contemporary theories and approaches
to evaluation in interdisciplinary contexts. The primary focus is on key evaluation terminology and
classifications of theories and approaches recommended by evaluation thought leaders. Workshop
participants will gain insight into how their own backgrounds, training, and contexts influences their choice
of or preference for particular approaches.

Participants are encouraged to bring a computer to use during the workshop. Please read the case found at
https://bit.ly/2Vw6Y4N prior to the Seminar.

Seminar 3
Approaches to impact evaluation
Organiser – Martin Barugahare (UN-Habitat)
Facilitator – Dr. Atika Pasha

Modality - Lectures, interactive exercises/practical applications of the methods Time - Monday, 10:45am Location - Conference Room 8

The seminar will introduce and discuss evaluation methods that can be used to determine causal and
attributable impacts of programmes/interventions. It will elaborate on how these methods differ from other
kinds of evaluations that are used commonly (process, M&E). Participants will learn how the framework
of the programme can determine the impact evaluation methods and design that are eventually applied,
and what programme and participant related issues can threaten the validity of this design. The session
aims to provide participants with an understanding of advantages of impact evaluation, and some technical
understanding of methods and designs.
 Seminar 4
Use of Qualitative Analysis Software for
Evaluations e.g. NVIVO
Organisers – Martin Barugahare (UN-Habitat), Sara Holst (FAO)
Facilitators – Victor Kiwujia (Lida), Ronald Waiswa (Lida)

Modality - Training datasets, online materials, and guiding analysis information Time - Monday, 10:45am Location - Training Room 6

This seminar is aimed at strengthening evaluation capacity of participants in use of qualitative evaluation
data. The training will provide participants with a thorough understanding of qualitative evaluation data and
a pragmatic step-by-step process for conducting qualitative data management, analysis and visualization.
The hands-on and very interactive training provides participants with improved knowledge & understanding
of qualitative M&E data; practical skills for qualitative data management, analysis and visualization using
NVIVO; and opportunity to share knowledge and experiences in qualitative M&E data analysis.

Participants are required to bring a computer to use during the session.

Seminar 5
Revisiting the Foundations of Humanitarian
Evaluation: Reflections on current practice and
future directions
Organiser – Jane Mwangi (UNICEF)
Facilitators – James Darcy (Independent Consultant) and Neil Dillon (ALNAP)
Collaborator – Ada Ocampo (UNICEF)

Modality - Presentations and associated discussion groups Time - Monday, 10:45am Location - Conference Room 11

The seminar will be a reflection from recent theory and practice on humanitarian evaluations, and the
session will consider the following:
• Context and trends in humanitarian evaluation
• Challenges from the humanitarian evaluation literature:
• Reflections on the function of humanitarian evaluations
• Evaluation evidence in the wider diagnostic framework
• Implications for UN agencies humanitarian evaluation work
Evaluation Practice Exchange Sessions
Theme 1: Managing Evaluations

Session 1.1
Providing comprehensive evidence
without losing utility
Organiser – Harvey Garcia (FAO)
Collaborators – Carlos Tarazona (FAO), Robert Stryk (UNICEF), Sara Holst (FAO)

Modality - Facilitated Oxford-style debate Time - Tuesday, 8:30am Location - Conference Room 8

In this seminar we will debate the different scenarios related to the use of evaluations. The session will
touch base on various variables that affect the length of the evaluation process such as the time it takes to
engage stakeholders and conduct a rigorous process (e.g. data collection), peer reviews, feedback loops,
and writing process. It will also address the need for comprehensive evidence in evaluation in the context of
development-related versus humanitarian-related evaluation.

Session 1.2
Coaching junior evaluation officers -
sharing our experiences
Organiser – Alena Lappo (FAO)
Collaborators – Alena Lappo (FAO), Katinka Koke (UNITAR), Marco Segone (UNFPA)

Modality - Testimonials from junior evaluators, discussions in a “speed dating” and a “negotiation game” format and a presentation
Time - Tuesday, 11:00am Location - Conference Room 8

Building on UNEG’s competency framework and the priorities set by the Evaluation Agenda 2020, this
session focuses on discussing challenges and opportunities for building capacities of junior evaluators and
evaluation officers from the perspective of both junior and senior UNEG colleagues. It will also provide
insights to young evaluators and their supervisors on defining a career path with meaningful and specific
goals as well as discuss useful steps in finding opportunities to reach them.

Session 1.3
Best Practices for Recommendation follow up.
Can we maximize use with the right procedures?
Organiser – Robert Stryk (UNICEF)
Collaborators – Michael Spilsbury (UN Environment), Ada Ocampo (UNICEF)

Modality - Presentation and group discussions Time - Wednesday, 3:00pm Location - Conference Room 10

There are different ways that recommendations can be followed up, from very loose procedures to rigid
procedures. Some are more engaging than others. The session aims to clarify what are the determining
factors to use different methodologies for recommendation and follow up, and what are emerging good
practices for these situations.
 Session 1.4
Whose evaluation is it anyway and why does it
matter? Let’s talk about country evaluations
Organisers – Ada Ocampo (UNICEF), Harvey Garcia (FAO), Ada Ocampo (UNICEF)
Collaborators – David Thomas Rider Smith (JIU), Harvey Garcia (FAO),
Robert Stryk (UNICEF), Carlos Tarazona (FAO), Sara Holst (FAO)

Modality - Talk show Time - Wednesday, 8:30am Location - Conference Room 8

The discussion in this session will be centered around the various aspects of ensuring national ownership
of evaluations and how it contributes to national evaluation system and fulfillment of the Agenda 2030.
More specificly, the discussion will include experiences from national (or government) evaluations done
by governments / CSOs of their own programmes, joint evaluations with UN, donor, government and CSO
involvement, and UN evaluations with inputs from government, CSOs and donors.

Session 1.5
As managers, we also make mistakes:
Let’s share them and learn from them
Organiser – Sara Holst (FAO)
Collaborator – Roberto Borlini (WFP)

Modality - Vignettes and group discussions Time - Wednesday, 2:00pm Location - Conference Room 10

This session will focus on personal reflection on how making mistakes, owning them and learning from them
is part of the professional journey for everyone, including managers.
As leaders, we gain credibility and authenticity when we acknowledge our own failings and are open in
discussing these with our peers and our staff. Leaders can create and reinforce a culture that counteracts
the “blame game” and makes people feel both comfortable with and responsible for surfacing and learning
from failures. This requires consistently reporting failures, small and large; systematically analyzing them;
and proactively searching for opportunities to experiment.
 Theme 2: Evaluation Standards, Norms and Principles

Session 2.1
We ask others to demonstrate impact: How can we
demonstrate the impact of our evaluations?
Organiser – Grace Igweta (WFP), Evaline Diang’a (WFP)
Collaborators – Purity Njagi (UN-Habitat), Marian Read, Jane Oteba Njoroge (RCO)

Modality - Plenary and round table discussions Time - Tuesday, 8:30am Location - Conference Room 10

This session will engage participants in a discussion around factors that hamper the use of evaluations,
from the design of the evaluation to its actual production, the latest thinking around enhancing use
of evaluations, and how impact of such use would and/or should be measured. It will use the concept
of “evaluation as intervention” and advance it along the lines that if we are to measure the impact of
our evaluations, we have to treat each evaluation as an intervention in the context within which it is
commissioned, managed and conducted. As with any intervention, a good intervention design is one that is
accompanied by a well thought out intervention theory (variously known as programme theory or theory of
change).

Session 2.2
Taking stock of evaluations towards achieving the
vision of 2030 Agenda: Open theory of change
Organiser – Juha I. Uitto, (GEF)
Collaborators – Suppiramaniam Nanthikesan (UNFPA), Oscar Garcia (IFAD),
Patricia Hurtado Vidal (ILO), Mikal Khan (FAO), Veridiana Mansour (FAO)

Modality - Fishbowl Time - Tuesday, 11:00am Location - Conference Room 10

Theory of change is an important aspect of infusing rigour and shaping the evaluations to capture
the intended consequences. As a rule (with notable exceptions, like IFAD), we seem to fall short of
systematically internalizing externalities when we use the theory of change as the basis for our evaluations.
Evaluations that are built around the internal logic of the intervention risk missing important dimensions in
the broader context.

This session aims to promote awareness and understanding of this gap. The participants will explain how
approach and innovations can help capture this dimension of constructing theories to address unplanned
side effects (positive or negative) on inequalities and/or related to environmental sustainability.

Session 2.3
Is impartiality of decentralized evaluations even
possible? What are the mechanisms and features
to ensure this?
Organiser – Robert Stryk (UNICEF), David Rider Smith (JIU)
Collaborators – Guy Thijs (ILO), Ricardo Furman (ILO)

Modality - Lectures, interactive exercises/practical applications of the methods Time - Tuesday, 3:00pm Location - Conference Room 8

This session is mainly aimed at those evaluation functions that have a significant interest in decentralized
evaluation and will explore different factors to be considered as well as methods of operation that will allow
better quality, more impartial decentralized evaluations. It will tackle questions such as, What is the rule
of decentralised evaluation in UN System Organisation, What are the challenges and how is the quality of
evaluations addressed?
 Session 2.4
Evaluations and audits: Experiences from combined
functions synergies or competition?
Organiser – Julia Engelhardt (WIPO)
Collaborators – Adan Ruiz Villalba (WIPO), Soo Mee Baumann (IAEA),
Robert Stryk (UNICEF)

Modality - Case studies and discussions Time - Wednesday, 8:30am

Location - Conference Room 10 with a secondary room in Conference Room 7

The session will provide examples of where evaluation functions could have synergies with audit and
evaluation. While audit and evaluation are complementary from an accountability perspective, yet they are
both different. Audits are done from a compliance perspective while evaluations are done from a learning
perspective and strong stakeholder involvement. The session will also provide examples of how constraints
and weaknesses could be transformed into strengths for both functions.

Session 2.5
OECD DAC evaluation criteria: recent developments
and potential for evaluation in the context of UN
reform, changes in UN development system and
Independent System Wide Evaluation
Organiser – Mikal Khan (FAO)
Collaborators – Julia Engelhardt (WIPO), Per Oyvind Bastoe (NORAD and DAC
EvalNet), Meghan Kennedy Chouane, (OECD-DAC and DAC EvalNet), Guy Thijs (ILO)

Modality -Vignettes and group discussions Time - Wednesday, 11:00am Location - Conference Room 3

So far, the OECD DAC criteria have provided a benchmark for evaluation managers, practitioners and users,
and while their application and interpretation have varied broadly, they remain an important reference
point. This session provides a forward-looking learning and exchange opportunity among UNEG members
on the new OECD DAC evaluation criteria and their appropriateness and use in future work. There will be an
opportunity to reflect on the new criteria by applying them in examples of actual evaluations and providing
ideas and ideas and inputs for the UNEG Norms and Standards and UNEG partnerships.
 Theme 3: Evaluation Methods

Session 3.1
Expanding our toolkit of evaluation methods:
Examples from emerging UN practice
Organisers – Tina Tordjman-Nebe (UNICEF), Andrew Fyfe (UNCDF)
Collaborators – Inga Sniukaite (UN Women), Alena Lappo (FAO),
Veridiana Mansour Mendes (FAO), Pietro Tornese (UNCDF)

Modality - Presentations and Q&A Time - Tuesday, 8:30am Location - Conference Room 11

The goal of this session is to look in detail at a number of predominantly qualitative evaluation approaches:
contribution analysis, qualitative comparative analysis and outcome mapping that have been applied by
different UN agencies: UNICEF, UNCDF, UNWOMEN and FAO in recent evaluation exercises. The evaluations
will be a mix of programme and thematic evaluations with a focus mainly on the UN’s support to Member
States’ development policy under the SDGs.

Participants are encouraged to bring a computer to use during the session. Please read the preparatory
materials at https://bit.ly/2GUC1hj.

Session 3.2
Making the best use of Theories of
Change in Evaluations
Organiser – Roberto Borlini (WFP)
Collaborator – Caspar Merkle (UN Women)

Modality - presentations and group discussions Time - Tuesday, 11:00am Location - Conference Room 11

In this session, we expect participants to share their experiences with theories of change, from development
to their application during evaluation exercises. Colleagues will go into break-out groups and discuss the five
“W” of theory of change: why (why we need them), when (when we develop them), who (who develops them),
what (what do they look like), where (where they should be applied). Finally, we will also discuss challenges
and lessons about the “how” (how we go about developing a TOC).

Session 3.3
How to go about the management of Joint
Evaluations: A best response game
Organiser – Jacqueline Flentge (WFP)
Collaborators – Alexandra Chambel (UNFPA), Bushra Hassan (UNDP Regional
Bureau for Asia-Pacific)

Modality - Plenary Introduction and Dynamic group activities Time - Tuesday, 3:00pm Location - Conference Room 11

With the global 2030 Agenda defining multi-stakeholder partnerships a tool for the achievement of the
SDGs, the undertaking of joint evaluations takes momentum. This session will allow participants to get
a better understanding of joint evaluation-related aspects that require coordinated decision making;
recurrent practical challenges and ways those can be overcome; and opportunities for coordinated
(i.e. among interested UNEG member agencies) planning and learning on joint evaluations.
 Session 3.4
Low-cost, useful and good quality evaluations?
Organisers – Jane Mwangi (UNICEF), Michele Tarsilla (UNICEF),
Ada Ocampo (UNICEF)
Collaborators – Martina Rathner (UNESCO), Javier Guarnizo (UNIDO),
Ricardo Furman (ILO)

Modality - Panel discussion and group work Time - Wednesday, 2:00pm Location - Conference Room 11

This session intends to trigger a discussion on the challenges and lessions associated with the management
and conduct of low-cost evaluations that could both meet professional evaluation quality standards and be
useful to evaluation commissioners and other relevant stakeholders. In an effort to better understand the
pros and cons of this specific type of evaluations, participants from different organizations will share their
experiences and reflections on the correlations existing amongst low-cost evaluations, quality and utility.

Session 3.5
Evaluating Policy Support
Organisers – Veridiana Mansour Mendes (FAO), Alena Lappo (FAO)
Collaborators – Andrew Fyfe (UNCDF), Pietro Tornese (UNCDF),
Julia Engelhardt (WIPO), Felix Herzog (UN-ESCWA), Xuebing Sun (FAO)

Modality - Showcase results, group work and presentation Time - Wednesday, 8:30am Location - Conference Room 11

This session aims to explore the challenges experienced by colleagues in the UN System when evaluating
policy support e.g. related to SDGs. Participants will work in small groups on real case challenges and will
jointly develop possible solutions by proposing methods and approaches to tackle them. At the end of the
session participants are expected to have: i) a common understanding on the challenges involved in this
type of evaluation; ii) knowledge of different practices experienced by the different organizations; and iii) a
set of options to evaluate policy support.

Lunch time seminar – Sustainable Development Goals: How prepared are

we for evaluation of interventions?
Organisers – Jillian Campbell (UN Environment), Ludgarde Coppens (UN Environment), Robert Ndugwa
(UN-Habitat), Donatien Beguy (UN-Habitat)
Collaborators – Tiina Piiroinen (UN Environment), Susanne Bech (UN-Habitat)

Modality - Presentation engaging the audience Time - Tuesday, 2:00pm Location - Conference Room 7

In the current context of UN reform, working at country level and integrating the principles of agenda 2030 such as
leaving no-one behind and the economic, social and environment dimensions, SDGs are pushing complex agenda.
Most SDGs are somehow connected, and their implementation will need to be synchronized for attainment of
sustainable development at the local, national and global levels.Through the lense of ongoing work on urban and
environment related SDG interventions, this session will discuss how certain indicators can provide information to
better assess changes and impact of interventions and how this has implications for evaluation.
UNON Compound map

Conference
Conference Room 3,
Room 7,8
UNEG Secretariat
Upper Concourse Lower Concourse

BLOCK 4N
2N 3N BLOCK
CK
BLO EASTERN CARPARK
Conference E BLOCK 1 N
BLOCK
3S 4S
BLOCK
Room 10,11 2S
CK
BLO
CONFERENCE BLOCK 1 S
Rooftop WEST CONFERENCE
EAST

Q Q
P P
NN
M
F
UN
SACCO

UN FCU

G CATERING LIBRARY T UU
BUILDING BUILDING SS
T
RR
MAIN
LOADING LOBBY
BAY

W
W
V
V
X
X
CARPARK
DOCUMENTS
& PUBLISHING
Prefab 13/14

Prefab 17/18 Prefab 15/16 Prefab 9/10 Prefab 5/6

Security Desk Pre
fab
Prefab 11/12 1
Prefab
7/8
DELEGATES Ask for assistance Pre
PARKING fab
Training Rooms 2
FIRE ENGINE Pre
fab
Lower Concourse 3

STAFF BUS
Prefab 4

WETLAND

ROAD NO.3
Walk from
entrance to
BOMB BLAST
main lobby. MEMORIAL
GARDEN
GATE

PAVILION

MAIN
GATE

UEVEN
UN A
Logistical Information

Registration and Access to the UNON Wi-Fi

Compound


Free Wi-Fi is available in the UNON


Participants attending the UNEG Compound and all meeting rooms.
Evaluation Week 2019 must have Select the wireless network connection
completed their registration online to named ‘VISITORS’ or ‘DELEGATES’ to
obtain an access pass to the UNON enjoy internet. You are, however,
Compound. UN staff attending UNEG requested to limit the number of
Evaluation Week 2019 with New York personal gadgets you connect to the
and Geneva issued blue UN access internet at the same time to ensure that
badges, can use these badges to access the connection is working efficiently.
the UNON compound. On the first
day of the conference, there could be
a bit of wait at the entrance, so we Catering


recommend you arrive in good time. Coffee and refreshments will be
For security reasons, the identification provided during coffee breaks.
badge must be worn and visible to Please note that the UNON Compound
UNON Security staff at all times while in does not use single use plastics.
the UNON Compound. Drinking water will be available, and
Entry to the UNON Compound is via participants will be provided with a
the UN Avenue – off Limuru Road. re-usable water container. There will be
The pedestrian access is through the water dispensers in or near the meeting
Pavilion, north of the main vehicle rooms.
access gates opposite the United States There are several caterers within the
Embassy. Vehicles can drop off their UNON Compound contracted to provide
passengers at a drive-in space in front food and beverages for breakfast,
of the gate. lunches as well as snacks during coffee
breaks for staff and visitors.
Main Cafeteria: located on the
upper level of the main concourse, is
Transportation available to participants from 12:00 to


Many hotels can assist you in obtaining 14:00PM for lunch. The cafeteria offers
transport. Some of the hotels provide a selection of first and second courses,
transport to the UNON compound. grill and salads.
If this is not available the following Amaica: located at the ‘Old Cafeteria’
companies can assist you: offers Eritrean/Ethiopian food and grills/
• Hilltop: +254 (0)20 272 3270 barbecued specials as well as a wide
array of Kenyan cuisine.
• Jaycab Taxis: +254 (0)20 721 0520 or
+254 733 750455/+254 723 239750 Delegate’s Lounge: located on the
lower level, next to Conference Room 1.
• Jatco: +254 (0)20 444 8162 or +254 733
The restaurant offers a selection of à la
701494/+254 722 648383
carte meals.
• Jimcab: +254 (0)20 712 2565 or +254
Coffee Stations: at the New Office
737 333222/+254 722 711001
Facility (UN Environment and UN-
• Kenatco: +254 (0)20 250 6790 or +254 Habitat blocks) offer salad bar, soup,
705 780011/+254 705 780016 wraps and sandwiches.
• Uber and Taxify apps also work well
and are reliable in and around Nairobi
Agenda for Professional Development Seminars
and Evaluation Practice Exchange Sessions

* Badge Collection Monday from 8-9am and Tuesday - Wednesday from 8-8:30am
Monday 13 May 2019 Professional Development Seminars
09:00 - 10:00 Opening session - Conference Room 3
10:00 - 10:45 Group Photo and Coffee break

PDS 2 - Evaluation PDS 4 - Use of

PDS 1 - Big data for PDS 5 - Revisiting the foundations
theories and PDS 3 - Approaches qualitative analysis
evaluation: How to of humanitarian evaluation:
approaches: Applications to impact evaluation software for
10:45 - 12:30 walk the walk Reflections on current practice and
in evaluation practice evaluations e.g.
future directions
Conference Room 8 NVIVO
Conference Room 10 Conference Room 11
Training Room 4 Training Room 6
12:30 - 13:30 Lunch
PDS 4 - Use of
PDS 2 - Evaluation PDS 5 - Revisiting the foundations
PDS 1 - Big data for qualitative analysis
theories and PDS 3 - Approaches of humanitarian evaluation:
evaluation: How to software for
approaches: Applications to impact evaluation Reflections on current practice and
13:30 - 15:30 walk the walk evaluations e.g.
in evaluation practice future directions
NVIVO
Conference Room 8
Conference Room 10 Conference Room 11
Training Room 4
Training Room 6
15:30 - 16:00 Coffee Break
PDS 2 - Evaluation PDS 4 - Use of
PDS 5 - Revisiting the foundations
PDS 1 - Big data for theories and PDS 3 - Approaches qualitative analysis
of humanitarian evaluation:
evaluation: How approaches: Applications to impact evaluation software for
16:00 - 17:30 Reflections on current practice and
to walk the walk in evaluation practice evaluations e.g.
future directions
Conference Room 10 Conference Room 8 NVIVO
Conference Room 11
Training Room 4 Training Room 6

Tuesday 14 May 2019 Evaluation Practice Exchange Sessions

EPE 2.1 We ask others to demonstrate impact: EPE 3.1 Expanding our toolkit of evaluation
EPE 1.1 Providing comprehensive
How can we demonstrate the impact of our methods: Examples from emerging UN
08:30 - 10:30 evidence without losing utility
evaluations? practice
Conference Room 8
Conference Room 10 Conference Room 11
10:30 - 11:00 Morning tea

EPE 2.2 Taking stock of evaluations towards

EPE 1.2 Coaching junior evaluation EPE 3.2 Making the best use of Theories of
achieving the vision of 2030 Agenda: Open theory
11:00 - 13:00 officers – sharing our experiences Change in evaluations
of change
Conference Room 8 Conference Room 11
Conference Room 10

13:00 - 15:00 Lunch

14:00 - 15:00 Lunch time seminar - Sustainable Development Goals: How prepared are we for evaluation of interventions? Conference Room 7
EPE 1.3 Best Practices for EPE 2.3 Is impartiality of decentralized evaluations
Recommendation follow up. Can ever possible? What are the mechanisms and EPE 3.3 How to go about the management of
15:00 - 17:00 we maximize use with the right features to ensure this? joint evaluations: A best response game
procedures? Conference Room 11
Conference Room 10 Conference Room 8

17:00 Welcome reception (UNON Delegates Lounge)

Wednesday 15 May 2019 Evaluation Practice Exchange Sessions

08:30 - 10:30 EPE 1.4 Whose evaluation is it EPE 2.4 Evaluations and audits: Experiences EPE 3.5 Evaluating policy support
anyway and why does it matter? from combined functions synergies or Conference Room 11
Let’s talk about country evaluations competition?
Conference Room 8 Conference Room 10
10:30 - 11:00 Coffee break
EPE 2.5 OECD DAC evaluation criteria: Recent developments and potential for evaluation in the context of UN reform, changes in UN
11:00 - 13:00 development system and independent system wide evaluation
Conference Room 3
13:00 - 14:00 Lunch
EPE 1.5 As managers, we also make
mistakes: Let’s share them and learn EPE 3.4 Low-cost, useful and good quality
14:00 - 16:00 from them   evaluations?
Conference Room 10 Conference Room 11
16:00 - 16:15 Coffee break
Wrap up session
16:15 - 17:45
Conference Room 3
19:00 Self-paid dinner