Engagement

Toolkit
Methods, tips and best practices
to design effective participatory
processes

Prepared by EFSA based on desk research undertaken by Prospex and iCONs

(outsourced activity)
Last updated: March 2021

European Food Safety Authority

Via Carlo Magno 1A – 43126 Parma, Italy
Tel. +39 0521 036 111│ www.efsa.europa.eu
Table of contents
The toolkit’s scope and objectives ...................................................................... 7
Introductory section .......................................................................................... 9
Glossary ............................................................................................................ 9
Recommendations for an effective engagement process .......................................... 9
Consider engagement as a continuous and circular process ................................... 9
Adopt a multi-level approach (EU and Local) ..................................................... 10
Some general principles to bear in mind ........................................................... 11
EFSA’s engagement model ............................................................................... 13
Why does EFSA engage, and with whom? ............................................................ 13
Three streams, one framework........................................................................... 14
The ‘Preparedness’ stream................................................................................. 14
The ‘Quality of science’ stream ........................................................................... 15
The ‘Stakeholder dialogue’ stream ...................................................................... 15
The inventory of engagement methods ............................................................ 16
Objectives ....................................................................................................... 16
Broadening pool of experts and stakeholders .................................................... 16
Introduce new and/or more agile engagement formats ....................................... 16
Enhance the quality of scientific outputs through engagement ............................. 16
Enhance the transparency of the risk assessment process................................... 16
Structure ......................................................................................................... 16
How to identify the right method(s)................................................................. 18
Purpose-based selection .................................................................................... 18
Stream-based selection ..................................................................................... 19
Recommended methods for the ‘Preparedness’ stream ....................................... 20
Recommended methods for the ‘Quality of science’ stream ................................. 20
Recommended methods for the ‘Stakeholder dialogue’ stream ............................ 22
ANNEX I - Engagement methods A to Z ............................................................ 24
Academic social networking sites ........................................................................ 24
After action review (AAR) .................................................................................. 25
Brainstorming .................................................................................................. 27
Buzz groups ..................................................................................................... 29
Challenge prize ................................................................................................ 31
Community based participatory research ............................................................. 33

2
Community of practice ...................................................................................... 35
Concept mapping ............................................................................................. 37
Consensus Conference ...................................................................................... 40
Crowdsourcing ................................................................................................. 42
Delphi Method/Technique .................................................................................. 43
Design Thinking, Project-in-a day ....................................................................... 45
Discussion forum .............................................................................................. 47
E-conference ................................................................................................... 48
Envisioning the future ....................................................................................... 50
Expert knowledge elicitation (EKE) ..................................................................... 52
Expert Interview with audience .......................................................................... 53
Focus group ..................................................................................................... 54
Gamestorming ................................................................................................. 56
Gamification .................................................................................................... 57
Group interview with a co-design session ............................................................ 58
Icebreakers and energizers ................................................................................ 60
Innovation jam ................................................................................................ 63
Interview......................................................................................................... 64
Most significant change ..................................................................................... 66
Online jam....................................................................................................... 69
Online Platform ................................................................................................ 71
Online Survey .................................................................................................. 72
Open innovation challenge ................................................................................. 74
Nominal group technique ................................................................................... 75
Participatory design .......................................................................................... 77
Participatory modelling...................................................................................... 79
Participatory sensing......................................................................................... 81
Participatory workshops .................................................................................... 82
Fishbowl ....................................................................................................... 83
Innovation Challenge ..................................................................................... 86
Knowledge Fair.............................................................................................. 87
Open space technology................................................................................... 89
Unconference ................................................................................................ 92
Six thinking hats............................................................................................ 94
World café .................................................................................................... 96

3
 Peer Assist ...................................................................................................... 99
PESTEL and SWOT combined ........................................................................... 102
Public consultations ........................................................................................ 104
Public events ................................................................................................. 104
Q-methodology stakeholder selection ................................................................ 106
Reflexive interactive design ............................................................................. 107
Research agenda camp ................................................................................... 109
Science Café .................................................................................................. 111
Science shops ................................................................................................ 113
Science week ................................................................................................. 114
Scoping study ................................................................................................ 116
Stakeholder working groups ............................................................................ 118
Swot Analysis ................................................................................................ 119
TOP 100 list ................................................................................................... 122
TOPSY TURVY (or Reverse Brainstorming) ......................................................... 124
User committee .............................................................................................. 126
VIPP Cards Collection Clustering ....................................................................... 128
Vision factory ................................................................................................. 130
ANNEX II - Best Practices from EU and International entities ........................ 134
Stakeholder Engagement – EU Agencies ............................................................ 134
European Union Aviation Safety Agency (EASA) ............................................... 134
European Chemicals Agency (ECHA) .............................................................. 134
EIP-AGRI .................................................................................................... 135
European Citizen Science Association (ECSA) .................................................. 136
European Cooperative Programme for Plant Genetic Resources (ECPGR) ............. 137
European Commission and the European Economic and Social Committee (EESC) 137
Stakeholder engagement – International Agencies.............................................. 138
The International Center for Tropical Agriculture (CIAT) ................................... 138
Biodiversity International .............................................................................. 139
Consultative Group for International Agricultural Research (CGIAR) ................... 139
Center for International Forestry Research (CIFOR) ......................................... 141
The Food and Agriculture Organization (FAO) .................................................. 142
United Nations (UN) ..................................................................................... 149
World Vegetable Center (WorldVeg) ............................................................... 154
World Fish Center ........................................................................................ 155

4
 World Agroforestry....................................................................................... 156
International Potato Center ........................................................................... 157
International Maize and Wheat Improvement Center (CIMMYT) ......................... 158
International Livestock Research Institute (ILRI) ............................................. 159
Global Investment Facility (GEF).................................................................... 160
Stakeholder Engagement – other organisations .................................................. 161
The University of Washington ........................................................................ 161
Danish Government ..................................................................................... 162
IBM – Innovation Jam .................................................................................. 163
“Wissenschaftsladen”, the The Bonn Science Shop - Germany ........................... 163
wer-weiss-was.de - Germany ........................................................................ 164
Gute Frage - Germany.................................................................................. 164
Wellcome – United Kingdom.......................................................................... 165
Expert optix – United Kingdom ...................................................................... 165
Aalto University - Finland.............................................................................. 166
Business Finland - Finland............................................................................. 166
Donau-Universität Krems und Bundesministerium für Kunst, Kultur, öffentlichen Dienst
und Sport - Austria ...................................................................................... 167
Impact Hub - Austria.................................................................................... 167
Science Shop Austria - Austria....................................................................... 167
Science Shop Vienna - Austria ....................................................................... 168
LISAvienna - Austria .................................................................................... 168
European Science Engagement Association (EUSEA) - Austria ........................... 169
RAND corporation - US ................................................................................. 170
Innocentive - US ......................................................................................... 170
Productivity Commission - Australian Government’s independent research and
advisory body ............................................................................................. 171
U.S. Department of Health & Human Services - National Institute on Minority Health
and Health Disparities .................................................................................. 172
U.S. Government ......................................................................................... 173
USAID United States Agency for International Development .............................. 173
Annex III | Target audience identification methods....................................... 175
Overview ....................................................................................................... 175
List of methods and their goals...................................................................... 175
Comparative table to weigh advantages and disadvantages .............................. 176
Description of the methods .............................................................................. 178

5
 Method for Transdisciplinary Research ............................................................ 178
Mapping stakeholders associated with societal challenges ................................. 179
Stakeholder identification in concept mapping ................................................. 181
Stakeholder Circle Methodology ..................................................................... 182
Prospex-CQI: Criteria-Quota-Individual .......................................................... 184
Pyramid Research ........................................................................................ 185
Identification Public-Public Partnerships .......................................................... 186
Stakeholder identification beyond classification................................................ 188
Mapping Stakeholders from Social Media ........................................................ 189
Smart Sheet Stakeholder Mapping ................................................................. 190
Expertise finding tools ..................................................................................... 192
Social Networking ........................................................................................ 192
Scientific literature ....................................................................................... 193
Knowledge Base .......................................................................................... 193
(Community) Threaded Discussions and Community Questions and Answers ....... 193

6
The toolkit’s scope and objectives
Participatory processes are widely used in all domains to increase the transparency and
legitimacy of policies. This happens worldwide and at all levels.
The regulatory framework is not an exception. EFSA operates at the interface between
policy makers and society and plays a pivotal role in ensuring the trustworthiness of the
whole system. Not by coincidence, the ‘Transparency Regulation’1 emphasizes public
engagement as ‘the’ method to build and maintain trust in the risk assessment process
underpinning EFSA’s scientific production.
Whether it is about involving civil society, stakeholders or partners, all want to make their
voice heard, valued and reflected into trackable actions or deliberations. Being trustworthy
requires taking this demand into consideration, irrespective of the audience addressed,
and believing in it as an opportunity to establish a win-win relation while sharing visions,
knowledge, expertise and outcomes.
Diversity is strength in engagement. It brings quality and quantity of ideas, but also
credibility and legitimacy. This requires opening internal processes to embrace inputs
coming from outside.
This document provides tips and resources to inspire the choice of the best solutions to
design effective participatory processes. It was prepared under EFSA’s ‘Relationship
Management Project’2 to be used by EFSA in enhancing its engagement efforts in line with
the provisions of the Transparency Regulation’.
At the same time, the document provides information as per point 5a of the EC request to
EFSA for provision of technical assistance in the field of risk communication (ref. EFSA-Q-
2020-00213) as follows: ‘Carry out a comprehensive mapping of all different types and
levels of engagement activities and the appropriate tools and channels depending on the
different target audiences; this mapping should provide an overview of
advantages/disadvantages of the different tools and channels taking into account the
relevant risk factors and include ‘best practices’ based on literature review and input from
existing research, where relevant’.
The document is structured as follows:
▪ An introductory section with a glossary providing definitions to clarify the
terminology used in the toolkit and general recommendations for an effective
implementation of the engagement methods presented in the Annexes.
▪ An outline of EFSA’s engagement framework to put things in context.
▪ An inventory of 50+ online and offline methods stemming from desk research
and capturing the main trends in participatory processes. The inventory is
complemented by an Excel version to quickly filter the methods according to
multiple criteria (Annex I).

1
Regulation (EU) 2019/1381 of the European Parliament and of the Council of 20 June 2019 on the transparency
and sustainability of the EU risk assessment in the food chain and amending Regulations (EC) No 178/2002, (EC)
No 1829/2003, (EC) No 1831/2003, (EC) No 2065/2003, (EC) No 1935/2004, (EC) No 1331/2008, (EC) No
1107/2009, (EU) 2015/2283 and Directive 2001/18/EC (OJ L 231, 6.9.2019, p. 1), to be found at: https://eur-
lex.euiODa.eu/legalcontent/EN/TXT/PPF/?urUCELEX:32019RI381&from=EN
2
An internal EFSA project set up to manage organisational changes required to implement new provisions of
Regulation (EU) 2019/1381. For more information:
https://www.efsa.europa.eu/en/stakeholders/transparency-regulation-implementation

7
 ▪ A description of 40+ best practices from EU and international organisations,
providing examples of how some methods of the inventory are successfully
implemented in different domains worldwide.
▪ A repository of target audience identification methodologies to increase
outreach beyond the existing audiences.
In conclusion, this document is a toolkit to design participatory processes to engage with
third parties (be them partners or stakeholders) in a mutually rewarding way. For target
audiences - to follow closely EFSA’s work and give their input; for EFSA - to support the
quality and legitimacy of our scientific outputs; for all - to build a trustworthy and
transparent relationship based on an open and fruitful dialogue.
While this document was prepared by EFSA, considering its own engagement
model, it provides a catalogue of methods and tools for a broader community of
practitioners operating at the interface between policy and society, which can
apply the listed methods when engaging in an open dialogue with interested
parties.

8
Introductory section

Glossary
This toolkit relies on the use of a precise terminology. The following definitions are
provided, being essential to establish a common vocabulary and ensure the understanding
of the key concepts presented in this guide to avoid misinterpretations.

Term Definition Examples

An activity aimed to engage an audience to
Method meet a specific objective. Possible synonyms: Event, public consultation
model, mechanism, format
Software used to gather
Tool A functional instrument to execute a method opinions during a workshop
(polling)
An opportunity to carry out a stakeholder
Engagement See The ‘Quality of science’
engagement method within a specific step of
window stream
the EFSA risk assessment process
A specific domain of EFSA’s engagement
Engagement See EFSA’s engagement
framework, with own audiences, methods and
stream model
underpinning processes

Recommendations for an effective engagement process

Consider engagement as a continuous and circular process

Irrespective of the domain in which it occurs, engagement is a process that does not end
with the conclusion of a single activity.
It is a circular process that grows out mutual exchanges, insights, feedback and frequency
of interactions. It is subject to and influenced by boundary conditions related to the place
where it is carried out (e.g. social, political and economic conditions; technology and
language) and it evolves over time.
EFSA’s Stakeholder Engagement Approach (SEA) aims to involve registered stakeholders
through a combination of permanent and targeted mechanisms serving different purposes
(Information, Consultation, Engagement). The approach is outlined and implemented as a
linear approach, while embedding circularity may better reflect the main objective of any
engagement activities: achieving greater value together, while building a trustful relation.
The following figure illustrates the benefits brought by conceiving engagement as a
continuous and circular process, growing in output quality while incorporating feedback.

9
Adopt a multi-level approach (EU and Local)
The adoption of a multi-level approach (EU and Local) is beneficial to raise awareness,
encourage participation, incorporate feedback, build a solid relation with stakeholders and,
ultimately, build trust in EFSA’s work.
EFSA’s Focal Points, for example, can play the role of local multipliers and interface of
EFSA towards local audiences while implementing local engagement activities, establishing
a tight connection between EFSA’s work and the local context. The implementation of a
multi-level approach will allow EFSA to reach both a wide audience and to narrow down
the purpose of local engagement activities, thus increasing outreach and participation as
well as building and nurturing communities of local experts.
The following figure outlines the multi-level approach just described.

10
Some general principles to bear in mind
The following key principles must be kept in mind when planning any engagement method.

Category Description
▪ Plan carefully. What is the scope of your mandate and what will it change
or achieve? How and when should stakeholders be involved? How are
opinions received, treated and processed? Is feedback given regularly and
transparently? How will data, inputs and opinions be managed? Experts
may have to overcome several barriers to bring value to the process. Make
it easy for them to navigate with clear signposts and robust processes.
▪ Be prepared to inject resources in the process. Building a dynamic
community requires energy. Shortcuts are few; successful engagement
Organisational requires resources, determination and managerial support.
and managerial
principles ▪ Ensure you have internal buy-in. Is there a consensus and
understanding of the long-term benefits of engaging with stakeholders
throughout the risk assessment process? Make sure everyone is on-board.

▪ Mix tools, methods and messages. One size does not fit all. Decide
how you want to engage. Cross-reference with your goals before deciding
on a mix of methods to achieve your objectives. An online method with
rich content and proactive facilitation may help outreach, but the more
vocal and interested members of the community may provide input more
effectively in a participatory workshop. Consider expectations and
differences when you plan your activities.
▪ Plan promotion: ensure timely access to information is key to ensure
that stakeholders can plan their participation. Make them aware of the
opportunities well in advance and let them know how they can contribute.
▪ Use established, recognized and trusted channels before
attempting to set up your own. For instance, if your objective is to
Communication engage with experts, leverage the channels they are already familiar with.
principles
▪ Consider any barriers that might exist in the community when planning
your communication activities. These could be linguistic or functional.
▪ Content is the King, Linking is the Queen. Content alone does not
guarantee success, especially if you are reaching out to a new audience.
Consider teaming up with other institutions targeting the same
stakeholders you are interested in to broaden your outreach.

▪ Put yourself in their shoes. Compromise, consensus and common

ground play and important role in engagement. Cultivate a deeper
understanding and appreciation of other perspectives. Methods like
simulation games and role-playing can build mutual understanding for
complementary action.
▪ Ensure transparency and regular feedback on how stakeholder input
has been considered in a process.
Relational
▪ Evaluate participation with statistics, feedback forms, exit interviews
principles
or surveys. Ensure a range of participant profiles complete to get a range
of opinions on the impact of the engagement method implemented and

11
Category Description
its appropriateness to achieve results and outcomes. Be prepared to
change your plans if something turns out to be ineffective.

▪ Co-design is a way to jointly produce a mutually valued outcome.

It is a fundamental change to the historical ‘producer and consumer’
relationship between, for instance, farmers and citizens. Co-design can
also be considered in the terms of an emerging Quintuple Helix - the active
flow of information and ideas among five sectors of society (government,
academia, business, non-profit organisations and citizens) – to tackle the
ongoing systemic change in society.
▪ What’s in it for me? Quality information, benefits, reputation,
recognition, reward – or just the pleasure of sharing knowledge or getting
Mutual benefit
something done. These are just some of the reasons why someone would
principles
engage, stay motivated to do so, and encourage others to do the same.
Tap into the aspirations and interests of your audience as well as your
practical constraints to be effective.

12
EFSA’s engagement model

Why does EFSA engage, and with whom?

To deliver its mandate, EFSA cannot work in isolation. EFSA regularly engages with third
parties for essentially three reasons:
▪ to ensure preparedness and to support the quality of its scientific production;
▪ to ensure relevance for both policymakers and society;
▪ to foster the sustainability of the risk assessment process through the
involvement of partners, becoming co-owners of its result.

EFSA belongs to a complex ecosystem of entities having in common a strong interest in

food safety. However, their degree of engagement with EFSA and the nature of the
interactions in place differ significantly from one audience group to the other. The same is
true for the channels and methods used to manage this relation.

The main focus of the toolkit is on stakeholders and partners, but the same methods
may be considered to engage with the general public and customers as well.

13
Three streams, one framework
EFSA’s engagement approach is an organic framework based on three streams of
activities, as outlined in the figure below:

Each stream has its own peculiarities and objectives. The three streams are underpinned
by different processes, but they are orchestrated by a common governance ensuring that
they work coherently for the achievement of an overarching goal: enhancing the openness,
transparency, inclusiveness and trustworthiness of EFSA’s operations by ensuring a fruitful
dialogue with third parties.
The streams require fit-for-purpose and effective engagement methods. This toolkit
provides insights on the most suitable models for each of the above domains.

The ‘Preparedness’ stream

EFSA operates at the interface between science, policy and society. Each domain is
evolving rapidly and imposes standing measures for EFSA to anticipate future scientific
trends or challenges, deploy effective social listening approaches and continue delivering
up to the expectations of policy makers and consumers. In one word, to stay relevant.
Relevance, however, can only be achieved by being open to inputs from outside. The
‘Preparedness’ stream relates to the involvement of third parties in EFSA’s forward-looking
reflections to detect and characterize new issues brought, for example, by new food/feed
production technologies, by emerging societal concerns and expectations, or by the
evolving regulatory framework.
This toolkit suggests a series of online and offline methods to support this process.

14
The ‘Quality of science’ stream
This stream is related to the engagement activities rolled out during the lifecycle of a
generic mandate or an application. The nature of interactions in this domain is mandate-
related and topic-based; third party input is functional to ensuring access to relevant data,
promoting the understanding of EFSA’s work in that domain and enhancing the
transparency of the risk assessment process leading to a scientific output.
During the lifecycle of an application or a generic mandate, there are specific windows for
public engagement to occur. The figure below outlines the different engagement windows
in the lifecycle of an application (upper stream) or a generic mandate (lower stream).

This toolkit classifies the methods according to the windows in which they may be deployed
(Pre-mandate, Plan, Do, Verify, Report).

The ‘Stakeholder dialogue’ stream

EFSA’s Stakeholder Engagement Approach (SEA) identifies the following permanent
mechanisms to engage registered stakeholders on a regular basis:
▪ The Stakeholder Bureau and the Annual Stakeholder Forum
▪ The Roundtables with NGO’s and industry organisations
This toolkit suggests several methods to complement the mechanisms in place to engage
stakeholders in this domain and ensure more agile and effective interactions.

15
The inventory of engagement methods

Objectives
The methods described in the Annexes aim to achieve the following common objectives:

Broadening pool of experts and stakeholders

Expansion of expertise is vital to bring in the necessary knowledge and get fresh insights
around specific matters while raising awareness about EFSA’s role with not-yet-engaged
audiences.

Introduce new and/or more agile engagement formats

The inventory includes a series of online methods complementing the existing engagement
models of the SEA with new online and offline formats to ensure more focused, agile and
sustainable interactions.

Enhance the quality of scientific outputs through engagement

Engagement is key to support the quality of scientific outputs by leveraging the expertise
and data available outside EFSA. Beside the methods already in use at EFSA – e.g. calls
for data, public consultations and crowdsourcing – the inventory proposes additional
models to collect external input or data during the lifecycle of a mandate or an application.

Enhance the transparency of the risk assessment process

The aim of the Transparency Regulation is to foster the openness and transparency of EU
risk assessment by introducing specific measures to ensure a high degree of public scrutiny
in all steps of this process. The inventory reinforces the portfolio of EFSA’s engagement
models with new methods to help achieve the transparency objective more effectively.

Structure
Each method in the inventory is described in an individual table based on the dimensions
below to help you identify the best match between your needs and the right engagement
models to fulfil them:

Table section Table fields and description

Method: name of the specific method.

Short description: providing a short description of the method.

Objective: goals that can be met by implementing the method.

General outline Target: target addressed.

Geographical scope: whether the method is suitable for EU and/or

Local engagement.
Online/Offline: whether the method is suitable for online and/or
offline execution.

16
Table section Table fields and description

Impact: expected impact of the methods implementation (Inform,

Consult, Involve).
Engagement stream: whether the method is suitable to be
implemented in the ‘Preparedness’ and/or ‘Quality of science’ and/or
‘Stakeholder dialogue’ streams.
Engagement window: in case the method is suitable for the ‘Quality
of science’ stream, this fields suggests the appropriate engagement
Window to which the method may be applied.
Detailed description: provision of a detailed description for the
method’s implementation steps and process.

Benefits: listing the benefits achievable through the method.

Risks: listing possible risks to be faced during its implementation.

Required tools: listing the functional tools supporting the method’s

execution.
Implementation
Timeframe: providing a tentative timeframe needed for
implementation.
Required skills/resources: listing the skills needed for its
implementation.

Useful links: references to external sources describing the method.

Examples of implementation: how others have used the method.

17
How to identify the right method(s)

Purpose-based selection
Engagement must be linked to a predefined purpose. The table below groups all methods
based on the goal they help achieve, no matter the context or engagement stream in
which they may be applied. Some methods may be suitable for more than one purpose.
The icon marks the offline methods; the icon indicates the online models. If both
icons appear, the method can be implemented in both environments.

Purpose Methods

Brainstorming Open space

Challenge prize Participatory workshop

Concept mapping Pestel analysis

Consensus conference Stakeholder forum

Design thinking, project in a day

TOP 100 list
Generate new
ideas Discussion forum TOPSY TURVY

Envisioning the future Vision factory

Innovation jam

Communicators Lab Online survey

Delphi method Peer assist

Discussion forum Public consultation

Q-methodology stakeholder
Discussion group
selection
Expert Interview with audience
Reflexive interactive design

Roundtable with NGOs and

Expert knowledge elicitation
Learn from and industry stakeholders
consult experts
Focus group Scientific colloquium
and stakeholders
Interview Scoping study

Knowledge fair User committee

Nominal group technique Webinar

Online platform

18
 Purpose Methods

Call for data Interview

Crowdsourcing Online platform

Gamification Online survey

Gather Data Expert knowledge elicitation Participatory sensing

Academic social networking sites

Online platform

Participation at stakeholders’
Gamification
events

Information session Science café

Inform
Knowledge fair Science week

Academic social networking sites

Nominal group technique

After Action Review Online platform

Communities of practice Open space

Discussion forum Participatory modelling

Learn and share
lessons within Focus group Participatory workshops
groups
Knowledge fair Science shop

Community based participatory

Participatory workshop
research

Crowdsourcing Science shop

Design thinking, project in a day

Reflexive interactive design

Group interview with a co-design Roundtable with NGOs and

session industry stakeholders
Co-design
Participatory design Vision factory

Participatory modelling

Stream-based selection
The following tables group the methods by engagement stream for a quick selection in
case the context in which engagement will occur is already defined.
The icon marks the offline methods; the icon indicates the online models. If both
icons appear, the method can be implemented in both environments.

19
Recommended methods for the ‘Preparedness’ stream

Online platform Focus group After Action Review

Discussion forum Challenge prize Brainstorming

Crowdsourcing Science week Most significant change

Innovation jam Interview Nominal group technique

Science café Open space Online survey

Academic social networking Stakeholder working group Expert Interview with

sites audience
Q-methodology stakeholder
Gamification Communicators Lab
selection
Expert knowledge elicitation
Participatory workshop Participatory design

Knowledge fair Participatory modelling Discussion group

Vision factory User committee Mandate working group

Scientific colloquium Scientific conference Delphi method

Concept mapping Visit of delegations Participatory sensing

Design thinking, project in a

Consensus conference SWOT analysis
day

Pestel analysis Envisioning the future TOPSY TURVY

Top 100 list Science shop

Recommended methods for the ‘Quality of science’ stream

Window Methods

Ad-hoc meetings with NGOs Participatory design

Brainstorming Participatory modelling

Challenge prize Participatory workshop

Pre-Mandate Community based participatory
Peer assist
research

Community of practice Pestel analysis

Concept mapping Public consultation

20
Window Methods

Q-methodology stakeholder
Discussion forum
selection

Delphi method Reflexive interactive design

Expert knowledge elicitation Scoping study

Group interview with a co-design

Participatory workshop
session
Roundtables with NGOs and
Interview
industry stakeholders

Nominal group technique Stakeholder working group

Online platform SWOT analysis

Online survey User committee

Ad-hoc meetings with NGOs Online survey

Call for data Participatory workshop

Community based participatory

Participatory design
research

Crowdsourcing Participatory sensing

Risk Discussion forum Peer assist

assessment
process Discussion group Public consultation
(‘Plan’, ‘Do’,
‘Verify’ windows) Roundtables with NGOs and
Expert knowledge elicitation
industry stakeholders

Focus group Stakeholder working group

Gamification Technical meeting

Interview User committee

Online platform

Academic social networking sites

Online platform

Participation to stakeholders’
After Action Review
Communication events
/ Dissemination
(‘Report’ window) Gamification Science café

Information session Webinar

Most significant change

21
Recommended methods for the ‘Stakeholder dialogue’ stream

Roundtables with NGOs and

Stakeholder bureau Stakeholder forum
industry stakeholders
Participatory workshop
Online survey Brainstorming

Online platform Participatory design Gamestorming

Group interview with co-
Focus group
design session

22
Annexes
ANNEX I - Engagement methods A to Z

Academic social networking sites

General outline
Method Academic social networking sites
Academic Social Networking Sites (ASNSs) offer new ways to
communicate, collaborate and gather knowledge. ASNSs bring
Short description
researchers and researches at one place. Flexibility in exchange of
ideas and open discussions lead to free flow of information.
• Connect researchers and make it easy for them to share and
access scientific output, knowledge and expertise.
Objective
• Disseminate results.
• Promote events or workshops.
Target Researchers, experts, academia
Geographical
Global
scope
Online/offline Online
Impacts Inform, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window (‘Quality
Report
of science’
stream only)
Implementation
Social networking is about linking people and organizations around
similar interests via internet-based tools. Most social networking
sites involve creating a user profile, inviting contacts and “friends” to
your personal network, creating short updates or blog posts, having
discussions and sharing resources. Established social networking
Detailed sites, such as Facebook and LinkedIn allow for the creation of
description groups. You can also create a subject-specific or group-specific
network using Ning. Access to an enlarged network of people makes
social networking a valuable advocacy and communication strategy.
It is also a great way to enable colleagues who work in different
offices and in different time zones to easily share knowledge and
experiences.
Benefits Potential to reach a very broad audience.
When posting on a social network you are publicly exposed to
Risks
potential critics (also from non-experts).

24
 ResearchGate is a European social networking site for scientists
and researchers to share papers, ask and answer questions, and find
collaborators.
LinkedIn allows members (both workers and employers) to create
profiles and "connections" to each other in an online social network
which may represent real-world professional relationships.
Yammer is a freemium enterprise social networking service used for
private communication within organizations. Access to a Yammer
network is determined by a user's Internet domain so that only
individuals with approved email addresses may join their respective
Required tools
networks.
Ning is an online service that allows users to create their own social
networks or online communities of practice, providing a more
“focused environment” in which to network, validate and build on
existing knowledge and good practices. Many online communities
have been set up for professional or interest groups where members
can network with one another in a more private space. They often
have full social networking functionality such as profiling, message
posting, discussion forums and online chat. Ning also allows you to
participate in other existing networks.
Timeframe Continuous
Required skills Setting up a social media strategy and building an online community
and resources requires time.
Today EFSA uses Researchgate to consider potential experts for the
Comments Working Groups and to advertise relevant activities and events. The
potential of this social network may be further explored.
https://www.researchgate.net/publication/280069078_Academic_So
Useful links
cial_Networking_Sites_What_They_Have_to_Offer_for_Researchers
https://reliefweb.int/sites/reliefweb.int/files/resources/FC4003368F0
Examples
378D6C12570180039867F-care-tsu-17may.pdf

After action review (AAR)

General outline
Method After action review (AAR)
The After Action Review (AAR) is a simple process to review a
project, an activity, an event or a task. In an AAR, the individuals
Short description involved discuss what happened, why it happened, what went well,
what needs improvement, and what lessons can be learned from the
experience with a view to doing as well or better next time.
Objective Review a project, an activity, an event or a task
Target Researchers, academia, risk assessors, farmers, consumers, NGOs
Geographical
European
scope
Online/offline Online and offline

25
Impacts Consult
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window (‘Quality
Report
of science’
stream only)
Implementation
Call the meeting soon after the event: AARs should be conducted
when the people involved in the experience are still available and
their memories are fresh.
Create a climate conducive to frank sharing: Trust, openness
and commitment to learning are key.
• Emphasize that AARs are learning events, not critiques.
• Everyone is regarded as an equal participant and group
members should not fall into junior/senior dynamics.
• Set ground rules.
• If possible, hold the AAR in a different location from the work
environment to create the right climate.
Appoint a facilitator: The right facilitator is a key factor in
contributing (or not) to the right climate for the AAR. Ideally this
should be someone with a good understanding of the issue (to keep
focus) and the AAR methodology, but who was not closely involved
in the project or activity, in order to be objective.
Ask the 4 key questions: a. What was supposed to happen? b.
What actually happened? c. Why were there differences between the
planned and the actual? d. What did we learn? After revisiting the
Detailed
objectives of the activity, it is always a good idea to start with the
description
positive points, i.e. “what went well out of what happened?” For each
point, keep asking “why?”
• For problematic areas, ask “what could have gone better?”
instead of “what went wrong?” Understanding “why” is
equally fundamental.
• Allow enough time for reflection.
• Encourage all members to contribute, if necessary by using
flip charts and colored cards.
• Probe answers before recording them as lessons learned or
good practices.
Record the AAR It is important to have a clear and well-
documented account of the AAR:
• The name of the activity and main data related to it, including
the names of the people involved in the activity and those
participating in the AAR, Lessons learned, Good practices,
Guidance and recommendations for the future, Key
documents related to the activity.
Share the learning Make the results of the AAR known to:
• Those involved in the activity

26
 • Others who can benefit from the learning (such as those
embarking on a similar activity)
• Management and other parts of the organization that could
take measures to redress or improve in areas that would
benefit future activities of a similar nature.
The application of AAR, over time, leads to new or revised
Benefits knowledge, new and more effective ways of working together, and
eventually to mastery within the context of the project
Avoid using AAR for evaluating performance or for assigning credit or
Risks
blame; to do so will likely kill the process.
Copies of the AAR framework with the four key questions.
Required tools Pens/pencils, a flipchart with the large AAR framework used by the
facilitator, Cards and sticky notes (optional).
The timeframe for an AAR depends on the event being reviewed.
While AARs should ideally be as short and concise as possible, more
Timeframe complex events will require more time. For example, while an AAR at
the end of a training course can last as little as half an hour, you
may need as much as a full day for a more complex activity Review.
Required skills
and resources
It is important to use AARs for key, recurrent and strategic activities
to ensure a constant flow of learning and improvement. It can be
conducted in person, on the telephone, or online with tools such as
instant messaging, teleconferencing, wikis and forums. Remember to
provide adequate incentives for participation in a forum.
This method could be used to learn lessons on the participatory
Comments
methods that have been put in place during the lifecycle of a
mandate (to run after an opinion is adopted). However, the reflection
would be on the process (i.e. if the engagement methods put in
place were successful or not), but not on the quality of the scientific
output itself. For example, it could be used to assess the success of a
pilot.
Useful links http://kstoolkit.org/After+Action+Review
https://reliefweb.int/sites/reliefweb.int/files/resources/FC4003368F0
Examples
378D6C12570180039867F-care-tsu-17may.pdf

Brainstorming
General outline
Method Brainstorming
Brainstorming is a technique to generate ideas. Participants are
asked to think and quickly generate ideas around a question,
Short description problem or opportunity. The only rules are "no idea is too crazy" and
"generate ideas, not critique." Without these barriers the oddest and
most unexpected ideas can lead the discussion to something very

27
 constructive. It is often used as a divergent process to stimulate
creativity and innovation and paired later with convergent processes
to cluster and evaluate the ideas.
Objective Collect ideas, encourage creativity, inspire innovation.
Target Researchers, NGO, academia, farmers, staff
Geographical
European, International
scope
Online/offline Offline
Impacts Inform, Involve
Engagement
‘Preparedness’, ‘Quality of science’, ‘Stakeholder dialogue’
stream(s)
Engagement
window (‘Quality
Pre-mandate
of science’
stream only)
Implementation
How to use:
• The facilitator sets out the two rules of brainstorming:
1. There are no bad or ideas that are too crazy. Let your
imagination lose. Sometimes the craziest idea is the seed of a
fabulous idea.
2. Offer only ideas, not judgment of the ideas. That comes
later. Judgement during brainstorming can stop the flow of
ideas and creativity.
• The facilitator asks the question or states the brainstorming
challenge.
• Participants speak out their ideas or write down their ideas to
share them later. If spoken aloud, a facilitator captures the
ideas on flip charts or white boards. Use of cards to write
Detailed
down individual ideas can be useful. Importantly, you let the
description
brainstorming take and follow its own course, without
directing it.
• When people stop offering ideas, let there be silence for a
minute or two. Sometimes there are a few more ideas that
come up out of that silence.
• Signal the end of brainstorming and, if appropriate, move on
to the sorting and evaluating of the ideas produced. The
ideas can be collated and processes through subsequent
methods such as grouping, and evaluating (ranking, voting,
etc.).
• This can also be a good point for reflection on the experience.
People are often exhilarated and surprised by the creativity
that was unleashed.
Benefits

28
 Some people feel that brainstorming can dilute ideas by moving too
Risks quickly. Notice if seeds of ideas need to be taken and nurtured after
the brainstorming.
• Plenty of cards (10 x 20 cm/ 4 x 8 inch) or post-it notes of
different colours, and marker pens, (for recording and
clustering ideas)
• Flipchart or laptop & projector (for displaying the
Required tools
Brainstorming question)
• Flipchart paper (placed on walls/boards) or pin boards, for
putting up post-its/cards. Coloured dots/stickers (optional,
for ranking exercise)
Creative processes are difficult to limit in time and require a certain
flexibility. Depending on the brainstorming technique you choose,
reflect on what timespan should be reasonable and where you can
Timeframe
build in flexibility. A brainwriting exercise can be done in 20 minutes,
whereas a Top 100 list can be mainstreamed into a two to three day
training activity
Required skills Facilitator and Rapporteur (optional for groups up to 10; required for
and resources larger groups)
Where people are reluctant to speak up, use written brainstorming,
Comments
asking people to write down one idea per card or post it note.
Useful links http://kstoolkit.org/Brainstorming
Examples

Buzz groups
General outline
Method Buzz Groups
Buzz Groups are a method for quickly and efficiently gathering
feedback on a topic or responding to a specific question during a
plenary (a session that includes all participants of an event).
Short Without moving from their seats, participants form mini-clusters of two
description or three people and engage in free discussion – or ‘buzz’ – for a few
minutes on a given question. Buzz Groups provide a welcome change
of pace for participants, helping to enliven and energize large group
meetings and events.
Buzz Groups can be applied to obtain participant feedback quickly, or
Objective to facilitate engagement with a topic, without breaking away from
plenary.
Target Researchers, users, industry, NGOs, CSOs
Geographical
Global, European, Local
scope
Online/offline Offline and online
Impacts Consult, Involve

29
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
There are three basic ways to apply Buzz Groups: to obtain feedback,
to engage participants with a topic, or to generate questions after a
presentation.
Instruct participants to form groups of either two or three with their
immediate neighbours, without moving from their seats. Prefer groups
of two for smaller sessions, and groups of three for larger ones. These
are the Buzz Groups. The point is to allow participants to communicate
with the person(s) next to them while keeping them seated in plenary.
Pose a specific question to obtain feedback. This may be a question
you devised previously, or one that has come up during the meeting.
Ask each group to discuss it and formulate their ideas, in a maximum
of 3 - 5 minutes.
When the 3 - 5 minutes is up, call a halt to the discussions and ask the
presenter of each Buzz Group to share their group’s response to the
question, in plenary, in one minute or less. If desired, record the
Detailed
feedback on cards or a flipchart sheet.
description
Variations:
Buzz Groups using card collection: Ask the Buzz Groups to discuss
a question and record their responses on cards (one card per
response). The cards are then collected and clustered based on
similarities. Use a pin board to group the similar cards together. A
collage of different idea clusters should emerge. This is especially
useful in groups of around 30 participants where feedback is expected.
Buzz Groups to define presentation content - for expert or fact-
finding presentations: Buzz Groups can be used to generate
questions to be answered during a presentation by an expert. This
allows participants to define the presentation content, thereby
increasing relevance and promoting engagement. Sufficient time
should be allowed for the expert to prepare the presentation;
depending on the expert and the issue, this could be minutes, hours or
even a day.

• Splitting people up into groups and moving them to different

locations for short discussions takes time and requires more
effort; Buzz Groups make short discussions easy and are
particularly useful when time is limited.
Benefits • Buzz Groups are useful as a follow-up to a presentation
(especially if the presentation is particularly complex). They
help to shift the session out of lecture mode, and they enable
participants to have quick discussions to check on facts, reflect
on what was presented, exchange ideas, and link

30
 ideas/concepts together. Buzz Groups can also help connect
one session to the next, during a multi-session event, by
posing leading questions related to the upcoming session.
• Buzz Groups can also be used as an icebreaker near the
beginning of a workshop. They are a useful way to encourage
quiet people to get involved in the discussion and contribute,
since they may be more comfortable talking in small groups.
Risks
Required tools Flip charts, paper hanging on the wall, a whiteboard
10 - 20 minutes (approx. 5 minutes ‘buzzing’, and 5 - 15 minutes to
Timeframe
share feedback in plenary)
Required skills
One or more facilitators
and resources
Comments
https://www.unicef.org/knowledge-
Useful links
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Examples

Challenge prize
General outline
Method Challenge prizes
Challenge prizes offer a reward to whoever can first or most effectively
meet a defined challenge. They act as an incentive for addressing a
Short
specific problem, rather than being a reward for past achievements. A
description
challenge prize can incentivise innovation, focus attention on a
particular issue and unlock financing and other resources.
• To solve big problems and, if successful, produce major
breakthroughs in human knowledge and practice.
• Raise awareness or encourage investment in a neglected issue
or problem.
Objective
• Encourage new collaborations and partnerships.
• Gather new information or data.
• Identify good ideas or excellent practices, and build capacity of
new innovators.
Target Researchers, industry, civil society, academia, CSOs
Geographical
EU, international, national
scope
Online/offline Online
Impacts Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)

31
Engagement
window
(‘Quality of Plan
science’ stream
only)
Implementation
Challenge prizes can be used to solve problems in almost any field.
The formula is simple: offer a financial reward for the first or best
solution to a problem, attract the best innovators, and give them the
support they need to compete.
Prizes specify a problem to be solved and incentivize solvers anywhere
to address the issue in whatever way they deem best. Published
criteria define what success looks like, without prejudging how it is
achieved.
Challenge prizes can incentivize new thinking and reward the best
solutions, wherever they come from, however they work. They go
beyond the usual suspects and reach innovators that other funding
mechanisms miss.
Challenge prizes can also bring together innovators and help them
thrive. Prizes help innovators by raising their profile, bringing them
into contact with expertise, investment and new customers. And
alongside the cash prize at the end, prizes typically have other support
too, such as seed funding, help with networking, mentoring, testing or
access to legal and marketing support.
Detailed
Finally, prizes can unlock systemic change. They don’t just create
description
solutions to a narrow technical problem. They can raise awareness of a
broader issue with the public, and they can shape policy and inform
regulators. Done right, they can create whole new technologies and
markets, and shape them in a socially beneficial way.
Challenge prizes are particularly suited to solving problems that share
some key characteristics:
• Problems that are defined well enough so that a clear and
unambiguous goal for innovators can be set.
• Problems that would benefit from the fresh thinking that comes
from new innovators; for instance, because the field is
stagnant, has few players, or there is a related field that is
much more dynamic.
• Problems where a prize could attract new innovators to address
them, within a reasonable budget and timescale.
• Problems where the additional funding and attention the prize
would bring would plausibly accelerate progress (and not just
fund what’s already happening).
• Problems where the solution could thrive in the market (or find
continued funding) after the prize is awarded.
Challenge prizes can help funders maximise value and manage risk
because resources are allocated to competitors who deliver innovation.
Benefits
Challenge prizes stimulate and support new ideas and new
people/groups to become active problem solvers.

32
 • Setting up a prize often requires a significant amount of
research in order to identify the right challenge. Failure to set a
suitable end goal is likely to fundamentally undermine the
effectiveness of the challenge.
• The competitive nature of challenge prizes may not be best
suited to complex societal issues. Stimulating enough attention
around the challenge to encourage individuals or teams to carry
the risks associated with working towards an uncertain reward.
• Understanding why a challenging issue has not been met is
crucial since a prize may not resolve the deep systemic barriers
to innovation.
Risks
• Narrowly defined challenges may risk excluding more
unpredictable solutions. Therefore, the problem and solution
must be defined appropriately or left open in a way that allows
for unpredictable effective solutions to emerge.
• Crowdsourcing exercises to define ‘challenges’, rather than just
crowd-sourcing ‘solutions’, may lead to social innovations that
are human focused rather than technology focused.
• Challenge prizes tend to be technology/product innovation
focused rather than social innovation focused. Some recent
challenges have focussed on complex social/environmental
problems.
Required tools
Challenge prizes are likely to take around a year to set up and could
Timeframe
take several years to complete.
Required skills Intermediate skills are required in: Subject-matter expertise,
and resources IT skills, Event organisation skills, Project management skills.
Nesta is an innovation foundation and can organize it
Comments https://www.nesta.org.uk/
It could be used for innovation initiatives (similar to hackathons)
NESTA challenge prizes practical guide
Useful links https://www.nesta.org.uk/toolkit/challenge-prizes-a-practice-guide/
https://challenges.org/
Examples United States Agency for International Development

Community based participatory research

General outline
Method Community based participatory research
The community is involved in all stages of the research process, from
setting the questions, to framing and doing the research, interpreting
Short the results and communication. Research is focused on better
description understanding and then improving a certain situation. If combined with
actions to implement findings, this leads to a cycle of participatory
action research.
• Improving intervention design and implementation by facilitating
Objective participant recruitment and retention.
• Increasing the quality and validity of research.

33
 • Enhancing the relevance and use of data.
• Increasing trust and bridging cultural gaps between partners.
• Providing resources for the communities involved.
Target NGOs, researchers, academia, risk assessors, consumers
Geographical
Regional, Local
scope
Online/offline Offline
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Plan, Do, Verify, Report
science’ stream
only)
Implementation
Community-based participatory research (CBPR) is a "collaborative
approach to research that equitably involves all partners in the research
process and recognizes the unique strengths that each brings. CBPR
begins with a research topic of importance to the community, has the
aim of combining knowledge with action and achieving social change to
improve health outcomes and eliminate health disparities."

Detailed
description

The US National Institutes of Health give the following advantages of

community-based participatory research:
Benefits
• Joining partners with diverse expertise to address complex
public health problems.

34
 • Improving intervention design and implementation by facilitating
participant recruitment and retention.
• Increasing the quality and validity of research.
• Enhancing the relevance and use of data.
• Increasing trust and bridging cultural gaps between partners.
• Providing resources for the communities involved.
• Benefiting the community and researchers alike through the
knowledge gained and actions taken.
The potential to translate research findings to guide the development of
further interventions and policy change.
Risks
Required tools
From 1 month upon to continuous, depending on already established
Timeframe
partnerships.
Required skills Requires intermediate subject-matter expertise and advanced
and resources facilitation and project management skills.
This method combines elements of Science Shops (Civil Society Driven
Research) and Participatory Action Research, and Citizen Science as
well. Projects can be part of larger themes of continuous attention (i.e.
Comments
programmes). Students and research institutes can also be part of the
research groups. Community-researchers can be trained as well.

http://accelerate.ucsf.edu/files/CE/manual_for_researchers_agencies.p
Useful links
df
Examples

Community of practice
General outline
Method Community of practice
Communities of practice (CoPs) are groups of people who share a
specific area of work, interest, or passion, in a knowledge domain. CoPs
share knowledge, learn from each other and interact regularly with
peers in an enabling networked environment.
Short Organizations use CoPs to share knowledge based on specific themes
description and areas, to break down organizational silos, and stimulate
interdisciplinary teamwork. CoPs are driven by the willing participation
of their members (principle of self-selection) and are focused on
learning, sharing knowledge and strengthening capacity. Engaged in
developing expertise and solving problems.
• Sharing and learning about practices in which colleagues are
interested
Objective • Encouraging professional development and accelerating learning
• Responding more rapidly to stakeholders’ needs and inquiries
• Learning by doing (action learning)

35
 • Identifying, documenting and sharing best practices
• Collective problem-solving
• Reducing duplication and avoiding re-inventing the wheel
• Connecting “islands of knowledge”, fostering cross-functional
and cross divisional collaboration.
Target Researchers, academia, students, NGOs
Geographical
European, International
scope
Online/offline Online/Offline
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
What makes a community of practice succeed depends on the purpose
and objective of the community as well as the interests and resources
of the members of that community. Wenger, who first proposed the
method, identified seven actions that could be taken in order to
cultivate communities of practice:
1. Design the community to evolve naturally – Because the nature
of a community of practice is dynamic, in that the interests,
goals, and members are subject to change, CoP forums should
be designed to support shifts in focus.
2. Create opportunities for open dialog within and with outside
perspectives – While the members and their knowledge are the
CoP's most valuable resource, it is also beneficial to look outside
of the CoP to understand the different possibilities for achieving
Detailed their learning goals.
description 3. Welcome and allow different levels of participation – Wenger
identifies 3 main levels of participation. 1) The core group who
participate intensely in the community through discussions and
projects. This group typically takes on leadership roles in guiding
the group 2) The active group who attend and participate
regularly, but not to the level of the leaders. 3) The peripheral
group who, while they are passive participants in the
community, still learn from their level of involvement. Wenger
notes the third group typically represents the majority of the
community.
4. Develop both public and private community spaces – While CoPs
typically operate in public spaces where all members share,
discuss and explore ideas, they should also offer private
exchanges. Different members of the CoP could coordinate

36
 relationships among members and resources in an individualized
approach based on specific needs.
5. Focus on the value of the community – CoPs should create
opportunities for participants to explicitly discuss the value and
productivity of their participation in the group.
6. Combine familiarity and excitement – CoPs should offer the
expected learning opportunities as part of their structure, and
opportunities for members to shape their learning experience
together by brainstorming and examining the conventional and
radical wisdom related to their topic.
7. Find and nurture a regular rhythm for the community – CoPs
should coordinate a thriving cycle of activities and events that
allow for the members to regularly meet, reflect, and evolve.
The rhythm, or pace, should maintain an anticipated level of
engagement to sustain the vibrancy of the community, yet not
be so fast-paced that it becomes unwieldy and overwhelming in
its intensity (Wenger, McDermott & Snyder 2002).
Members of communities of practice are thought to be more efficient
Benefits
and effective conduits of information and experiences.
Risks
Technologies can support communities of practice. They can range from
the simple use of a mailing list to the more expanded use of online
Required tools social networks, combing discussion forums, blogs and wikis. The use of
technology needs to support the overall goal of the community of
practice and must bring added value.
Timeframe
Required skills
and resources
Comments Already in place at EFSA
Useful links https://en.wikipedia.org/wiki/Community_of_practice
Examples

Concept mapping
General outline
Method Concept mapping
Concept mapping is a structured process, focused on a topic or
construct of interest, involving input from one or more participants, that
produces an interpretable pictorial view (concept map) of their ideas
and concepts and how these are interrelated. Mapping is based on
Short
multivariable statistical analyses in which statements produced during a
description
brainstorming session are grouped in weighted clusters. A mind-map,
also known as a concept map, is a diagram that depicts connections
between organizations, people, concepts, ideas, tasks or events. These
elements can be represented through many types of media, including

37
 text, images and video. The elements are generally connected to each
other visually through arrows, often labelled with text.
This is an approach particularly designed for facilitating consensus in
Objective
the understanding and organization for various concepts.
Target Researchers, industry, NGOs
Geographical
Local, European
scope
Online/offline Offline/online
Impacts Consult, Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan
science’ stream
only)
Implementation
Concept mapping is a general method that can be used to help any
individual or group to describe their ideas about some topic in a pictorial
form. There are several different types of methods that all currently go
by names like “concept mapping”, “mental mapping” or “concept
webbing.” All of them are similar in that they result in a picture of
someone’s ideas. It is primarily a group process and so it is especially
well-suited for situations where teams or groups of stakeholders have
to work together.
A concept mapping process involves six steps that can take place in a
single day or can be spread out over weeks or months depending on the
situation.
The first step is the Preparation Step. There are three things done
here. The facilitator of the mapping process works with the initiator(s)
(i.e., whoever requests the process initially) to identify who the
Detailed
participants will be. A mapping process can have hundreds of
description
stakeholders participating, or relatively small groups of between 10 and
20 stakeholders involved. Second, the initiator works with the
stakeholders to develop the focus for the project. In the Generation
Step the stakeholders develop a large set of statements that address
the focus. For instance, they might generate statements that describe
the specific activities that will constitute a specific social program. Or,
they might generate statements describing specific outcomes that might
occur as a result of participating in a program. A wide variety of
methods can be used to accomplish this including traditional
brainstorming, brainwriting, nominal group techniques, focus groups,
qualitative text analysis, and so on. The group can generate up to 200
statements in a concept mapping project.
In the Structuring Step 1. each participant sorts the statements into
piles of similar ones and 2. rates each of the statements on some scale.

38
 Usually the statements are rated on a 1-to-5 scale for their relative
importance. The Representation Step is where the analysis is done.
The statistical analysis – multidimensional scaling – takes the sort data
across all participants and develops the basic map where each
statement is a point on the map and statements that were piled
together by more people are closer to each other on the map. The
second analysis – cluster analysis – takes the output of the
multidimensional scaling (the point map) and partitions the map into
groups of statements or ideas, into clusters. In the fifth step –
the Interpretation Step – the facilitator works with the stakeholder
group to help them develop their own labels and interpretations for the
various maps. Finally, the Utilization Step involves using the maps to
help address the original focus. On the program side, the maps can be
used as a visual framework for operationalizing the program. on the
outcome side, they can be used as the basis for developing measures
and displaying results.
• Concept mapping helps people to think more effectively as a
group without losing their individuality. It helps groups to
manage the complexity of their ideas without trivializing them or
losing detail.
• Mind-maps are useful for conceptually breaking down ideas to
illustrate links and connections between people, concepts, ideas
and organizations. These links become clearer when they are
Benefits
visualized.
• In a problem solving context, mind-maps can be drawn to
describe four types of “concepts”: problem description, causes,
effects and solutions.
• Mind-maps can be used to provide structure to argument.
• In meetings, mind-maps are useful for graphically representing
and structuring the results of brainstorming activities.
Risks
Flip chart and markers for the note-taker. Tables, chairs. There are
many online/downloadable tools that allow for the
Required tools creation of mind-maps or concept maps, such as: http://cmap.ihmc.us/
http://freemind.sourceforge.net/wiki/index.php/Main_Page
http://www.mindomo.com, http://www.mindmeister.com
Timeframe 4 hours – 1 day
Required skills
Skilled facilitator
and resources
Comments Alternative workshop format
Useful links https://socialresearchmethods.net/kb/concept-mapping/
The National Center for Biotechnology Information
Examples
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5451901/

39
Consensus Conference
General outline
Method Consensus Conference
The purpose of the consensus conference is to enrich and expand a
Short debate on a socially controversial topic. A group of citizens rather than
description experts and politicians get together and set the agenda and the basis
for assessment within a problem area.
The consensus conference aims to give citizens a meaningful
opportunity to influence on policy decisions and assessing issues
Objective relevant for society. It has also been used for social experiments,
research projects and as a means for promoting social awareness and
public debate.
Target Consumers, citizens, researchers, academia, NGOs, risk assessors
Geographical
National, Regional, Local
scope
Online/offline Offline
Impacts Consult, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
Participants are selected from a group of citizens who are invited to
apply. Individuals who are invited are members of the lay public that
have no specific knowledge of the issue at hand. This citizens’ panel is
chosen to be demographically representative of the public.
Members of the citizens’ panel participate in 2 preparatory weekends
and are given material prepared by a communicator to gain a basic
Detailed
understanding of the issue at hand. The panel then participates in a 4-
description
day conference. Over the duration of the conference, the citizens’ panel
participates in a Q&A session with experts, where they get opposing
views. Citizens then prepare a final document containing their views,
opinions, stances, and recommendations for the issue. On the final day
of the conference, the panel then discusses their final document with
policy- and decision-makers.

• The consensus conference gives an opportunity to hear citizen

voices – a large group of society who are usually not asked
about their view on a specific problem. The inclusion can reduce
Benefits
democratic deficit of the citizens and give them ownership of the
process and a sense of living in a successful democracy as they
act advisers to the politicians.

40
 • The consensus reached by the citizens’ panel contributes to
politicians, experts and society as a whole on the ideas and
concerns of ordinary citizens.
• As it often is experts and policy makers who sets the agenda,
the consensus conference allows for ordinary citizens to have a
say and influence the debate.
• The voice of the citizens reflects views and concerns that
politicians don’t necessarily see.
• The citizens’ panel make recommendations with awareness and
knowledge and this can influence the policy-making process in a
new way. This opens for a more comprehensive decision-
making.
• The consensus conference is well suited for a new topic early in
the development process to frame the debate. It can help shape
a problem area that is not yet widely discussed by different
parties especially at the political level. It is also suitable for
topics in need of new inputs, development or a new agenda.

• The recommendations can’t be used if the development or

application of the technology or problem is not an object of
political decision making. The consensus conference is most
suited to topics which do not have a clear policy option.
• The media may focus on the disagreements rather than the
agreements.
• The eternal criticism: can the recommendations formed by 10-
Risks
30 citizens be regarded as the general opinions of the entire
population? Using random stratified sampling can create a group
that is demographically representative of a population. The
results of the consensus conference will not be the only form of
evidence that decision makers use.
• The consensus conference does not match a problem area that
is too far in the development process.
Required tools
The process requires 12 months of preparation. Event duration: 3-4 day
Timeframe
long
Subject-matter expertise: Intermediate
IT skills: Basic
Required skills
Facilitation skills: Advanced
and resources
Event organisation skills: Advanced
Project management skills: Advanced
The citizen’s panel is made up of interested citizens. Here we describe
how the participants of panels for consensus conferences are selected
by the Danish Board of Technology (the Danish consensus conference
model). Participants are advertised for in 5-6 regional newspapers that
Comments
provide comprehensive geographical coverage of Denmark, or in a
number of national newspapers. Persons interested in participating are
invited to apply to the Secretariat by submitting a short (no more than
one page) description of themselves, the knowledge they have on the

41
 topic and the reason why they wish to participate. On the basis of the
submitted applications, a panel of 10-14 lay people is selected.
Basically, the panel members are unpaid volunteers but compensation
for loss of income is offered by the Board.
The panel is selected so that it is composed of people with varied
backgrounds based on a number of socio-demographic criteria: age,
gender, education, occupation and area of residence.
It is essential that no member of the lay panel is an expert in the topic
or represents special interests in the field. However, when individual lay
people are particularly concerned about the topic being debated, they
are deemed to have a special interest that is acceptable. This was the
case, for instance, in the conference on infertility (October 1993) where
many of the lay-panel applicants (65 people, i.e. 49 per cent) suffered
from involuntary infertility. The steering committee included two of
these lay people in the panel, because they were considered to be
affected by the topic as lay people - not as representatives of special
interest groups per se.
Finally, physical and mental disabilities may be impediments to
participation in the project (the conference and preparatory phases are
very hard work and extremely concentrated).
The selection procedure does not ensure that the panel comprises a
statistically representative sample of the population, but the panel is
selected from interested people in such a way that several attitudes are
represented. Although applicants for the lay panel are not explicitly
asked to reveal their attitudes, these are usually apparent from their
applications.
https://en.wikipedia.org/wiki/Consensus_conference
Useful links
http://people.ucalgary.ca/~pubconf/Education/grundahl.htm
Examples

Crowdsourcing
General outline
Method Crowdsourcing
Crowdsourcing is a sourcing model in which individuals or organizations
obtain goods and services, including ideas and finances, from a large,
Short relatively open and often rapidly evolving group of internet users; it
description divides work between participants to achieve a cumulative result. The
word crowdsourcing itself is a portmanteau of crowd and outsourcing,
and was coined in 2006.
Engaging a diverse and broad spectrum of people whose skills, talents,
Objective
and knowledge are key to solving problems and driving innovation.
Target Civil society, researchers, farmers
Geographical
Global
scope
Online/offline Online

42
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Do
science’ stream
only)
Implementation
As from 2018, EFSA has been testing crowdsourcing as a
complementary method of engaging stakeholders and the public in the
process of scientific assessment. Crowdsourcing offers opportunities for
widening EFSA’s evidence base, promoting data sharing and re-use,
accessing new expertise, encouraging public enthusiasm and support
for science, and – ultimately - increasing transparency and trust in
Detailed science.
description The feasibility study and testing phase concluded that crowdsourcing
would bring value in the following areas: data collection, validation and
sharing; innovation; systematic literature reviews; identification of
experts; consultation and co-creation.
So far, two innovation contests have been launched on an open source
platform with the aim to apply this method in the risk assessment
process more systematically.

• Technology allows to crowdsource large pools of participants,

Benefits which means a large dataset and more in-depth data analysis.
• Vast range of users provide huge diversity in their experiences.
Risks
Required tools Use of a crowdsourcing platform like Innocentive.
Timeframe
Required skills
and resources
Comments Already in place at EFSA
Useful links https://www.innocentive.com/

Examples U.S. Government

Delphi Method/Technique
General outline
Method Delphi Method/Technique
The Delphi method is a multiple iteration survey method that enables
anonymous, systematic refinement of expert opinion with the aim of
Short
arriving at a combined or consensual position. Its purpose is to
description
generate discussion and enable a judgement on a specified topic to be
made so that policy decisions can be taken which can claim to represent

43
 a given group's wants and views. The Delphi technique uses a series of
consecutive questionnaires to determine the perceptions of a group of
individuals. The Delphi method allows respondents to communicate
their opinions anonymously. Each questionnaire is considered a round.
The method is often used to prioritize research/topics.
Generate discussion and enable a judgement on a specified topic to be
Objective made so that policy decisions can be taken which can claim to represent
a given group's wants and views.
Target Researchers, CSOs
Geographical
EU, international, national, regional, local
scope
Online/offline Offline/online
Impacts Consult
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Plan
science’ stream
only)
Implementation
There are four features which characterize the Delphi method and
distinguish it from other group decision-making processes. They are
anonymity, iteration with controlled feedback, statistical group response
and expert input. The Delphi method is presented as an alternative to
Detailed the committee process or the one-off questionnaire as a means of
description obtaining a group's opinion or judgement on a topic. It is often seen as
having an important contribution to make in facilitating controlled and
rational group communication. The method has been extensively used
for exploring policy issues and facilitating decision making by business
organisations and government agencies, as well as foresight studies.
The method offers several benefits, such as the use of an expert panel,
controlled anonymous feedback, and development of consensus. The
Benefits
anonymous feedback supports interdependent feedback not based on
personal differences or hierarchies between the involved experts.
The Delphi method is an alternative to the committee process, or one-
off questionnaire, although its ability to produce a convergence and
Risks consensus on a given topic should be viewed with caution. The Delphi
method takes more time, than a survey. It requires a lot of resources (a
high number of experts involved in the panel, and is time-consuming).
Required tools
The Delphi method needs a one year time frame at a minimum, (1-2
Timeframe years). There are different phases including: survey, data analysis and
feedback (next survey round). Before the process begins it is not

44
 possible to predict how many rounds the whole process of the Delphi
would need.
Subject-matter expertise: Advanced
IT skills: Basic
Required skills
Facilitation skills: Advanced
and resources
Event organisation skills: Basic
Project management skills: Intermediate
Already in place. This is one of the expert knowledge elicitation (EKE)
Comments
techniques
https://www.rand.org/pardee/pubs/futures_method/delphi.html
Useful links
https://en.wikipedia.org/wiki/Delphi_method
Examples

Design Thinking, Project-in-a day

General outline
Method Design Thinking, Project-in-a day
Design Thinking (DT) is a design methodology that provides a solution-
based approach to solving problems. It is extremely useful in tackling
complex problems that are ill-defined or unknown, by understanding
the human needs involved, by re-framing the problem in human-centric
ways. This is prosecuted by stimulating the creation of innovative ideas
in brainstorming sessions and by adopting a hands-on approach in
Short
prototyping and testing through the five stage of Design Thinking. It is
description
a general approach, that could be applied to many occasions and will
empower anyone to apply the Design Thinking methods in order to
solve complex problems that occur around us — in companies, public
administration, urban scenarios. This method is very interesting when
we are looking for “rapid prototyping” and “learning by making” as a
strategy for doing effective innovation.
The objective is to stimulate the unlock forms of value, not available
clearly at the beginning of the process and to help create a sort of
Objective “multiplier effects”. Hence, we also need to design processes that allow
us to spot new patterns, encourage the evolution of new ideas, and help
new ideas scale to the point where they have impact.
Target Researchers, Academia, Industry, Consumers, NGOs, Citizens
Geographical
Local, national, European
scope
Online/offline Offline
Impacts Inform, Consult, Involve
Engagement
‘Quality of science’
stream(s)
Engagement
Pre-mandate
window

45
(‘Quality of
science’ stream
only)
Implementation
The aim of this engagement method is to stimulate the unlock forms of
value, not available clearly at the beginning of the process and to help
to create a sort of “multiplier effects”. Hence, we also need to design
processes that allow us to spot new patterns, encourage the evolution
of new ideas, and help new ideas scale to the point where they have
impact.
The five stages of Design Thinking are:
1. Empathizing: Understanding the human needs involved.
2. Defining: Re-framing and defining the problem in human-centric
ways.
3. Ideating: Creating many ideas in ideation sessions.
4. Prototyping: Adopting a hands-on approach in prototyping.
Detailed
5. Testing: Developing a prototype/solution to the problem.
description
The design thinking could be implemented through laboratories that
involve directly participants. 25-40 participants per laboratory is a good
number that enable, one single expert facilitator to manage the
participants in groups of 4-5 persons.
It is important to define with accuracy the participants to the lab,
maybe splitting it in more round, but in general it is a process that
involve not more than 40-50 persons, from begin to end. The lab is not
strictly closed, so everyone could propose participants to the lab, and in
general they are welcomed always, even if the process is already in
phase 2 or 3.
Participants need to be invited in the first event and then must be kept
in the loop for the entire process.
This methodology is very useful to figuring out, how to make
information available, more understandable and clearer for all
participants. This method is very interesting when we are looking for
“rapid prototyping” and “learning by making” as a strategy for doing
effective innovation.
This method is also very appropriate as a participatory method, because
Benefits
the complexity of the interactions cannot possibly be anticipated by
even the smartest of plans, it is important to make plan and test them
in participatory ways, letting others participate in the innovation
activities.
A design mindset is not problem-focused, it’s solution-focused and
action-oriented.
If not managed well, it can result in failure. If not sufficiently motivate,
Risks it might result difficult to keep the commitment of the participants over
different phases
Beside to the expert facilitator and to some assistant to the facilitator (1
each 2 groups, in order to be supportive) it is necessary to have an
Required tools
enough flexible large room available:
• Table and chairs, not fixed

46
 • Several large sheet of white paper
• 1 Whiteboard for the facilitator
• As much as possible blocks of Post-it of several colours, color
markers (big top)
• ActionCam/HiFi-Cam to acquire groups activity (and enable
participants dynamics analysis offline)
• Double check if it is possible to attach to the wall room surface
posters and printed materials
The timeline is typically of 2-3 months, in order to separate the
Timeframe
different phases.
Required skills
The role of the facilitator is key and a skilled person is needed.
and resources
This method could work in some specific reflexions involving
Comments stakeholders (i.e. strategy definition) or to facilitate reflections during
the early stages of the RA process (problem formulation)
https://www.user-participation.eu/planning-the-process/step-5-
participatory-methods/planning-the-future-visions-strategies-
Useful links projects/project-in-a-day-method-description
https://www.interaction-design.org/literature/article/5-stages-in-the-
design-thinking-process
Project in a day tested in Trento, Kosice, and Vas County
https://www.user-participation.eu/planning-the-process/step-5-
Examples participatory-methods/planning-the-future-visions-strategies-
projects/project-in-a-day-method-description/showcase-project-in-a-
day

Discussion forum
General outline
Method Discussion Forum
A discussion forum is a virtual place on the internet where
conversations can take place and information can be shared more easily
Short
among a geographically dispersed group of people. Discussion forums
description
are typically created around a specific topic of common interest or for a
specific user group around a piece of work.
• Conversations supporting a global community of practice.
• Holding a week long asynchronous online meeting in a web forum.
• Carrying out a peer assist with colleagues around the world.
• Informal places to create and nurture relationships. Structured or
informal training and learning groups, especially where conversation
Objective
is useful.
• Project coordination and teamwork.
• Informal information and knowledge sharing.
• Asynchronous meetings as an alternative to face-to-face meetings
and conference calls.
Target Researchers, experts, academia, farmers, NGOs

47
Geographical
Global
scope
Online/offline Online
Impacts Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify
science’ stream
only)
Implementation
There are many ways to use discussion forums, from formal structured
Detailed
conversations to informal “cafés”. The difference between them is the
description
focus and duration of the conversation.
• When people are in different places and time zones, making
synchronous interactions more difficult, discussion forums can be
useful.
• When people are working in a second language and the slower pace
Benefits of a web-based discussion allows more time to make meaning
across languages.
• When it is important to know who said what and when they said it,
because the discussion forum lists who made a post and when they
posted it. This is especially useful when trying to track project work.
Risks
Required tools Discussing group software: Dgroups, Yahoogroups, Googlegroups
Timeframe
Required skills
and resources
Comments Like other methods already in place at EFSA
https://www.unicef.org/knowledge-
Useful links
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Examples

E-conference
General outline
Method e-Conference
An e-conference is a temporary online forum on a specific topic. E-
conferences are typically carefully planned out, have clear time frames
Short
and focus around specific topics. E-conferencing is usually done via the
description
Web. There is the possibility for server-based e-conferencing as well.
E-conferencing can also take the form of audio and/or video

48
 conversations, message swapping, file sharing and other forms of
electronic interaction. All these aim at simulating the experience of
being in the same room. E-conferencing can happen in real time, with
everyone interacting at once, which is called ‘synchronous
conferencing”. It delivers live streaming audio and video from the
multiple participants of the conference.
E-conferencing can be used for business meetings, educational sessions
Objective
or other types of events.
Researchers (organizers), policy makers, CSOs, users, industry,
Target
consumers
Geographical
EU, international
scope
Online/offline Online
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify
science’ stream
only)
Implementation
The main steps when organizing an e-conference are:
• Choosing the relevant topic.
• Choosing a “digital venue” (the platform/software to be used).
• Choosing time spot which is suitable to the different
participants.
Detailed
• Appointing the discussion chair person.
description • Contacting speakers and participants.
• Promoting the e-conference.
After the e-conference, a final synthesis document which discusses and
summarises the major themes and findings of the conference discussion
should be delivered.
• Participants can be located all over the globe.
• It is cheaper to participate in e-conference than to attend a
Benefits
meeting.
• Synergy with face-to-face activities.
• Vulnerable to technical breakdowns;
• If the internet connection is not good, key speakers, presenters
Risks
or guests may get disconnected from the conference, missing or
taking with them valuable information.
Required tools
Timeframe Depending on the project needs the time line may vary.
Required skills Subject-matter expertise: Intermediate
and resources IT skills: Advanced

49
 Facilitation skills: Advanced
Event organisation skills: Advanced
Project management skills: Basic
Comments Already in place at EFSA
Useful links http://actioncatalogue.eu/method/7408
Examples The Food and Agriculture Organization - FAO

Envisioning the future

General outline
Method Envisioning the future
A scenario-building method that invites collective reflection about
Short plausible futures. It works by imagining a time in the future (three to
description six years ahead) and assumes that the organization, section or field
presence has achieved important goals.
• To think about the future.
• To develop a vision of where you want to go.
• To explore alternative solutions for the future.
Objective
• To motivate and inspire a team and create cohesion around
common goals.
• Particularly suited for retreats on planning and/or team building.
Target Risk assessors, researchers, academia
Geographical
Local, European
scope
Online/offline Offline
Impacts Inform, Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
Decide which organizational entity will be the focus of the exercise. It
can be the entire office, a field presence, a section, a cross-division
team, or a branch. Normally, this will depend on who is participating in
the exercise.
Detailed
Agree on when in the future the envisioning exercise will take place.
description
Normally, it should be more than one year ahead — otherwise the
future will simply match with the completion of the annual workplan,
which would defeat the “vision” element of the activity. Conversely,
setting a timeframe too far in the future could take away the element of

50
 reality which is important to maintain. If the time chosen is 20 years
down the road, for instance, participants will tend to think in more
utopian terms and disconnect the vision from the operational steps to
achieve it. If the purpose is to emphasize the vision element, a five- to
six-year timeframe would be appropriate. If the emphasis is more on
medium-term goals and actions and how to achieve them, a shorter
timeframe of two to three years would be best.
Announce the positive assumption about the future. This can simply be:
“Imagine that in four years’ time your team will have achieved
important goals”. It can also be more specific, for instance: “Imagine
that in four years’ time: “the treaty bodies system will be considerably
improved”.
The positive assumption is accompanied by a number of questions
aimed at eliciting the elements and details to explain what was
achieved, why and how.
Divide participants into groups of four to six people to answer the
questions.
After the group work, re-convene in plenary, where each group
presents their future vision. A competition element can be introduced in
which participants are asked to vote for the presentation that convinced
them the most. If the exercise is in the form of a competition, this
should be announced from the outset so that each group can prepare to
convince and captivate the plenary with a well-organized and colourful
presentation.
Alternatively, the plenary can build a common vision with the
contributions from the different groups through a consensus process. In
this case, the envisioned future will be the result of a fully collective
process, with more ownership from the entire group.
Benefits
Plan in advance, otherwise the future will simply match with the
Risks completion of the annual workplan, which would defeat the “vision”
element of the activity
Flip charts for each group
Required tools Instructions for each participant
Markers and pencils.
Timing (1 hour 30 minutes to 3 hours). This example covers 2 hours:
• Instructions (10 minutes)
• Individual preparation time (20 minutes)
Timeframe
• Group preparation time (1hour)
• Group presentation time (20 minutes)
• Voting on the best proposal (10 minutes)
Required skills
Skilled facilitators
and resources
Alternative workshop format. Interesting for strategy development
Comments
purposes and for emerging risks (forward-looking approach)
Useful links http://acnudh.org/wp-content/uploads/2011/11/slitoolkit.pdf

51
Examples

Expert knowledge elicitation (EKE)

General outline
Method Expert knowledge elicitation (EKE)
Expert knowledge elicitation is a method to elicit from
knowledgeable experts quantitative parameters and their
uncertainties in a probabilistic way. Experts can be asked for specific
Short information (facts, data, sources, requirements, etc.) or for
description judgements about things (preferences, utilities, probabilities,
estimates, etc.). Currently, this method is used for mandate-related
engagement, although its potential could be worth exploring for
Engagement for preparedness, too.
Objective Online
Target Consult
Geographical
Already in place at EFSA
scope
Online/offline Offline
Impacts Inform, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify
science’
stream only)
Implementation
Detailed
description
Benefits
Risks
Required tools
Timeframe
Required skills
and resources
Comments
Useful links
Examples

52
Expert Interview with audience
General outline
Method Expert Interview with audience
The Expert Interview is a participatory question and answer session in
which members of the audience take the lead in asking a subject matter
expert questions that are oriented towards their needs and interests,
Short within the context of his/her expertise. Instead of the expert giving a
description presentation on what s/he thinks people want to know, this tool lets
participants decide on the questions they consider important. The tool
can be used to ‘interview’ up to three experts and can accommodate
any number of participants.

• To introduce a topic or theme at the beginning of a workshop/

multi-session event.
• To provide an alternative to a formal podium presentation or
keynote with experts.
Objective
• To elicit knowledge from resource persons without requiring
them to do a lot of preparation.
• To introduce new concepts or potentially ‘hot topics’ that
participants may be curious about but have little knowledge of.
Target Researchers, academia, practitioners, users, industry, NGOs, CSOs
Geographical
Global, European, local
scope
Online/offline Offline
Impacts Inform, Consult
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
The Expert Interview is perfect for sharing knowledge and experiences
from a few experts in a dynamic, engaging and participatory way.
There are no presentations. Instead, participants are free to ask any
relevant questions within the designated topic (based on the area of
expertise of the experts). Experts (who may be academics,
Detailed practitioners, or both) respond directly and can have conversations with
description participants. Spontaneous questions from participants can often elicit
more, and more relevant, knowledge from experts than one-way
presentations in which the expert tries to anticipate what s/he thinks
the audience wants to know. The open layout of the Expert Interview
helps make it less intimidating, and more conducive to participation,
than a panel.

53
 Don’t use more than 3 experts. Otherwise the Q&A may become
tediously long, and the session will resemble an ordinary panel
discussion. There is no limit to the number of participants
This is a great way to get subject matter experts to share their
Benefits
knowledge in a less traditional setting.
Risks
Microphones: One per expert, plus 1-2 for participants asking questions
Open space or room, large enough to accommodate participants
Chairs (see ‘Set up the room’ below)
Required tools Pin board (optional)
Flipchart sheets and marker pens (optional)
Laptop computer (optional)
LCD projector (optional)
Timeframe 60 – 90 minutes
Required skills Facilitator
and resources Rapporteur
The Expert Interview is similar to the Fishbowl. Key differences include
that the physical setup for the Expert Interview keeps the experts at
the front of the room, and the Expert Interview does not allow experts
Comments
to leave the discussion and be replaced.
The Expert Interview is the face-to-face equivalent of the Online Jam in
its Ask Me Anything version.
https://www.unicef.org/knowledge-
Useful links exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
http://kstoolkit.org/Expert+Interview
Examples

Focus group
General outline
Method Focus group
The focus group is a qualitative method which is used to determine the
preferences of people or to evaluate strategies and concepts
A qualitative method which provides information on attitudes,
perceptions and opinions of participants, obtained through an open
discussion led by a facilitator. Such purposive sample, rather than
Short
a statistically representative sample of a broader population, elicits
description
more in-depth views on a specific topic, which would be more
challenging to obtain through a quantitative method (e.g. survey).
The method has originally been designed for market research.
Participants are selected according to certain characteristics in common
that relate to the research topic and are grouped into 8-10 people.
Generate or evaluate hypotheses and ideas in conjunction with a
Objective
quantitative method, or as a primary data-collection method.

54
Target Researchers, NGOs, Industry, Consumers, Citizens
Geographical
European, National, Regional
scope
Online/offline Offline
Impacts Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Plan
science’ stream
only)
Implementation
The focus group is a method is designed to help learning more about
community and groups' preferences and opinions. The questions
participants are asked are typically qualitative and open-ended,
therefore the information is open to interpretation. The answers have
depth, nuance, and variety. Group dynamics, interaction and non-verbal
communication need also be observed. The focus groups can reveal
what the participants are really thinking and feeling, even though their
responses may be harder to score on a scale.

The 3 main characteristics of the focus groups are:

• The group focuses on a specific topic.
Detailed
• There is a facilitator (or trained leader) and his/her job is to
description
keep the group focused on discussing the specific topic.
• There is some careful planning behind the group's composition
and the group discussion in order to create an environment in
which people feel free to talk openly. Some members of the
group may need to be encouraged by the facilitator to express
their opinions.
The focus groups are structured and directed, yet, allowing for the free
expressions of opinions by the participants, they can gather a lot of in-
depth information in a relatively short time. The method is often used
to generate or evaluate hypotheses and ideas. At the end, the
information gathered in the discussion should be summarized in writing.
More interactive environment and better flow of ideas than in individual
interviews. This method can produce deeper insights on the
Benefits participants’ attitudes, ideas and preferences than other methods as it
allows for direct observation of the participants’ immediate reactions as
well as more in-depth discussions on the research topic.
Due to the small number of participants, the results are not
Risks
representative for the target group.
Required tools
Timeframe Sessions should last around 1 - 2 hours.

55
Required skills Requires advanced subject-matter expertise and advanced facilitation
and resources skills.
Currently already used by EFSA for social research. Potentially the use
Comments
of this method could be expanded.
https://ctb.ku.edu/en/table-of-contents/assessment/assessing-
Useful links
community-needs-and-resources/conduct-focus-groups/main
Examples Examples on how this tool is used by EU agencies: EIP-AGRI

Gamestorming
General outline
Method Gamestorming
A game may be thought of as an alternative to the standard business
Short
meeting. A game suspends some of the usual protocols of life and
description
replaces them with a new set of rules for interaction.
Gamestorming can be used to achieve diverse objectives:
• For fresh thinking and new ideas
• For teambuilding
Objective • For vision and strategic meetings
• For problem-solving
• For update and review meetings
For decision making
Internal staff, researchers, NGOs, farmers and primary producers,
Target
distributors, practitioners, consumers
Geographical
Local
scope
Online/offline Offline
Impacts Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
If you are a knowledge worker, you must become, to some degree,
creative. This may sound a bit scary, but the fact is that successful
creative people tend to employ simple strategies and practices to get
Detailed
where they want to go. It is more like a workshop with a set of tools
description
and strategies for examining things deeply, for exploring new ideas, for
performing experiments and testing hypotheses, to generate new and
surprising insights and results.

56
 Different games, grouped according to the game objective, can be
found here: https://gamestorming.com/
Benefits Involves stakeholders in a funny way
A quite innovative approach to creativity is needed; this may not always
Risks
be accepted in scientific processes
Games may require sticky notes, poster paper, markers, random
Required tools
pictures from magazines, or provoking objects.
Most games involve 3 to 20 people and last from 15 minutes to an hour
Timeframe
and a half.
Gamestorming skills include asking questions (opening, navigating,
examining, experimenting, closing), structuring large diagrams,
Required skills
sketching ideas, fusing words and pictures into visual language, and
and resources
most importantly, improvising to choose and lead a suitable game or
invent a new one.
Comments Could be a format for the Annual Forum of the SEA
Book: Gamestorming: A Playbook For Innovators, Rulebreakers, And
Useful links
Changemakers by Dave Gray https://gamestorming.com/about/
Examples

Gamification
General outline
Method Gamification
Short Gamification is the application of game-design elements and game
description principles in non-game contexts.
Gamification can be used for education purposes, behavioural change
Objective
purposes, crowdsourcing (for example Foldit game)
Target Researchers, academia, civil society
Geographical
Global
scope
Online/offline Online and Offline
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Do, Report
science’ stream
only)

57
Implementation
Early gamification strategies use rewards for players who accomplish
desired tasks or competition to engage players. Types of rewards
include points, achievement badges or levels, the filling of a progress
bar, or providing the user with virtual currency. Making the rewards for
Detailed accomplishing tasks visible to other players or providing leader boards
description are ways of encouraging players to compete.
Another approach to gamification is to make existing tasks feel more
like games. Some techniques used in this approach include adding
meaningful choice, onboarding with a tutorial, increasing challenge, and
adding narrative.

• Increase engagement
• Gather more data about users for further analysis and use
Benefits • Use of both, internal and external motivation
• Real time feedback
• Building relationships with users
Through gamification's growing adoption and its nature as a data
aggregator, multiple legal restrictions may apply to gamification. Some
refer to the use of virtual currencies and virtual assets, data privacy
laws and data protection, or labour laws. The use of virtual currencies,
Risks in contrast to traditional payment systems, is not regulated. The legal
uncertainty surrounding the virtual currency schemes might constitute a
challenge for public authorities, as these schemes can be used by
criminals, fraudsters and money launderers to perform their illegal
activities.
Required tools Gamification software (if online) or materials
Timeframe
Required skills
IT advanced skills required
and resources
Comments https://en.wikipedia.org/wiki/Gamification
Book: Gamestorming: A Playbook For Innovators, Rulebreakers, And
Useful links
Changemakers by Dave Gray https://gamestorming.com/about/
Examples on how this tool is used by European agencies: Center for
International Forestry Research - CIFOR
Examples on how this tool is used by European agencies: The
Examples
International Center for Tropical Agriculture - CIAT
Examples on how this tool is used by other organizations: The
University of Washington

Group interview with a co-design session

General outline
Method Group interview with a co-design session
Short The group interview with a co-design session will provide feedback
description about the research scenarios presented. The proposed group interview

58
 can be held within one single event or several smaller successive
events. Overall, we recommend to engage at least 35 people in this
consultation round. 5-8 is the minimum number of participants for one
event (if several ones), and to fit around one table. Participants can be
participants who would have been involved at a previous step - if any -
and new ones, we suggest a balanced setting, in which at least half of
the participants are original citizens. We recommend to over-recruit and
plan for more than 30 participants to be there, if you are looking to
engage at least 30 (in order to consider the no-shows). This method
requires to have some steps of the process online instead of offline.
• Enrich research scenarios
Objective • Get feedback to the research scenarios and the opportunity to deepen
it through the discussion
Target Civil society, researchers, academia, policy makers, risk assessors
Geographical
EU, international, national, local
scope
Online/offline Online/offline
Impacts Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Report
science’ stream
only)
Implementation
This method aims at:
• Stimulating the discourse and prompting citizens towards
enriching the following parts of the Scenarios: state of the art,
research questions/direction and expected impact.
Detailed
• Shaping and enriching this content with regard to the call format
description
you are aiming for a policy consultation: specific challenge,
scope and expected impact.
• Capturing the citizens’ views on the scenarios in the most
authentic way possible.
• Relative simplicity of the method.
• The idea of combining participants, who have already
participated at an earlier stage of scenario building with new
ones on the other hand brings the feeling of continuity and on
Benefits
the other hand – freshness and new ideas.
• People feel comfortable and work with big interest in their
preferred and chosen scenarios.
• It is not an exhausting activity.

59
 • The results were generated by different groups, so we could see
the real picture of what is really interesting for our citizens and
what opinion they have.
• The method by itself makes the process active, so it was more
interesting for citizens.
• A workshop will be a lot of work for your team and for
participants: it needs to be as interesting as it can be.
• Limited interactions among the tables: thus participants at the
last stage tend to choose the scenario they worked on plus
another one, hardly ever considering to choose 2 scenarios they
did not worked on.
Risks • Results of the prioritization of enriched scenarios are influenced
at least by two factors: 1) the presentation skills of a person
describing the enriched scenario (not surprising), 2) the fact
that some selected scenarios could cover covered similar topics.
• Difficult to make sure that unpopular scenarios are not left
behind by the participants and that every scenarios as a good
chance to go through the entire process.
Required tools
It is a 6 hours process. You can do an alternative process, which last
Timeframe
only for 3 hours.
Subject-matter expertise: Basic
IT skills: Advanced
Required skills
Facilitation skills: Intermediate
and resources
Event organisation skills: Intermediate
Project management skills: Intermediate
It could be applied at the beginning of the scientific process (i.e.
Comments framing of the question) and at the end (i.e. communication and
dissemination)
Useful links http://actioncatalogue.eu/method/7447#title_skills_required
This method was applied by the EU funded project CIMULACT
Examples
http://www.cimulact.eu/

Icebreakers and energizers

General outline
Method Icebreakers and energizers
Icebreakers help establish connections quickly and informally. This
sense of connectedness and free communication will help the group
achieve the work-related or learning objectives of the event.
Short Icebreakers are especially useful when participants are from diverse
description cultural, ethnic or organizational backgrounds. When used correctly,
icebreakers can also energize the group, highlight participants’
particular strengths and subtly introduce themes to be explored later in
the event.

60
 The main objective is to facilitate interpersonal exchange, establish
Objective connections quickly and informally and foster participation to make an
engagement method more effective
Target Researchers, Users, Industry, NGOs, CSOs
Geographical
Global, European, Local
scope
Online/offline Offline and online
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
Every event or workshop has participants, but often they don’t
participate actively enough. Sometimes this is because the event has
too many presentations, with ‘participants’ being put into the role of
passive listeners for the most part. But sometimes an event may have
plenty of opportunities for participation, but it may start off on the
wrong foot with a lengthy speech or ‘keynote’. This can have a negative
effect on the level of participation. Experienced event facilitators have
observed that the first couple of sessions often set the tone for the
remainder of an event; if these sessions are non-participatory, then a
non- participatory tone is set for the event, and participants will usually
behave accordingly. Below a short-list of icebreakers:

Speed Dating:
The Speed Dating (or Speed Networking) icebreaker is best used in
Detailed
meetings or workshops where most people don’t know each other. It
description
enables each meeting participant to meet a large number of other
participants in a relatively short time. A structured format helps
eliminate awkwardness: Participants prepare 1-minute ‘elevator
speeches’ about themselves, then exchange introductions through a
simple rotation process.
• Requirements: Facilitator, room with large empty space (no
chairs or tables), alarm clock/ bell / other audible signal, A4
paper and pens for participants, 20 - 30 participants
• Timeframe: 30 - 40 minutes

Four Quadrants:
An active exercise that keeps people on their feet, Four Quadrants is
best used in large groups as an initial icebreaker, a means of gradually
introducing event themes and topics, or as an after-lunch energizer.

61
It can be difficult to mobilize the energy of a large group of participants
who don’t know each other well, especially when you don’t have much
time in your event schedule. Four Quadrants is ideal for these
situations. By allowing participants to group themselves according to
their responses to simple ideas or statements, it gives them an
opportunity to identify with others in an easy way, even if they come
from very different backgrounds.
Normally Four Quadrants begins with participants responding to
common leisure or work activities. Participants must choose from
among four standard responses, ranging from very positive to negative,
hence the ‘Four Quadrants’. By progressing gradually to statements
about workshop themes or topics, these can be introduced in a subtle
way, and gauge participant interest levels and degree of experience.
The dynamic nature of Four Quadrants also makes it a great energizer
before embarking on long sessions.
• Requirements: Facilitator, 4 flipchart sheets, 30 or more
participants; room: ideally empty, or with plenty of room for
people to move around and gather in corners.
• Timeframe: 30 minutes

Walking billboard:
This is an informal way to open a meeting. Participants prepare and
wear a personal ‘billboard’: a sheet of flipchart paper with their name
and answers to interesting questions about themselves. They then
mingle for a short time; the billboards help catalyze conversations.
The Walking Billboard is best used at the beginning of a meeting or
workshop. Participants can converse with previously unknown persons
more easily thanks to the personal responses (to previously selected,
common questions) on the billboards. This exercise strikes a note of
informality and, with the paper billboard covering participants’ outfits
and name tags, helps even out implied hierarchies and cultural barriers.
Pre-determined questions: Use this method when time is limited.
Instead of asking participants to come up with questions, the facilitator
provides 3 - 5 questions at the start of the activity. Participants prepare
their billboards right away, and then mingle and converse.
• Requirements: facilitator, flipchart paper (one sheet per
participant), colour marker pens, 15- 40 participants
• Timeframe: 30 minutes

Mistaken identity:
This is a quick and easy way for participants to get to know one other at
the beginning of an event. When they first enter the meeting room at
the start of the day, participants receive a name card with someone
else’s name on it, and are asked to mingle and find that person.
Use Mistaken Identity when you want a quick icebreaker that does not
need much preparation. It can be used effectively in groups of up to
around 50 people. As people walk around looking for the person whose
name card they are holding, expect them to meet other people and
make connections very quickly, with little effort.

62
 • Requirements: facilitator, large name cards with participant
names prewritten in large letters, 20- 50 participants
• Timeframe: 10- 15 minutes
Benefits
Risks
Required tools
Timeframe
Required skills
and resources
Comments
https://www.unicef.org/knowledge-
Useful links
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Examples

Innovation jam
General outline
Method Innovation Jam
Massive online discussion that enables a diverse set of participants to
put forward innovative ideas (related to a broad topic) and also to build
Short upon each other’s ideas. Given the deluge of ideas, subject-matter
description experts and moderators as well as technology (e.g., text-analysis tools)
must be employed to channel the idea generation and bring coherency
to the discussions.
Objective Identifying innovative and promising “Big Ideas”
Target Researchers, academia, NGOs, consumers, staff
Geographical
Global
scope
Online/offline Online
Impacts Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
Using Web sites, wikis, forums and other online tools, Jam participants
Detailed
generate new ideas. From those ideas, the organizer focuses on several
description
major topics for the second part of the Jam and invited participants to

63
 build on the ideas within those topics. As a result of this process, 10
distinct ideas are implemented.
Facilitates the sharing of different perspectives of the problem, the
Benefits
contribution of ideas, and building upon each other’s ideas are needed.
It holds the risk of attracting a large set of seemingly conflicting ideas
Risks
that don’t lead to a workable solution in a timely manner.
Required tools Online platform to host the jam
Timeframe 2-3 days
Required skills
Requires advance IT skills.
and resources
Comments IBM https://www.ibm.com/products/innovation-jam
Theme analysis and qualitative research must be conducted after the
jam to identify major themes and insights, as well as the most
Useful links
promising ideas and concepts. Useful for reflections on AI and other
innovation areas.
Examples on how this tool is used by International agencies: UNITED
NATIONS
Examples
Examples on how this tool is used by others: IBM – Innovation Jam,
Danish Government

Interview
General outline
Method Interview
Interviews are used to explore the views, experiences, beliefs and
motivations of individuals on specific matters. Interviews as a
qualitative method are believed to provide a more in-depth
understanding of a certain topic than would be obtained from purely
quantitative methods (for example questionnaires). Interviews are,
Short
therefore, most appropriate where:
description
• little is known about the phenomenon under investigation;
• detailed insights are required from individual participants.
In addition, they are appropriate for exploring sensitive topics, where
participants may not want to talk about such issues in a group
environment.
Interviews can be used to explore the views, normative positions,
Objective
experiences, beliefs and motivations of an individual participant.
Target Researchers, academia, policy makers, NGOs, industry, consumers
Geographical
European, international, national, local
scope
Online/offline Online/offline
Impacts Consult

64
Engagement
‘Preparedness’, ‘Quality of Science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do
science’ stream
only)
Implementation
Interviews are a standard way for all kinds of journalists (print, radio,
TV and web, as well as freelancers) to collect information. You might
also want to conduct interviews yourself with staff members or
stakeholders to gather material for recordings or publications.
Interviews are normally one-on-one, with a single reporter posing
questions to one person. But similar techniques apply to other
situations, such as group interviews, panel discussions, debates, and
question-and-answer sessions during news conferences.
Interviewing techniques:
• Structured interviews – a list of predetermined questions is
asked. There is little or no variation in the questions. There is no
scope for follow-up questions to responses. This type of
interview is quick and easy to conduct. However, it is hard to
collect deep answers through structured interviews.
• Semi-structured interviews (SSI). This is guided
interviewing and listening in which only some of the questions
and topics are predetermined; other questions arise during the
interview. The interviews appear informal and conversational,
but are actually carefully controlled and structured. Using a
Detailed
guide or checklist, the multidisciplinary team poses open-ended
description
questions and probes topics as they arise. New avenues of
questioning are pursued as the interview develops. SSIs are a
central part of all participatory methods.
• Unstructured – this interview typically starts with an open
question and then develops according to the response given. It
can be difficult to manage, and to participate in, as the lack of
predetermined interview questions provides little guidance on
what to talk about which many participants find confusing and
unhelpful. However, being the most explorative type,
unstructured interviews might prove the best option when
“depth” is needed.
• Types, sequencing, and chains of interviews. Many types of
interviews may be combined in sequences and chains. These
include key informant interviews, by asking who the experts are
and then putting together a series of interviews; and group
interviews, which may be groups convened to discuss a
particular topic (focused or specialist groups), groups comprising
a mix of people whose different perceptions illuminate an issue
(structured groups), casual groups, and community groups.

65
 • Useful to obtain detailed information about personal feelings,
perceptions and opinions.
• More detailed questions can be asked.
Benefits • High response rate.
• Ambiguities can be clarified and incomplete answers followed
up.
• Interviewees are not influenced by others in the group.
• Face-to-face interviews can be time-consuming and costly. If
available resources are limited, telephone/Skype interviews can
Risks be done instead;
• Different interviewers may understand and transcribe interviews
in different ways.
Required tools
Time should be allocated for preparing questions for structured or semi-
structured interviews. Preparation time can vary greatly depending on
the complexity of the topic and the level of expertise of the interviewer.
The length of an interview varies as well. One interview typically lasts
from around 30 minutes to 2 hours. The length of the interview
Timeframe
depends on the complexity of the topic, on the complexity and number
of questions asked, on the specific circumstances. Personal interviews
would typically last longer than telephone/skype interviews. After the
interview, time should be allocated for transcribing the interview and
analysing the results.
Required skills
Requires basic subject matter expertise and basic facilitation skills.
and resources
Comments Already in place at EFSA for social research
https://www.academia.edu/746649/Methods_of_data_collection_in_qua
Useful links
litative_research_interviews_and_focus_groups
The Food and Agriculture Organization - FAO,
Examples
http://www.fao.org/3/W5830E/w5830e08.htm

Most significant change

General outline
Method Most significant change
Most Significant Change is a qualitative and participatory method for
monitoring and evaluation. It helps you to monitor and evaluate the
Short performance of your activities through the participatory collection of
description stories that reflect significant change resulting from activities. The
method was originally invented by Rick Davies and has undergone
several adaptations.
• To evaluate an activity, project or programme.
Objective • To build ownership among the stakeholders of a project.
• To share visions and values.

66
 • To facilitate a dynamic dialogue when working with diverse
outcomes and multiple stakeholders.
• To capture “hard to capture” data about changes in hearts and
minds.
Target Researchers, NGOs, CSOs, risk assessors, academia
Geographical
European, International
scope
Online/offline Online/offline
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
The Most Significant Change (MSC) technique, invented by Rick Davies,
is a form of participatory monitoring and evaluation. It involves the
collection and selection of stories of change, produced by programme or
project stakeholders. MSC can be used in projects and programmes
where it is not possible to precisely predict desired changes beforehand,
and is therefore difficult to set pre-defined indicators of change.
MSC is normally used as an ongoing monitoring technique, assessing
change throughout the lifetime of a programme or project. However, its
focus on change (outcome and impact) means it can easily be adapted
for use in evaluations as well. MSC is most useful where:
• it is not possible to predict in any detail, or with any certainty,
what the outcome of a project or programme will be;
• outcomes vary widely across beneficiaries;
Detailed
• there is no agreement between stakeholders on which outcomes
description
are the most important;
• interventions are expected to be highly participatory.
As with any Monitoring and Evaluation (M&E) methodology, MSC can be
used to help plan future activities. However, it is not a planning tool,
and is only normally used within a project or programme once enough
time has elapsed for change to have occurred.
The Most Significant Change technique involves the participatory
collecting of stories.
• Anecdotes are collected from stakeholders with a focus on
change that has happened as the result of an activity, project or
programme.
• A systematic selection of the stories is then made, with a slight
bias in favour of success and impact.

67
 • Based on this selection, the actors involved in the evaluation
exercise start in-depth discussions on project impact and about
the value of the reported changes.

• While the Most Significant Change (MSC) technique produces

stories suited for monitoring and evaluation, many of the stories
that come up can be used for other purposes as well (such as
communication, advocacy, and planning). Stories can be
collected before a face-to-face workshop, and during the event
itself the stories and lessons learned can be discussed. The
stories collected through this method can also feed into manuals
and guidelines to illustrate guidance with experiences and
examples.

• MSC can be used to monitor and evaluate projects or

programmes that do not have predefined outcomes and cannot
therefore be monitored or evaluated using pre-defined
indicators.
• For the same reason, MSC is better equipped to handle
unexpected change than many other methodologies.
• MSC is a participatory technique that helps to identify changes
in people’s lives from their own perspectives. This helps projects
and programmes understand how changes are seen through the
Benefits
eyes of different stakeholders.
• MSC encourages analysis as well as data collection. Individuals
must explain why they believe one change is more important
than another. This helps contribute to the learning process.
• MSC requires no special professional skills to develop and
administer.
• Unlike some other methodologies, project and programme staff
often feel comfortable experimenting with MSC even if they have
no previous experience or training.

• MSC is not designed to access information on predicted,

quantifiable indicators, and is therefore less appropriate for
capturing expected change across large numbers of
stakeholders.
• MSC is not designed to provide comprehensive information
about the changes brought about through a project or
programme. Where assessment of typical change is needed,
MSC needs to be complemented by other methodologies.
• MSC may require considerable resources and different sets of
Risks
stakeholders to be re-visited at regular intervals. As with any
participatory methodology, MSC can be very time-consuming if
done properly.
• MSC is not always very good at accessing information on
negative changes.
• As with any tool or methodology there are a few potential
biases. These include the bias towards stories of success, bias
towards those who are good at telling (or writing) stories, and
subjectivity in the story selection process. There are ways to

68
 overcome all of these biases, but they need to be recognised if
they are to be addressed
Required tools
Timeframe
Required skills Get support from senior management and assign someone to lead the
and resources entire process
Instead of a one-time application it might be interesting to run several
cycles of the technique.
This method could be used to learn lessons on the participatory
methods that have been put in place during the lifecycle of a mandate
Comments
(to run after an opinion is adopted). However, the reflection would be
on the process (i.e. if the engagement methods put in place were
successful or not), but not on the quality of the scientific output itself.
For example, it could be used to assess the success of a pilot.
https://www.intrac.org/wpcms/wp-content/uploads/2017/01/Most-
Useful links significant-change.pdf
https://europa.eu/capacity4dev/file/28239/download?token=lWZXyl9R
Examples http://agris.fao.org/agris-search/search.do?recordID=QW2016000489

Online jam
General outline
Method Online Jam
An Online Jam is a virtual gathering to exchange ideas and views on a
predetermined topic, or to ask questions to a panel of experts. The
interaction takes place at a specific time, usually for about one - two
Short hours, on a social network platform – either an enterprise (internal)
description platform such as Yammer (where these events are called YamJams), or
a public (external) platform such as Twitter (Twitter Chats or Tweet
Jams). Discussion takes place via text posts, sometimes with images or
files attached; audio and video are not normally used.
Objective Exchange ideas, ask questions on an online platform
Target Researchers, Users, Industry, NGOs, CSOs
Geographical
Global, European, Local
scope
Online/offline Offline and online
Impacts Consult, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window Not applicable
(‘Quality of

69
science’ stream
only)
Implementation
There are two basic types of Online Jam. The first is for members of a
team or community who are geographically separated. The Online Jam
allows quickly gathering their inputs and holding a brainstorming
session.
The second type of Online Jam is called an ‘Ask Me Anything’ (AMA)
session. It involves one or more experts, or persons with unique
experiences/insights, responding to questions from interested
participants, in real time. AMAs are becoming increasingly popular
because they offer an easy way for participants to interact directly with
experts and ask them whatever they want.
An Online Jam is different from a Skype/Lync meeting: In an Online
Jam, all inputs are documented in writing so the record can be easily
shared afterwards without the need for someone to take minutes or
listen to a recording. An Online Jam is also different from an E-
Discussion: An Online Jam takes an hour or two, and all participants are
present at the same time. By contrast, an E-Discussion takes place over
days or weeks, with participants contributing on their own schedules.
The output of an Online Jam is a record of all questions, answers and
discussion, copied directly from the social networking platform and
sometimes lightly edited for grammar and spelling (but not changed in
substance). In addition, the facilitator can prepare a discussion
Detailed summary to distil long conversations into a shorter format. The record
description of an Online Jam can serve as a reference and/or as a basis for planning
and implementing related projects/ initiatives. Online Jams are good
when you need to gather ideas quickly and get a team focused on a
particular activity or initiative. If you need deep reflection on challenges
or issues, e.g. in order to develop policy or refine guidance, then an E-
Discussion may work better. Refer to ‘E- Discussions’ in this Toolbox for
more information.
Ask Me Anything sessions are useful for quickly getting answers from
experts to lots of questions– sometimes more questions and answers
than face- to-face meetings of the same duration. They also allow
participants to interact directly with experts, without having to fly
everyone into the same location. An expert can be anyone with relevant
experience and insights from theory and practice concerning a relevant
topic, someone well-known in a particular field, or even a person who
has just completed a particularly interesting/important project or
assignment. Participation in an AMA session may be open to anyone in
an organization, to staff from multiple organizations, or even to the
public.
Facilitators of an Online Jam or AMA need to be very familiar with the
platform, and all participants, including any experts, need to have at
least basic familiarity. Facilitators should be able to identify the key
points of an online conversation, and summarize them quickly, in order
to effectively guide the session.
Participants: from 10 to 100 or even more.

70
 Online Jams are useful for quickly generating ideas and sharing views
among members of a team or community. They allow interaction across
distances without travel costs, and if well-run they are efficient for
Benefits
addressing lots of topics and for capturing the responses in text on the
platform. As a secondary benefit, they also help familiarize team
members with using online discussion platforms.
Risks
Enterprise social network platform (e.g. Yammer) or public social
Required tools
networking platform (e.g. Twitter)
10 - 20 minutes (approx. 5 minutes ‘buzzing’, and 5 - 15 minutes to
Timeframe
share feedback in plenary)
Facilitators: minimum two; the more participants expected, the more
Required skills facilitators there should be.
and resources Optional (for an Ask Me Anything): one or more experts/persons to
answer participant questions
Online Jams are good when you need to gather ideas quickly and get a
team focused on a particular activity or initiative. If you need deep
Comments
reflection on challenges or issues, e.g. in order to develop policy or
refine guidance, then an E-Discussion may work better.
https://www.unicef.org/knowledge-
Useful links
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Examples

Online Platform
General outline
Method Online platforms
Online platform designed to share relevant information and get real-
time feedback from relevant communities and experts. Online platforms
Short
can include discussion forums. Platforms to meet experts, join Focus
description
Groups activities, discover good practices. The platform can also be
used to collect and share data.
Canvas opinions while developing publications and other content; share
ideas; find documents and resources; share good practices. Enabling
Objective
knowledge sharing and communication among selected stakeholder
groups. Sharing of data.
Target Selected stakeholder groups
Geographical
Global
scope
Online/offline Online
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’, ‘Stakeholder dialogue’
stream(s)

71
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify, Report
science’ stream
only)
Implementation
Detailed
description
Benefits
Risks
Required tools
Timeframe
Required skills
and resources
Comments
Useful links
European Union Aviation Safety Agency - EASA, European Cooperative
Examples Programme for Plant Genetic Resources (ECPGR), The Food and
Agriculture Organization - FAO

Online Survey
General outline
Method Online Survey
Short Online surveys are web-based forms which are used to gather
description information from staff, stakeholders, constituents or the general public.
Consult staff, partners, advisors, and a wider stakeholder public. They
provide an anonymous and democratic way of gathering opinions and
views. Specific objectives:
• To solicit inputs from stakeholders
• To assess needs in a given area as a basis for future work or
Objective planning
• To evaluate events
• To increase transparency and participation in decision-making
processes
• To assess needs and requirements (before the activity)
• To evaluate impacts (after the activity)
Target Researchers, CSOs, Users, Industry, consumers, citizens
Geographical
Global, European, local
scope
Online/offline Online
Impacts Inform, Consult, Involve

72
Engagement
‘Preparedness’, ‘Quality of science’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify, Report
science’ stream
only)
Implementation
Stakeholder surveys are a questionnaire-based quantitative tool, most
often used by projects or organisations to increase their understanding
of the knowledge, attitudes, perceptions, interests and experiences of
their stakeholders – both internal and external. Findings are used to
make improvements in the delivery of programmes and/or services.
Conducting a stakeholder survey involves several steps:
Detailed • Defining the objectives of the stakeholder survey
description • Stakeholder Mapping
• Sampling of stakeholders for the survey
• Selecting or designing the survey instrument
• Survey implementation
• Data entry and analysis
• Presentation of findings and recommendations for actions
• Follow-up
Online surveys allow for fast analysis of results and reduce the errors,
which often result from collecting information from individuals or “by
hand”. However, while online tools make it easier to distribute surveys
and collect data, the design of surveys and polls is the most important
success element and is not technology dependent. Engaging
stakeholders in the monitoring and evaluation process not only
Benefits enhances the quality, scope and the depth of the survey but also yields
findings that are useful, relevant and credible. Other benefits:
• Useful to collect and collate quantitative data.
• Data can be used to compare results from another period of
time or against different stakeholder groups.
• It is a quick and cost-effective way to communicate with large
groups of people.
Online surveys hold the risk of attracting a large set of seemingly
conflicting ideas that don’t lead to a workable solution in a timely
manner. Other risks:
• Surveys are usually not useful to identify reasons behind
stakeholder opinions.
• They are less suitable for exploring complex topics or the
Risks attitudes of the interviewee.
• They are not as effective in establishing community
relationships or developing dialogue.
• Response rates addressing broader audiences may be limited,
survey rates are often less than 20%.
• Time consuming: Surveys in general, and stakeholder surveys
in particular, can be very time consuming and may therefore

73
 adversely affect the level of motivation and interest of the
stakeholders in the evaluation process.
• Expensive: Stakeholder surveys are an expensive proposition.
The choice of the data collection method is therefore vital.
• Stakeholder accessibility: When stakeholder groups are
geographically dispersed, when the organization in question is
yet to gain familiarity with stakeholders or when privacy issues
arise, it may be difficult to gain access to stakeholders.
• Stakeholder indifference: Stakeholders may lack interested or
motivation to participate in the survey process.
They can be created using free tools, such as Google forms, which tend
to have various limitations and/or advertising. Alternatively, you can
Required tools
create online surveys with fee-based tools such as Survey Monkey,
SurveyLab, Peakon, Zoomerang. EU Survey.
2 weeks-1 month for the design phase. The timeframe of the
Timeframe engagement activity (filling in questionnaires, analysis, feedback)
depends on the scope of the surveys
Required skills
IT tool, IT manager, data analyst
and resources
Comments Already in place at EFSA
https://www.civicus.org/documents/toolkits/PHX_H_Stakeholder%20Su
Useful links
rvey.pdf
The Food and Agriculture Organization - FAO, Consultative Group for
International Agricultural Research - CGIAR
Examples
FAO FSN Forum survey 2015: Linking your knowledge to policy
decisions (http://www.fao.org/fsnforum/activities/online-surveys)

Open innovation challenge

General outline
Method Open innovation challenge
Crowdsourcing competition to engage citizen-solvers in prize
Short competitions for top ideas and concepts as well as breakthrough
description software, scientific and technology solutions that help solve specific
problems.
Engaging a diverse and broad spectrum of people whose skills, talents,
Objective
and knowledge are key to solving problems and driving innovation.
Target Civil society, researchers, farmers, experts
Geographical
Global
scope
Online/offline Online
Impacts Involve

74
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Do
science’ stream
only)
Implementation
Detailed
description
Benefits
Risks
Required tools
Timeframe
Required skills
and resources
Comments
Useful links
Examples

Nominal group technique

General outline
Method Nominal group technique
Structured problem-solving or ideas-generating activity in which
individuals’ ideas are gathered and combined in a face-to-face,
nonthreatening group environment. The process is intended to promote
Short creative participation in group problem-solving. Each member of the
description group is invited to express their opinions that are used to generate a list
of priorities. Members may be asked to vote or rank priorities from the
list either formally or informally. The voting process may occur multiple
times.
Nominal group technique is designed to promote the free exchange of
opinions and the generation of a list of priorities in a structured and
non-hierarchical discussion forum (maximizes creative participation and
Objective ensures balanced output while utilizing each participant’s experience
and expertise to reach consensus on complex topics). The purpose is to
provide structure to a group discussion when the group is facing the
challenge of reaching agreement on complex topics.
Target Researchers, Industry, NGOs, other communities of interest
Geographical
European, International
scope

75
Online/offline offline
Impacts Consult, Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
Nominal group technique (NGT) is defined as a structured method for
group brainstorming that encourages contributions from everyone and
facilitates quick agreement on the relative importance of issues,
problems, or solutions. Team members begin by writing down their
ideas, then selecting which idea they feel is best. Once team members
are ready, everyone presents their favorite idea, and the suggestions
are then discussed and prioritized by the entire group using a point
system. NGT combines the importance ratings of individual group
members into the final weighted priorities of the group.
Use NGT when:
• State the problem, question, or issue that is the subject of the
brainstorming and ensure that everyone understands.
• Each team member silently thinks of solutions or ideas that
come to mind when considering the problem and writes down as
many as possible in a set period of time (5 to 10 minutes).
• Each member states aloud one idea. The facilitator records it on
the flipchart.
- No discussion is allowed, not even questions for
Detailed clarification.
description - Ideas given do not need to be from the team members'
written lists. Indeed, as time goes on, many ideas will
not be found on their original lists.
- A member may "pass" his or her turn and may then add
an idea on a subsequent turn.
- Continue around the group until all members pass or
until an agreed-upon length of time.
• Discuss each idea in turn. Wording may be changed only when
the idea’s originator agrees. Ideas may be stricken from the list
only by unanimous agreement or when there are duplicates.
Discussion may clarify meaning, explain logic or analysis, raise
and answer questions, or state agreement or disagreement. The
group may also combine ideas into categories.
• Prioritize the recorded ideas in relation to the original question
using multi-voting or list reduction. Typically, the solution with
the highest total ranking is selected as the final decision. Other
variations include estimating the amount of work required to
implement each solution by assigning it a point value; the
higher the point value, the more work involved.

76
 • Many ideas are generated – obviously the more ideas that are
generated the wider the range of options the group will have on
which to decide.
• The technique is useful for identifying problems, exploring
Benefits solutions and establishing priorities.
• It encourages everyone to contribute and prevents people from
dominating the discussion.
• The written generation of ideas encourages the commitment of
participants in taking part in the planned action.
• Does not resolve differences of opinion, as the primary purpose
of the discussion is clarification
• The ideas may be ill informed or impractical – it must be
explained that the process being carried out is not being done
so in a hypothetical sense but is a realistic problem requiring
realistic solutions.
Risks • The Nominal Group Technique is a good stand-alone technique
for simple issues but must be combined with other approaches
where the issue is more complicated or affects people outside
the sphere of influence within the group.
• Participants need to be able to read and write.
• Group members have to make themselves available for the
required time.
Paper and pen or pencil for each individual, flipchart, marking pens.
Required tools
Venue and catering
Timeframe 4 hours
Required skills Facilitator(s)
and resources Staff time for pre-planning
https://www.ncbi.nlm.nih.gov/books/NBK62556/
Comments
https://asq.org/quality-resources/nominal-group-technique
https://www.intrac.org/wpcms/wp-content/uploads/2017/01/Most-
Useful links significant-change.pdf
https://europa.eu/capacity4dev/file/28239/download?token=lWZXyl9R
Examples

Participatory design
General outline
Method Participatory design
Participatory design can be done together with citizens concerned about
a certain issue (e.g. the environment). The starting point is consultation
Short
with individuals and community organisations. This is followed by an
description
interactive design process which includes field tests with the users of
the developed technologies and devices.

77
 It successfully involves the stakeholders, designers, researchers, and
Objective end-users in the design process to help ensure that the end product
meets the needs of its intended user base.
Target Researchers, CSOs, citizens, users, consumers
Geographical
Regional, local
scope
Online/offline Online
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do
science’ stream
only)
Implementation
Participatory design sessions are simple exercises in which we give our
users the tools to create and design mockups of software or products
they would love to use in the “perfect world” scenario while also asking
Detailed them to explain why they built their perfect software or a product in
description that way.
From observing their building process and listening to their explanations
on why they built something in this or that way, we learn a lot of the
things we wouldn’t through a mere interview with the user.
Participation in design process can lead to a more effective design
outcome, because the knowledge and needs of the user are integrated
throughout the process. This helps to avoid misinterpretation or
misunderstandings between designer and user. When participants
Benefits invest their time in a project, they are more likely to continue caring
about the project after it finishes: this can result in them taking
initiative on maintenance and improvements. Participation can also lead
to social outcomes such as empowerment, skills development, and
increased confidence.
Co-creation is a fragile process which requires a lot of attention to truly
listen and be sensitive. It requires excellent facilitation and a
Risks participatory mind set to be effective. Product development is an
intensive process in resources. The added costs and benefits of doing
this in a participatory way are difficult to discern.
Consultation; Workshops; Design Workbooks (interaction design,
Required tools
research through design, ideation).
Timeframe Could be used over years
Subject-matter expertise: Advanced
Required skills
IT skills: Advanced
and resources
Facilitation skills: Intermediate

78
 Event organisation skills: Intermediate
Project management skills: Intermediate
http://actioncatalogue.eu/method/7427
Comments
http://kateferguson.org/documents/Participatory-Design-Handbook.pdf
https://www.intrac.org/wpcms/wp-content/uploads/2017/01/Most-
Useful links significant-change.pdf
https://europa.eu/capacity4dev/file/28239/download?token=lWZXyl9R
Examples www.ecdc.ac.uk

Participatory modelling
General outline
Method Participatory modelling
Focus groups are useful for gaining insight into various viewpoints on
issues. In Integrated Assessment Focus Groups, separate sessions are
organised where participants interact with computer models to gain
Short
insight into the effects of interventions on complex systems. The use of
description
computer models during a focus group has certain advantages. These
Integrated Assessment models serve as tools for analysing complex
issues, such as climate change, together with citizens.
Purposeful learning process for action that engages the implicit and
Objective explicit knowledge of stakeholders to create formalized and shared
representation(s) of reality
Target Researchers, citizens
Geographical
National, regional, local
scope
Online/offline Offline
Impacts Consult
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan
science’ stream
only)
Implementation
Focus groups are useful for gaining insight into various viewpoints on
issues. This has been described in detail in a separate fact sheet. In
Integrated Assessment Focus Groups, separate sessions are organised
Detailed
where participants interact with computer models to gain insight into
description
the effects of interventions on complex systems. The use of Integrated
Assessment (IA) models during a focus group has certain advantages.
These models serve as tools for analysing complex issues by including

79
 expert input. This is given both in face-to-face interaction, and by
inviting the participants to interact with scientific models with a user-
friendly computer interface. The participants get a feeling for the effects
of all types of interventions in complex systems, and the potential
results are predicted by underlying numerical models. This has been
described in detail in the chapter ‘Citizen interaction with computer
models’ (Dahinden et al., 2003).
When looking at climate change models four complexity dimensions are
recognised:
• Spatial – there are links between local activities and global
influences and vice versa;
• Temporal – there are both short term and long-term
perspectives which are very relevant;
• Uncertainties – in the assumptions on cause and effect in the
systems;
• Policies – of different entities across the world which influence
the system strongly.
The use of IA models helps to cope with these complexities
simultaneously. When building the models knowledge from various
disciplines is integrated and is used to predict cause and effect of a
large number of variables. A lot of work on the Integrated Assessment
Focus Group has been specifically designed for climate change but the
method can be also applied in other areas.
High degree of ownership and motivation towards change for the people
involved in the modeling process. It also helps to develop more
Benefits
acceptable solutions and often creates more consensus among the
stakeholders involved.
• If the model guides the discussions too much, it can limit the
discussions based on the assumptions behind the model.
• Some users in IA Focus Groups expected gaming environments
and were frustrated by complex interfaces.
• If a model has not been designed for lay people it will be
Risks necessary to offer continual technical assistance.
• Participants learn more from systems which reveal some of the
intermediate results and allow users to understand relationships
between variables.
• For the process to have results, the users need to trust the
models and the experts supporting the process
Required tools
A common time frame for the IA Focus Groups is 5 sessions of 2.5
Timeframe
hours over several days.
Subject-matter expertise: Intermediate
IT skills: Advanced
Required skills
Facilitation skills: Intermediate
and resources
Event organisation skills: Intermediate
Project management skills: Intermediate
Comments Advanced combination of participatory design and focus groups.

80
 http://owsgip.itc.utwente.nl/projects/complex/images/uploaded_files/D
Useful links
63_Voinov.pdf
Examples

Participatory sensing
General outline
Method Participatory sensing
Participatory sensing projects involve volunteers in the gathering of
data for research. This process is facilitated with ICT platforms which
Short
often include the use of hand-held devices such as smartphones. This is
description
one of the methods which is used within various forms of Citizen
Science.
Objective Gathering data for research
Target Researchers, citizens
Geographical
EU, international, national, regional, local
scope
Online/offline Online
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Do
science’ stream
only)
Implementation
Roles of participation
The core activity of the volunteers in the project is the gathering of
data, usually at a specific location and sometimes also at a specific
time. The platform needs to facilitate the easy submission of data to a
central location. Other possible roles of citizens can be in analyzing the
data. Sometimes this only concerns their own data but it can also relate
to the whole dataset or a subset in which they are personally
interested. There are also examples of projects where Civil Society
Detailed
Organizations are involved in the project definition of a project and also
description
involve their own members in the execution. In general, there are often
benefits for the volunteers, which can be in the form of increased
knowledge of the subject matter.

Infrastructure and tools

There are large variations among the types of participatory sensing
projects. An important distinction is to what extent at the start of the
project the tools are already available for the data collection. In some

81
 projects the tools have been developed from scratch and subsequent
projects have a much shorter lead time and lower investment of time
and money. For example, the tools developed in the original NoiseTube
project in Paris have been published with an open source license and
other parties are encouraged to organize their own participatory noise
mapping projects with the open framework.
Cost effective data collection and presentation;
Engaging citizens (or other stakeholder groups, such as affected
Benefits
employees) in research on (local) challenges.
Access to free or cheap publicity/dissemination.
A potential weakness is the quality assurance of the collected data.
Research has already been done on the quality of large volumes of data
of cheap sensors and volunteers compared to other data collection
approaches with more expensive sensors by professionals, but smaller
data volume. See for example research done for the NoiseTube tools.
Investment in platform development and infrastructure, including
Risks relations with participants, can be slow and large. Once the
infrastructure has been implemented it can be a source of ‘big data’.
Keeping volunteers connected and active in the project has proven to
be an obstacle for some projects. There are various motivations for
participating in scientific research. Often within one project there can
also be multiple motivators which vary across the ‘crowd’ of volunteers
which contribute to a crowd sourcing project.
Required tools
The complexity of data collection activities varies with the different
systems. Some require very little user training or instruction and can be
applied in a one-time event. On the other hand, some may be more
Timeframe complicated and require multiple data capture moments and more user
training. Most systems require a significant amount of time for the
development of the platform. Follow-up projects, or projects using
existing tools, will then usually have shorter lead times.
Subject-matter expertise: Intermediate
IT skills: Advanced
Required skills
Facilitation skills: Intermediate
and resources
Event organisation skills: Basic
Project management skills: Intermediate
Comments
https://en.wikipedia.org/wiki/Participatory_sensing
Useful links
https://dl.acm.org/doi/10.1145/1979742.1979768
www.energysense.nu
Examples
http://www.noisetube.net/index.html#&panel1-1

Participatory workshops
Participatory workshops are meetings that help participants analyse, share and enhance
their knowledge to plan, manage and evaluate development projects and programmes.

82
Visual aids – such as mapping, videos, illustrations, timelines, card sorting and ranking,
Venn diagrams, seasonal calendar diagramming and body maps - are often used to engage
participants and capture knowledge.
Workshops may be organised in several formats. The most widely used variants are
outlined below.

Fishbowl
General outline
Method Fishbowl workshop
A fishbowl conversation is a form of dialogue that can be used when
discussing topics within large groups. Best used in conferences and
workshops, the Fishbowl focuses the entire group’s attention on a
discussion among 3 - 6 people. Other people present become
observers, active listeners, and potential participants through a rotation
Short process, which reduces the distance between speakers and audience.
description Fishbowl facilitation is a simple, effective alternative to a plenum
discussion. In combining large group facilitation with small group
discussions, fishbowl creates a vivid and spontaneous discussion
format. The Fishbowl facilitation got its name from the way the
participants are seating. The chairs are placed in two circles: the inner
circle ("fishbowl") and one or more outer circle(s).
Facilitating dialogue between experts in a way that exposes others to
their knowledge while expanding the collective understanding of a
Objective
subject. This method is used to foster dynamic participation, avoiding
lengthy presentations.
Academia, NGOs, farmers and primary producers, business and food
Target
industry, distributors, practitioners, policy makers, consumers
Geographical
European, International, local
scope
Online/offline offline
Impacts Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do
science’ stream
only)
Implementation
Fishbowls involve a small group of people seated in the inner circle,
having a conversation in full view of a larger group of listeners. The
Detailed
participants in the inner circle discuss the topic(s) while all other
description
participants seating in the outer circles(s) listen and observe the
discussion.

83
 • Identify 3-6 experts (or participants who have experience) on
the issue to be discussed.
• Brief the experts/participants on the Fishbowl process.
Set up a small circle of chairs surrounded by a larger circle, with
3 or 4 additional chairs to facilitate mobility.
• Open the session with the experts in the centre circle.
• Explain the process, the objectives and the issue that will be
discussed.
You can opt for one of the two types of Fishbowl: open or closed.
An open fishbowl contains several empty chairs in the centre circle
from the outset. Any member of the audience can join the discussion by
occupying an empty chair at any time. A “fish” must voluntarily leave
the centre circle to free a chair. The discussion continues with
participants frequently entering and leaving the Fishbowl. Participants
can have more than one opportunity to move into the inner circle.
In a closed fishbowl, the facilitator splits the participants into two
groups (or more as needed) and assigns the role of speakers to one
group, and the role of observers to the other. The initial participants in
the inner circle speak for some time about the chosen subject. When
time runs out (or when no new points are added to the discussion), the
first group of participants leaves the fishbowl and a new group from the
audience enters. The new group continues discussing the issue. This
may continue until all audience members have spent some time in the
fishbowl. The closed fishbowl approach is only appropriate when all
participants have at least some level of knowledge about the subject.
The outer circle must always observe silently, and this principle should
be enforced diligently by the facilitator. Participants in the outer circle
can prepare questions and comments so that they are ready to move
into the inner circle. Once the topics or the time allocated have been
covered, the facilitator should summarize the discussion and open the
floor for a debriefing, after removing the inner circle of chairs. During
the debriefing, review key points, interesting comments and the group’s
feelings regarding particular issues. Participants must be allowed to
develop conclusions and express themselves freely.
Providing the participants with an overview document of the lessons
learned and a list of key resources can be helpful after the exercise has
ended.
A Feedback Fishbowl is a variation which systematizes interaction
between the inner and outer circle. The “fish” discuss the issue for 15
minutes, then turn their chairs around to face the outer circle. The
outer circle puts their comments and feedback directly to the “fish” in
front of them. Then members of the inner circle again face the centre
and incorporate the new information into the conversation (while the
outer circle remains silent). After the conclusion of this round, the two
groups change places, and the process is repeated.
Another variation is to invite people with similar opinions or experiences
to sit in a Homogeneous Fishbowl. This arrangement aims to provide
the outer circle with evidence and logic to support a cohesive
perspective. This helps to avoid wasteful disagreements at the early
stages of the discussion and creates clear concepts for debate.

84
 In contrast to the previous variation, in a Heterogeneous Fishbowl,
one person from each divergent viewpoint is invited to sit in the
fishbowl. The debate must be carefully managed by the facilitator to
ensure that it is productive and examines the full variety of opinions
equally.
Multiple Fishbowls are ideal for addressing issues with large groups,
or for cases when there are language barriers between participants.
Assign a moderator to each fishbowl to provide clear instructions as well
as support throughout the exercise. After the first discussions,
representatives from each fishbowl form a new central fishbowl, and
continue the conversation. According to the composition of the group,
decide if the debriefing would be more effective in the large group or
back in the original multiple fishbowls. Ensure that each moderator
records the reflections so that these can be shared in a resource for all
the participants.
Roleplays can be conducted in closed fishbowls. Divide participants
into as many groups as the number of roles you have prepared. Each
group then prepares a role, although only one of their members will
play it. The roleplay is then held in the middle of the room while the
other participants observe from outside. After the roleplay, close the
session with a debriefing. If the debriefing takes longer than the
roleplay, it means the exercise was thought-provoking.
Reduces distinctions between the speakers and the audience. This
method is alternative to traditional debates. It could be a valid
Benefits
substitute for panel discussions, allows to foster dynamic participation
and address controversial topics.
More reserved groups may require encouragement to take up a place in
Risks the inner circle. This can be helped by well-formulated objectives and
introductions to the subject matter.
Open space or large room with enough space for participants to move
around easily. One chair for every participant (plus three or four empty
Required tools
chairs). Flip chart and markers for the note-taker.
Microphone(s) (optional)
From 1 hour and a half to 3 hours. Suggestion on the break-down of
activities for a 1h30 session:
• Introduce the method and the objectives/guiding questions of
the discussion (10 minutes)
• Fishbowl discussion (1 hour)
Timeframe
• Debriefing (20 minutes)
• Introduce the method and the objectives/guiding questions of
the discussion (10 minutes)
• Fishbowl discussion (1 hour)
• Debriefing (20 minutes).
Required skills 1 or more facilitators to stimulate the discussion. Support team
and resources Optional: Rapporteur

85
 If the outer circle participants want to make more contributions after
Comments the fishbowl session has ended, open a blog, wiki or discussion forum to
continue capturing their comments, reflections and questions.
https://www.unicef.org/knowledge-
Useful links exchange/files/Fishbowl_production.pdf
http://acnudh.org/wp-content/uploads/2011/11/slitoolkit.pdf
https://experience.sap.com/skillup/fishbowl-a-user-research-method-
for-future-scenarios/
Examples
http://www.genresbridge.eu/fileadmin/templates/Genres/Uploads/Docu
ments/GenRes_Bridge_Sharing_Perspectives_workshop_report.pdf

Innovation Challenge
General outline
Method Innovation Challenge Workshop
Short Structured method to analyse specific topics, get hands-on experience
description on practical cases and incorporate stakeholders'/experts opinions
Fostering knowledge and best practices exchange. Case-driven
Objective
innovation. Share ideas and start discussions. Best Practices Sharing.
Target Researchers, NGOs, farmers cooperatives
Geographical
National
scope
Online/offline Offline
Impacts Inform, Consult
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
• Thematic paper: production of a thematic paper on the topic of
the workshop (e.g. regulation, fresh food, technology);
• Organisation of site visits (e.g. agriculture enterprises "best
Detailed
practices”)
description
• Thematic conferences with policy makers and experts on the
topic to present future scenarios/challenges and future
development of the workshop topic
Direct learning from real case examples, foster bottom-up innovation,
Benefits
practice-based discussions.
Risks High weight of local context (which may hamper replicability)

86
Required tools Flipcharts, post-its, presentations, Slido/Mentimeter.
Timeframe 1 full day
Required skills
Workshop moderation, language, deep knowledge of the topic
and resources
https://ec.europa.eu/eip/agriculture/en/event/skin-innovation-
Comments
challenge-workshop
https://www.unicef.org/knowledge-
Useful links exchange/files/Fishbowl_production.pdf
http://acnudh.org/wp-content/uploads/2011/11/slitoolkit.pdf
Examples

Knowledge Fair
General outline
Method Knowledge Fair (or Share Fair) workshop
An event designed for sharing large amounts of information from
numerous expert sources at a common venue with the help of visual
aids and displays. A number of well thought out formats such as chat
Short
shows and market places are used. Whether discussing topics within
description
large groups through fish bowls or world cafés or brainstorming in mind
mapping exercises, these formats encourage interaction and learning
among participants.
A face-to-face method to:
• share experiences
• promote best practices
Objective • understand peer perspectives
• provide a networking platform
• empower people
• solve problems
Target Consumers, researchers, NGOs, risk assessors, academia
Geographical
European, International, Local
scope
Online/offline Offline
Impacts Inform, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)

87
Implementation
A knowledge share fair is a buzzing, interactive and collaborative
workspace, with lively discussions and practical demonstrations. There
are facilitated sessions to learn how others have improved the
effectiveness and quality of their work, and opportunities for
networking, allowing people to connect and link ideas as well as
exchanging opinions in a natural way and relaxed setting. A share fair is
not a goal itself, but the beginning of a process. It will create new
partnerships; new exchanges between people that would like to work
together, in a different way. Events such as a share fair, that include
the application of knowledge sharing tools and methods, encourage
interaction and thus need careful planning. Even though a fair aims at
spontaneous and informal exchange of ideas, improvisation should
reside in the flow of thoughts and content, not in the approach.
• Advance preparation is required for any Knowledge Fair,
particularly if travel is involved.
• Identify key objectives and outline the major themes for the
contributors to explore.
• Invite relevant actors, institutions and individuals to participate.
Include people from diverse groups with different backgrounds,
to contribute to the complexity of ideas.
• Provide practical guidelines and individual assistance and allow
time for participants to create their displays in advance.
Detailed
• Standardized templates and display equipment will minimize
description
visual distractions and maximize the amount of information that
is absorbed and retained.
• Select a site for the fair in a high-traffic area, to attract as many
visitors as possible. This has to be done well in advance.
• Ensure that support is available early enough to assist the
presenters in setting up their displays.
• Identify in advance who will require power supplies, projectors
or other special equipment for their presentations.
• Publicize the fair as widely as possible, using methods
appropriate to the target audience.
• Arrange to have technicians on hand, as well as access to a
secretariat, should anyone require logistical support.
• Ensure that interested parties have barrier-free access to the
fair, and sufficient time to explore.
• Record the progress and reflections in “real time” using
discussion forums, blogs, or social networks.
• Collect the relevant documentation and end-of-activity reports in
a central location for accessible future reference
The different types of sessions that can be organized are: plenary
sessions, discussion sessions, projections, screenings (clips, videos,
slide shows...), video sessions, poster sessions, training sessions,
theatre plays, demonstrations, stands and information booths
Allows people to connect and link ideas as well as exchanging opinions
Benefits
in a natural way and relaxed setting.

88
 Too many parallel sessions: while people may enjoy the luxury of
choosing from over thirty to sixty events in three or four days, there
may be too many parallel sessions competing (consider quality versus
quantity, and do not plan too many sessions).
Risks Large groups: You may have 100-150 people in a session, and the
meeting room layout may also pose challenges in what session types
and facilitation approaches may be possible (consider breaking a large
room of 100 participants into smaller groups, using World café or other
participatory methods).
Displays or movable boards, flip charts, laptops and projectors, big
Required tools posters, pencils and markers, microphone, summary sheets for every
stand, name labels for every stand.
Duration of the activity: one to three days. Events such as a share fair
need careful planning. Even though a fair aims at spontaneous and
informal exchange of ideas, improvisation should reside in the flow of
thoughts and content, not in the approach. Planning should start at
Timeframe
least with one year in advance. Alternatively, the fair can be combined
with other events. For example, it is possible to dedicate a complete
afternoon to a knowledge fair with specific activities and then let it
stand in the background for demonstration and illustration purposes.
The organization of a Knowledge Fair requires careful planning,
Required skills resources and budget allocation. Multiple skills and resources are
and resources needed. The organization can be subcontracted to an event planning
agency or organizer.
A further analysis is the basis for written and visual documentation of
the methods results and recommendations. A personal presentation of
Comments these outcomes in the workshop provides the platform for a discussion
of their practical consequences and implementation.
It could be used for topic-based engagement
http://www.fao.org/3/i2538e/i2538e04.pdf
Useful links
http://www.fao.org/3/a-aq228e.pdf
The Food and Agriculture Organization - FAO, Center for International
Examples
Forestry Research - CIFOR

Open space technology

General outline
Method Open space technology workshop
The Open Space Technology is a method to organize participation
events basically of large and medium scale. The method is based on the
principles of passion, responsibility and commitment, bearing in mind
Short
the assumption that the most productive way to work is to work on a
description
topic for which one cares. It consists of:
• Guided development and implementation of an agenda that
deals with a central theme;

89
 • Uses the self-organizing capacity of participants to reach
learning and knowledge sharing objectives;
• Participatory approach to identifying relevant learning and
knowledge sharing content and methods.
• To support informal learning, brainstorming, networking, deal
making and collaboration within groups that have identified
common goals;
Objective
• To address highly complex central themes that no single person
or small group can understand completely;
• To design action plans.
Target Researchers, NGOs, policy makers, consumers, industry
Geographical
EU, International
scope
Online/offline Offline
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
A one day Open Space event has three parts:
• An introduction to the whole plenum, explaining the method and
what is expected of the participants in order to have a
successful event. It is followed by the agenda setting, where
workshop sessions are announced and scheduled and where the
participants register for the workshops of their choice (It all
takes a maximum of 1 hour – 15min for the introduction and the
rest of the time is dedicated to agenda setting and enlisting).
• The sessions themselves, where multiple workshops are
conducted simultaneously.
Detailed • A final round with the whole plenum in which the facilitator
description summarizes the events of the day and gives participants the
opportunity to comment on their experiences and lessons
learned.
Rules:
In the introduction, the facilitator should explain clearly how the event
is going to work. The method’s originator Harris Owen offers four
principles and one law as framework rules for an Open Space event:
• “Whoever comes is the right people”: especially important at
stakeholder events with a broad scope of participating
organizations, or where differences in status and hierarchic
position may occur.

90
 • “Whatever happens is the only thing that could have happened”:
sometimes the expectations of the participants differ from what
the event is really like, be it the event in general or discussion
dynamics.
• “Whenever it starts is the right time.”
• “When it is over, it is over”: these two principles concern the
productive time spent in a workshop session. When there is
nothing worthwhile to discuss anymore, it is better to close the
workshop and join another one or have a break instead of
clinging on the scheduled 90 minutes. This leads to the “Law of
two feet” which says that you are allowed to switch workshops
within an ongoing session or separate for a more intense
discussion in a smaller group or even a break. If a participant
feels, he or she is neither able to learn nor to contribute
something to the discussion, or the discussion turns into a
direction which is of no interest for him or her, he or she should
not waste time and leave the workshop and go to another one
which he or she finds more interesting. Neither should the left
group feel offended about this nor the person who left have a
guilty conscience. Additionally, it is a possibility to avoid or leave
workshops where a single person dictates the discussion
dynamics, issues and accepted facts without paying regard to
the opinions of the others.
Follow up:
A book of proceedings should be sent to the participants only a few
days after the event. It is a summary of the outputs of all workshop
sessions with a short overview evaluation. Then the gathered data can
be analyzed more closely to produce a report fitting to the objectives of
the project.
Participants decide which session they want to join according to their
Benefits interests and needs, which means all those who attend are the right
people.
Risks
Wall with the open space agenda. A print-out of the open space
Required tools principles and instructions. Template print-outs for reporting. Enough
flip charts for the different breakout sessions. Pencils and markers.
One to three days is the recommended length. A single day event can
produce a lot of information and data, lead to intense discussions,
information translation between stakeholders, networking and ideas for
new projects or other follow up actions. A two day event allows better
Timeframe recording and the opportunity to convene new workshops which have
developed out of the discussion process and dynamics of the first day
(e.g. the planning of a new proposal/project or issues which arose in
one workshop and could not be discussed completely). Additionally, a
three-day event allows more time for reflection.
Subject-matter expertise: Basic
Required skills
IT skills: Basic
and resources
Facilitation skills: Intermediate

91
 Event organisation skills: Intermediate
Project management skills: Intermediate
Despite the principle of voluntary self-selection, the
people/homepages/distribution lists, etc. to which the invitation is send
Comments should be chosen accordingly to the objectives of the event. For
example, making sure that every stakeholder is represented
sufficiently.
Useful links https://www.openspaceworld.com/brief_history.htm
Examples Center for International Forestry Research - CIFOR

Unconference
General outline
Method Unconference (or BarCamp)
An unconference is a participant-driven meeting. They are open,
participatory workshop-events, the content of which is provided by
participants. Typically, at an unconference the agenda is created by the
Short attendees at the beginning of the meeting. Anyone who wants to
description initiate a discussion on a topic can claim a time and a space. Some
unconference sessions are led by the participant who suggested its
topic; other unconference sessions are basically open discussions of the
session topic.
Avoid hierarchical aspects of a conventional conference, such as
Objective
sponsored presentations and top-down organization
Target Researchers, NGOs, academia, risk assessors
Geographical
Local
scope
Online/offline Offline
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
Unconferences are well suited to promoting interactions and networking
between attendees as they allow a more flexible agenda. Discussion
Detailed topics are shaped and influenced by participants, with exchanges of
description knowledge from many to many. This works particularly well when
discussion groups are relatively small, creating a flexible, creative, and
conducive environment for exchanges.

92
Depending on the mission and the goals of the participants,
unconferences can be organized in many different ways: events
accompanying a traditional conference, where participants organize
themselves to discuss topics without any pre-planned agenda, similar to
“bar camps,” where the program is rewritten or overwritten on-the-fly
by the participants using whiteboard schedule templates. Other
examples involving project-driven events include those mainly focused
on technology topics such as “hackathons.” During such events, small
sub-teams gather to work together on developing/addressing particular
parts of a software project. A little more organization is needed to
arrange a “curated unconference” where topics and structures are
collected by potential participants prior to the event. A group of
organizers, in a transparent and open procedure, then sort through
these ideas to build a structure of large and/or small-group discussion.
Other guidelines for their implementation cover:
• Having a clear and visible mission statement can be a very
effective way of focusing ideas for the content and structure of
the event.
• Minimize the Lecture-Style Presentations. One of the defining
features of an unconference is its inversion of the common
features of more traditional meetings, in particular academic
conferences.
• Involve participants in planning and structuring of the event.
Participant-centric thinking is perhaps the key feature that
differentiates unconferences from more traditional meetings.
Empowered participants, who know that they can directly
influence and contribute to the structure and content of a
meeting, tend to be much more invested in its success and
outcome.
• Provide an open, relaxed atmosphere. In order to make an
unconference a success, the atmosphere of the event should be
relaxed, open, friendly, and fun.
• An effective way to encourage communication and participation
is through ice-breaker activities during the early stages of the
event.
• Trust your community unconferences prioritize focusing on, and
engaging with, everyone who chooses to get involved in the
event. This is in contrast to more traditional meetings, where
the focus is much more on what the organizers have planned
and the scheduled session presenters. Thus, in an unconference
format, responsibility for the success of the event is more
equally distributed across all participants.
• Engaging in communication is one of the reasons why people
choose to come together for any meeting. One main
characteristic of unconferences is the emphasis on interactive
communication that gives all participants a chance to have their
contributions heard by others.
• A great way to extract the collective expertise, knowledge, and
experience of attendees during unconference sessions is to
encourage participants to identify and work together towards a

93
 common goal, and to document how they attempted to get
there. A good way to do this is to write down all ideas and
suggestions, so that later they can be sorted and considered.
Advantages of the unconference format include:
• a focus on topics that are relevant to the attendees (because
they suggested them);
• an opportunity for teamwork development;
• flexibility of schedule;
• an emphasis on contributions from every participant.
Benefits
The relationships built during an unconference often continue well past
the event. The interactions can lead to productive collaborations,
professional development opportunities, and a network of resources and
are very effective at building a community amongst participants. The
unconference format, therefore, gives participants experience in
working together.
Being a user-driven engagement event, the risk is to lose control on the
real goals and mission of the unconference. Some governance and
management tools need to be put in place while guaranteeing the
Risks
participants’ freedom in structuring the event. In an unconference
format, the organizers will be successful if they trust the community to
work with them to make the event a success.
It is important to have tools that allow attendees to share the
Required tools resources, ideas, and challenges of the session conversations. Space to
organize participatory workshops and materials
Timeframe From half-a day to a one-day event
Required skills
Facilitators, supporting teams
and resources
Comments An event organizer could be involved
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4310607/
Useful links
https://en.wikipedia.org/wiki/BarCamp
Examples UK Health Camp https://ukhealthcamp.com/

Six thinking hats

General outline
Method Six thinking hats workshop
The Thinking Hats exercise is a kind of roleplay in which different
perspectives are represented by hats of different colours. When a
participant is symbolically wearing a specific hat, they must seek to
perceive the situation through the lens associated with that colour. This
Short
method shows how different aspects of one’s personality can approach
description
a problem differently. It helps to achieve more comprehensive
perspectives and sounder solutions, by forcing the participants to step
outside the limits of their standard thought processes and points of
view.

94
 The main objective is to assist in the critical analysis of complex
situations by simulating diverse points of view in a controlled
Objective
environment. This method is normally used in the context of training
and can be considered a training technique.
Target Researchers, students, academia, risk assessors
Geographical
European, International, Local
scope
Online/offline offline
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
Six thinking hats is a powerful technique for looking at decision making
from different points of view.
It allows emotion and skepticism to be brought into what might
normally be a purely rational process, and it opens up the opportunity
for creativity within decision making
Each "Thinking Hat" is a different style of thinking. These are explained
below:
• White Hat: with this thinking hat, you focus on the available
data. Look at the information that you have, analyze past
trends, and see what you can learn from it. Look for gaps in
your knowledge, and try to either fill them or take account of
them.
• Red Hat: "wearing" the Red Hat, you look at problems using
Detailed your intuition, gut reaction, and emotion. Also, think how others
description could react emotionally. Try to understand the responses of
people who do not fully know your reasoning.
• Black Hat: using Black Hat thinking, look at a decision's
potentially negative outcomes. Look at it cautiously and
defensively. Try to see why it might not work. This is important
because it highlights the weak points in a plan. It allows you to
eliminate them, alter them, or prepare contingency plans to
counter them. Black Hat thinking helps to make your plans
"tougher" and more resilient. It can also help you to spot fatal
flaws and risks before you embark on a course of action. It's one
of the real benefits of this model, as many successful people get
so used to thinking positively that they often cannot see
problems in advance, leaving them under-prepared for
difficulties.

95
 • Yellow Hat: this hat helps you to think positively. It is the
optimistic viewpoint that helps you to see all the benefits of the
decision and the value in it. Yellow Hat thinking helps you to
keep going when everything looks gloomy and difficult.
• Green Hat: this hat represents creativity. This is where you
develop creative solutions to a problem. It is a freewheeling way
of thinking, in which there is little criticism of ideas. (You can
explore a range of creativity tools to help you).
• Blue Hat: this hat represents process control. It's the hat worn
by people chairing meetings, for example. When facing
difficulties because ideas are running dry, they may direct
activity into Green Hat thinking. When contingency plans are
needed, they will ask for Black Hat thinking.
It helps to achieve more comprehensive perspectives and sounder
solutions, by forcing the participants to step outside the limits of their
standard thought processes and points of view. Discussions can be
more productive. The method reduces the possibility that participants
Benefits
are permanently stuck in one mode of thinking. The structure allows
participants to easily track where the conversation is going. Facilitates
less defensiveness and greater participation. Facilitates deeper thinking
and creativity; thinkers only have to deal with one thing at a time.
Participants are not typically familiar with the process and need to learn
Risks it before they can use it. Facilitator needs to manage the tendency for
labelling (e.g. you are definitely a black hat person)
If hats are not appropriate, use T-shirts, coloured pens. Copies of the
Required tools descriptions and roles for the participants who have a specific role to
play. Pencils and markers. Flip chart.
Timing (1 hour to 1 hour 30 minutes): Introduction to thinking hats
Timeframe method and roles (15 minutes), Group processes and discussion (30-45
minutes), Collective debriefing (20-30 minutes).
Required skills
Skilled facilitator
and resources
http://www.fao.org/elearning/course/FK/en/pdf/trainerresources/PG_Si
xThinkingHats.pdf
Comments https://coast.noaa.gov/data/digitalcoast/pdf/met-activities-hats.pdf
de Bono Consulting - Six Thinking Hats
www.debonoonline.com/Six_Thinking_Hats.asp
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4310607/
Useful links
https://en.wikipedia.org/wiki/BarCamp
Examples

World café
General outline
Method World café (or Knowledge café)

96
 A World café is a structured conversational process for knowledge
Short sharing in which groups of people discuss a topic at several tables, with
description individuals switching tables periodically and getting introduced to the
previous discussion at their new table by a "table host"
World Cafés can create results to generate new ideas, to enable joint
decision-making on key strategic issues, to discover new ways for
collaboration, to reflect on the implications of a complex issue and in
identifying specific step(s) for further exploration and implementation.
Objective
The method facilitates:
• Sharing experiences, stories or project results.
• Problem solving.
• Planning.
Target NGOs, policy makers, researchers, citizens, consumers, industry
Geographical
European, International, Local
scope
Online/offline offline
Impacts Consult, Involve
Engagement
‘Quality of science’, ‘Stakeholder dialogue’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do
science’ stream
only)
Implementation
The World Café is a whole group interaction method focused on
conversations. A Café Conversation is a creative process for leading
collaborative dialogue, sharing knowledge and creating possibilities for
action in groups of all sizes. The environment is set up like a café, with
paper-covered tables for four supplied with refreshments. People sit
four to a table and hold a series of conversational rounds lasting from
20 to 45 minutes about one or more questions which are personally
meaningful to them. At the end of each round, one person remains at
each table as the host, while the other three travels to separate tables.
Detailed Table hosts welcome newcomers to their tables and share the essence
description of that table's conversation so far. The newcomers relate any
conversational threads they are carrying and then the conversation
continues, deepening as the round progresses. How to use it:
Seat four or five people at small Cafe-style tables or in conversation
clusters.
Set up progressive (usually three) rounds of conversations of
approximately 20-30 minutes each.
Questions or issues that genuinely matter to your life, work or
community are engaged while other small groups explore similar
questions at nearby tables.

97
 • Encourage both table hosts and members to write, doodle and
draw key ideas on their tablecloths or to note key ideas on large
index cards or placemats in the center of the group.
• Upon completing the initial round of conversation, ask one
person to remain at the table as the host while the others serve
as travelers or "ambassadors of meaning." The travelers carry
key ideas, themes and questions into their new conversations.
• Ask the table host to welcome the new guests and briefly share
the main ideas, themes and questions of the initial conversation.
Encourage guests to link and connect ideas coming from their
previous table conversations, listening carefully and building on
each other's contributions.
• By providing opportunities for people to move in several rounds
of conversation, ideas, questions, and themes begin to link and
connect. At the end of the second round, all of the tables or
conversation clusters in the room will be cross-pollinated with
insights from prior conversations.
• In the third round of conversation, people can return to their
home (original) tables to synthesize their discoveries, or they
may continue traveling to new tables, leaving the same or a new
host at the table. Sometimes a new question that helps deepen
the exploration is posed for the third round of conversation.
• After several rounds of conversation, initiate a period of sharing
discoveries and insights in a whole group conversation. It is in
these town meeting-style conversations that patterns can be
identified, collective knowledge grows, and possibilities for
action emerge.
By dividing a large group into smaller subgroups, conversations can be
made more focused, relaxed and participatory, with greater opportunity
for all participants to speak and contribute equally – thereby
encouraging authentic sharing of experiences and knowledge. Rotation
of groups from one table to the next adds value to the discussion, by
Benefits allowing a group to build on the previous group’s thoughts and ideas
about a particular issue. The tool works best with a mix of people
bringing different ideas and experiences. This tool is a good way to
bring people from different backgrounds together to think about a
complex issue and to find imaginative ways forward. Well facilitated,
this makes work fun.
Facilitators need to be experienced. If feedback is not analysed
Risks immediately after the event, you will risk losing some of the emerging
themes and imaginative solutions.
Flip charts, paper hanging on the wall, a white- board, colored cards,
markers for each table, visual instructions of the method (on
Required tools
PowerPoint or printed out), music or a bell to indicate when to rotate
tables. Venue and catering.
Recruitment: 3 – 4 months before the workshop;
Timeframe Data analysis: 1 – 2 months;
Feedback/Information of results: 1-2 weeks;

98
 Preparation of materials: 1-2 weeks;
Room booking: 1-6 months;
Duration of the activity: 90 minutes.
The basic process is simple and simple to learn, but complexities and
nuances of context, numbers, question crafting and purpose may mean
an experienced host needs to be recruited to help.
Required skills Subject-matter expertise: Basic/Advanced
and resources IT skills: Basic
Facilitation skills: Advanced
Event organisation skills: Advanced
Project management skills: Basic
The World Café is a trademark of the World Café Community
Foundation. The World Café Hosting and Consulting Services provide
professional hosting and consulting services. The method can be easily
replicated with own resources.
Comments
A main result is graphic recording, which involves capturing people's
ideas and expressions in words, images and colour. This documentation
is created by the participants of the World Café. It allows the group's
collective work to be shared with others as a framework and guide.
http://www.theworldcafe.com/
Useful links https://www.unicef.org/knowledge-
exchange/files/World_Cafe_production.pdf
http://scottishhealthcouncil.org/patient__public_participation/participati
Examples
on_toolkit/world_cafe.aspx#.XnpTFZNKiCQ

Peer Assist
General outline
Method Peer Assist & Virtual Peer Assist
A participatory method of learning with and through peers by sharing
experiences, insights and knowledge. The method is designed to
develop context-specific solutions to a challenge, based on participants’
previous practices and experiences. Collaborative analysis is carried out
in order to adapt action to a specific situation. The method enables
exchange of tacit knowledge and good practices in order to assist a peer
in a particular activity or challenge. It is analogous to a peer review for
Short a paper or publication, but instead of getting written comments on a
description document, a Peer Assist generates verbal comments and guidance (and
usually some references to relevant, pre-existing written materials) on
the plans for a project/ initiative/ other work-related challenge. A Peer
Assist provides a safe environment for a learning group to articulate a
challenge or problem they face, and for an advisory group to respond
by sharing what they know from their own relevant experience – in the
form of suggestions, anecdotes, good practices and recommendations –
to help address the challenge

99
 A Virtual Peer Assist is an online group conversation between a learning
group and an advisory group, taking place over a few hours, with all
participants online simultaneously.
Peer Assists are useful:
• For planning new projects/ initiatives.
• For planning projects/ initiatives in very new areas where there
is little or no prior knowledge or experience on the team,
and/or for planning projects that have high risks and costs to
failure.
• During implementation, when a complex technical challenge
reveals itself, and swift action is needed, but the responsible
Objective
team is uncertain of how to proceed.
The tool is ideal:
• To stimulate collaborative problem-solving
• To connect experienced peers
• To address challenges
• To elicit feedback
• To review projects
• To stimulate complex analysis
Target Researchers, Scientists, other communities of interest
Geographical
European, international
scope
Online/offline Online
Impacts Inform, Consult
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do
science’ stream
only)
Implementation
Peer Assists are best done before implementation begins, while plans
are being drafted or can still be changed and while the project team is
likely more willing to consider new approaches – i.e., ‘learning before
doing’. The learning group is composed of 1 - 6 persons combined total
for learning and advisory groups, and including a rapporteur, working
on the same project/ initiative. The advisory group (of similar size)
Detailed
consists of persons who have solid experience in facing comparable
description
challenges or solving similar problems. Advisory group members are not
necessarily from the same team; they may be from different teams or
even different organizations. A facilitated activity, the Virtual Peer Assist
is conducted online via web conferencing tools (such as Skype for
Business, Adobe Connect, etc.). It should be considered especially when
other sources of project advice (such as existing documentation,

100
 Knowledge Exchange communities, etc.) cannot offer the breadth and
depth of perspectives needed.
Peer Assists offer several advantages. The first of these comes from the
‘peer’ nature of the process: The advisory group and the learning group
usually consist of people with similar roles, backgrounds and common
interests, i.e. peers. These commonalities may make the learning group
more receptive to good advice and suggestions from the advisory
group. The online format of the Virtual Peer Assist enables organizers to
invite participants in any location. This may make it easier to cast a
Benefits wide net and find people with the right experience, even from other
organizations. It is usually easier (and much less expensive) to get a
couple of hours of online participation from a busy expert, than it is to
book their time for 1-2 days and fly them to a face-to-face meeting.
Peer Assists are also learning opportunities for the advisory group, who
get the chance to see their ideas discussed, adopted and applied in new
contexts. (Follow-up between the two groups is important for this to
happen).
Virtual Peer Assists require considerable coordination to set up.
Stick to problems or challenges that can be clearly defined: A Peer
Assist is not suitable for extremely complex problems that are difficult
Risks
to define, have too many unknown variables, or lack limits or end
points. In such cases the peer assist process will not produce tangible
solutions.
Offline: Flip chart for each Peer Assist group, Markers and pencils,
instructions for the Peer Assist method, Computers with internet access
(if required).
Required tools
Online: Web conferencing platform
For each participant: Computer and headset with microphone
T support
60-120 minutes. Really complex projects may require a face-to-face
Peer Assist, taking half a day or longer-
Agenda suggestion for a 1 hour 30 minutes event:
• Introduce the session and divide into groups (10 minutes)
• Facilitator explains process and roles (5 minutes)
Timeframe
• Peer assistee presents the case (5-10 minutes)
• Discussion and facilitation (45 minutes)
• Validate notes and plan follow-up (5 minutes)
• Plenary debriefing (after multiple Peer Assists) (15 minutes)
• Close the session (5 minutes).
Required skills 1 Facilitator, 1 rapporteur, ideally from the learning team
and resources
Comments Could work for Communicator’s Lab (see EFSA SEA approach)
http://www.fao.org/elearning/course/FK/en/pdf/trainerresources/PG_Pe
erAssist.pdf
Useful links
https://www.unicef.org/knowledge-
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf

101
 https://maarifa.ilri.org/2012/09/06/online-peer-assists-learning-about-
Examples concrete-solutions-and-better-questions-for-water-and-land-
management-researchers/

PESTEL and SWOT combined

General outline
Method PESTEL Analysis
PESTEL, a complementary method to SWOT, expands on the analysis of
external context by looking in detail at specific types of issues that
frequently have an impact on implementation of project/initiatives. The
term ‘PESTEL’ refers to the domains it considers: Political, Economic,
Social, Technological, Environmental and Legal. PESTEL involves
identifying the factors in each of these six domains that are relevant for
the project being considered. A special focus of PESTEL is identifying
Short trends. Thus, it is helpful for thinking proactively and anticipating
description change, rather than being overtaken by it. It is recommended to use
PESTEL and SWOT together. PESTEL complements SWOT by identifying
specific relevant factors (such as economic trends, social attitudes,
technological developments, etc.) that are significant for the project
being considered, and SWOT then classifies them as either
Opportunities or Threats. The more complex your context or operating
environment is, the more value PESTEL can offer, by identifying factors
that would be missed by SWOT alone.
• For strategic planning and organizational development
Objective
• To incorporate different perspectives on a situation
Target Researchers, users, industry, NGOs, risk assessors
Geographical
Global, European, Local
scope
Online/offline Offline and online
Impacts Consult, Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do
science’ stream
only)
Implementation
PESTEL analysis in each of the six domains: Political, Economic, Social,
Technological, Environmental and Legal. Share them with those who will
Detailed
be conducting the analysis, and ask them to identify specific relevant
description
examples in the operational context of the project or initiative that you
are considering. Remember, all PESTEL factors have relevance only in

102
 the specific operational context. Therefore, identify the ones that could
impact your project and focus your analysis on them.
Applying PESTEL is fairly simple: Of the nine steps to do a SWOT, only
steps 2 and 5 are done differently when using PESTEL. An extensive set
of PESTEL questions is provided to help participants identify more
quickly and easily the relevant factors in each of its six domains. If
you’re short of time, you can just do a SWOT. But if time permits, then
applying PESTEL and SWOT together results in a stronger analysis, a
better understanding of the current situation, and the potential for
improved decision-making. SWOT and PESTEL are flexible: They can be
applied for planning or decision-making concerning an entire
project/initiative, or alternatively it can be used to focus on specific
stages or components of a project
SWOT and PESTEL can be applied to large or small (but significant)
projects or decisions. When assigning persons to do PESTEL research
before the SWOT session, try to match the PESTEL domains with
persons who have knowledge of those domains. Thus, a media expert
would be strongest in the Social domain, a lawyer or someone with
legal background in the Legal domain, etc. Those who do the PESTEL
analysis should also participate at the SWOT so that they can explain
and support their choice of factors.
Creating, or helping create, a strategic plan or an action plan when
launching a project/initiative.
Benefits Weighing the pros and cons of major decisions.
Reviewing positioning on an ongoing project/initiative at a key moment
of reflection.
The method should not be used as an end in itself. Its value lie in using
Risks
it as a step in a process.
Flip charts, paper hanging on the wall, a white- board, colored cards,
Required tools
copies of the two-by-two table, Pencils and markers.
If time is very limited, or for small projects, do a quick SWOT in an hour
(remembering to identify the Opportunities and Threats first, and then
the Strengths and Weaknesses). With more time, or for
Timeframe projects/decisions with larger implications, do a full SWOT and PESTEL
in about 3 hours, plus preparation time. With even more time, or for
very significant projects/decisions, expand the time accordingly, up to a
full day workshop.
Required skills
Facilitator, rapporteur, supporting team
and resources
This reflection could be embedded in the evaluations made at the early
Comments
stages of a mandate (problem formulation)
https://www.unicef.org/knowledge-
Useful links
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Examples

103
Public consultations
General outline
Method Public Consultations
Public consultations are effective ways to collect comments from
Short
stakeholders and the public at different stages of the risk assessment
description
process.
Objective Researchers, Users, Industry, NGOs, CSOs
Target European, global
Geographical
Online
scope
Online/offline Consult
Impacts Already in place at EFSA
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Plan, Verify
science’ stream
only)
Implementation
Detailed
description
Benefits
Risks
Required tools
Timeframe
Required skills
IT tools
and resources
Comments
Useful links
Examples

Public events
General outline
Method Public events
EFSA holds a variety of digital and face-to-face public events with open
Short
registration targeting stakeholders and other interested parties. Some
description
of them – namely information sessions and scientific colloquia – are

104
 included in the list of targeted platforms of the SEA and/or in the
catalogue of support initiatives during the lifecycle of applications for
regulated products. Others are managed as standalone activities.
Formats include:
Scientific colloquia: meetings aimed to explore cutting edge research,
emerging risks, science or methods not yet covered by EFSA in any of
its mandates.
Scientific conferences: meetings where participants exchange
information and expertise, network with peers and showcase scientific
work in oral/poster presentations.
Information sessions: seminars aimed to transfer knowledge on
methodologies or explain EFSA’s scientific outputs (e.g. a guidance
document) to EFSA’s interested parties
Technical meetings with stakeholders: meetings hosted in
conjunction with a public consultation to present the scope of the
consultation itself and/or discuss its outcome with all the contributing
interested parties.
Webinars (digital information sessions): web-based seminars
aimed to explain a scientific output or methodology to a virtual
audience of stakeholders and promote its understanding.
Workshops: interactive meetings aimed to engage participants in
intensive discussion on a topic or a test case and collect input to inform
a (scientific) process.
Objective
Target
Geographical
European, global
scope
Online/offline offline / online
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify, Report
science’ stream
only)
Implementation
Detailed
description
Benefits
Risks
Required tools
Timeframe
Required skills
and resources

105
Comments
Useful links
Examples

Q-methodology stakeholder selection

General outline
Method Q-methodology stakeholder selection
Controversial issues in public debates involve stakeholders and experts
with a wide variety of viewpoints. The Q-methodology (also known as 'Q
Method', 'Q Méthode' or 'Q Méthodologie') is a research tool from the
social sciences which can be used to gain insight into the diversity of
perspectives. Furthermore, it can be used to select relevant participants
Short for further dialogue about the issues at hand. When organising a
description dialogue, it is fundamental to facilitate the meeting of stakeholders with
diverse viewpoints on the issue under discussion. Often the assumption
is made that by selecting participants on the basis of their affiliation, a
wide range of views is represented. The Q methodology can be a tool
for stakeholder selection where the emphasis lies on the representation
of diverse perspectives in the dialogue.
Gain insight into the diversity of perspectives of stakeholders in a
Objective controversial issue.
It can be used to select relevant participants for further dialogue.
Target Researchers, policy-makers, risk assessors, academia
Geographical
National, regional
scope
Online/offline Offline
Impacts Consult
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
The Q methodology involves three main steps:
• Definition of the concourse
• When looking at a specific issue, the ‘concourse’ is the sum of all
Detailed the statements about the issue.
description • Interviews and perspective identification
The sample of statements collected in the first phase is presented to the
interviewees, who each make a Q-sort. This is a ranking of the
statements in the Q-set according to their personal agreement or

106
 disagreement with the statements. At the end of each of the sessions
the interviewer has two sets of data, the Q-sort and the narrative where
the interviewee explains their choices.
Gaining an overview of a variety of perspectives in public debates about
Benefits controversial issues, which allows the creation of a dialogue among
stakeholders with different perspectives.
This method is not suitable for finding out the level of support for a
specific perspective. There is some measure of the extent to which they
are represented with various stakeholders, but due to the purposive
Risks sampling there is no measure of the extent to which each perspective is
supported by a wider public.
Compared to other more straightforward approaches to selecting
stakeholders, this is a time-consuming process.
Required tools
Timeframe Weeks
Subject-matter expertise: Intermediate
IT skills: No such skills required
Required skills
Facilitation skills: Intermediate
and resources
Event organisation skills: No such skills required
Project management skills: Basic
Interesting for mandates and topics where value based judgement lead
Comments
to polarised an conflicting views (pre-mandate engagement)
http://actioncatalogue.eu/method/7436
Useful links
https://en.wikipedia.org/wiki/Q_methodology
https://research.vu.nl/en/publications/q-methodology-to-select-
Examples
participants-for-a-stakeholder-dialogue-o

Reflexive interactive design

General outline
Method Reflexive interactive design
The reflexive interactive design process consists of different stages,
which could be seen as separate methods; the combination makes it
unique and effective. This method is applied when sectors have arrived
Short
at a lock-in situation, where different stakeholders disagree on values
description
or the nature of the problem. It is based on theories about systems
learning, systems research and takes an integrative approach on
problem-solving within unsustainable systems.
This method is applied when sectors have arrived at a lock-in situation,
Objective where different stakeholders disagree on values or the nature of the
problem.
Target Researchers, CSOs, policy makers, consumers, industry, risk assessors
Geographical National
scope

107
Online/offline Offline
Impacts Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
The method consists of three steps:
• Interviews are done with numerous stakeholders in the sector,
as well as NGO’s and the relevant ministry, to identify the
sustainability problem(s) of the subsectors.
• Collective System Analysis (CSA): After the interviews are done,
a workshop is organised. This CSA workshop is attended by
participants that showed a willingness and ability to innovate
and think outside the box during the interview. The aim is to get
insight into the whole production-consumption system and
especially to identify the main points where this system is
blocking innovation and where possibilities for innovation lie. In
the workshop, all participants write down the barriers they feel
are blocking sustainable development on post-its. These post-its
are placed on an Innovation Systems framework (a matrix
Detailed showing the entire sector and all its interactions), while the
description participants explain them to the rest of the participants. Then,
all participants reflect on the barriers listed, trying to determine
the main underlying causes. In the next round, the same
procedure is followed for current developments in or outside the
sector, that offer windows of opportunity for innovation towards
sustainability. At the end, one of the group members presents
the results to the other groups in a plenary discussion and
possible actions for improvement are proposed.
• Design Atelier(s): The interested participants from the CSA
workshop, then come together for two days to: 1. Identify what
they feel are important characteristics of a sustainable
production system; 2. Design a production system that meets
these demands. An artist is present and draws these designs at
the end of the day. In a plenary discussion these designs are
assessed and pros and cons of the designs are identified.
It brings together different parties that normally would not choose to sit
together and innovate together. Also, because of the strongly
participative character of the design process, the participants are more
Benefits
likely to accept the final design and have a sense of ownership. Because
of this, they are stimulated to actually translate the new ideas into real
initiatives in the field.

108
 There is a risk that the innovations that were designed in the Design
Ateliers are just that: designs. If they stay only theoretical and on
paper, these do not mean much. The challenge lies in taking this
method one step further and using these designs in further research
and innovation steps. In most of the projects listed at the end of this
fact sheet, one or more of the designs that came out of the design
atelier have been taken into practice and are being tested. Crucial to
this method is the selection of participants. The willingness of the
participants to cooperate and to find solutions together is a key
condition for the workshops to be successful. Some stakeholders in a
sector may not feel the need or may not be able to think outside the
box. A balanced group of participants is important, but researchers
Risks
have pointed out that creating a completely representative group is not
only impossible, but may also be inefficient when trying to create
innovative designs over a short period of time. Preliminary interviews
with potential participants serve to identify those parties that show a
willingness to cooperate, innovate and think outside the box. These
have an important role in choosing the right design atelier participants.
Knowing the sector in which you want to innovate and from which you
want to choose participants is important, in order to be able to identify
the bottlenecks in the sector in which innovation is being blocked or
where there are possibilities for further innovation. So, overall, the
preparation of the design atelier is essential in creating an efficient
design process and a truly innovative product.
Required tools
Timeframe From 2 to 6 months
Subject-matter expertise: Advanced
IT skills: Basic
Required skills
Facilitation skills: Advanced
and resources
Event organisation skills: Intermediate
Project management skills: Intermediate
Interesting for mandates and topics where value-based judgement lead
Comments
to polarised and conflicting views (pre-mandate engagement)
Useful links http://actioncatalogue.eu/method/7437
Examples http://www.wageningenur.nl/nl/show/Varkansen-1.htm

Research agenda camp

General outline
Method Research agenda camp
Aim of the method: to develop “research programme scenarios” i.e.
suggestions for research programmes addressing the underlying
Short societal needs, aspirations and commonalities developed in a previous
description phase. Note: it is not a method on its own, it has to be integrated in a
larger process. Based on the previous work, each need or aspiration is
presented with a description and related extracts of citizen visions to

109
 illustrate the needs. The process will lead the group from a need or
aspiration expressed by citizens to the description of a research
programme addressing these needs and aspirations. This process is a 2-
day co-creation workshop, with participants working in small groups
organized by tables.
Identify potential research topics. Gather researchers, policy makers,
and day-to-day citizens who have been working on visions of a desired
Objective
future. It may change the perspective of each group on the others and
on research.
Target Policy makers, researchers, citizens, consumers, academia
Geographical
EU, national, regional
scope
Online/offline offline
Impacts Consult, Involve
Engagement
‘Preparadness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan
science’ stream
only)
Implementation
The process will lead the group from a need or aspiration expressed by
citizens to the description of a research programme addressing these
needs and aspirations. PARTICIPANT SELECTION The selection of
participants from the following two groups is made as follows: For the
citizens if you organised several workshops to build visions (created the
needs), select participants from every workshop. The experts’ as well as
stakeholders’ field of expertise shall be as broad as possible to open up
as many options as possible for the research programme development.
Nevertheless, make sure you have some experts with area of expertise
related to each of the social needs. We recommend to recruit 10 to 15
citizens and 10 to 15 experts. The roles of the different groups of
Detailed participants are defined as follows:
description • The role of the citizens is to reinforce the authenticity of the
messages coming from the original visions elaborated in the
workshops, and, in addition, to bring their everyday experience
and insights at the table.
• The role of the experts & stakeholders is to bring their scientific
knowledge and to contribute especially if there are several key-
steps, such as ‘Finding Influencing Factors’, ‘Formulating
Research Questions’ and ‘Building the Research Agenda
Scenario’.
• The role of your team members is double: from one side they
participate by bringing their knowledge and experience, on the
other side they act as ‘hidden’ connectors among experts and

110
 citizens, as they are highly committed into the project and
interested in high quality results.
• The role of the facilitator is to supervise the whole process, to
organize the group work, to fill templates (or supervise this
activity, always checking that each template is properly filled
with the reference to the social need and table number), to offer
valuable suggestions / solutions especially when the table is
experiencing some troubles in content defining.
It produces potential research topics based on experience and
Benefits
perspective of the different categories of actors involved.
Attention is necessary to keep all the research topics connected to the
Risks
original visions
Required tools
Timeframe 1 day event
Subject-matter expertise: Intermediate
IT skills: Basic
Required skills
Facilitation skills: Advanced
and resources
Event organisation skills: Intermediate
Project management skills: Intermediate
This method could work for specific research events (e.g. RARA - Risk
Comments
Assessment Research Assembly)
Useful links http://actioncatalogue.eu/method/7445
This method was applied by the EU funded project CIMULACT
Examples
http://www.cimulact.eu/

Science Café
General outline
Method Science Café
Short Event organized in an informal setting as a place of dialogue with
description participants coming from all walks of life and academia.
Engage people in a conversation about the issues in science and
Objective
technology that affect their lives.
Target Academia, Consumer, Students, Researchers
Geographical
Local
scope

Offline (with the possibility of having persons connected remotely with

Online/offline
audio and video streaming and chat room)

Impacts Involve

111
Engagement
‘Preparadness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Report
science’ stream
only)
Implementation
Worldwide, science cafés continue to rise in popularity. Since 1998,
when the first science café were organized both in the UK and France,
all over the world science cafés have been set up with an estimation of
about 700 science cafés in 2014. In science cafés, both scientists and
the public can meet in an informal environment and participate in
discussions about science issues
Through this method, for the price of a cup of coffee or a cup of tea,
anyone can come to explore the latest ideas in science and technology.
In this way, the academic context becomes a bar where scientific
experts are invited to give a short talk and then the floor is open for
Detailed
discussion.
description
The café format is very flexible and adapts to many different purposes,
information sharing, relationship building, deep reflection and action
planning. This method is particularly effective in surfacing the collective
wisdom of large group of diverse people.
Participants in science café events can gain new knowledge and
perspectives on a certain topic through their interaction with the
experts and the rest of the attendees. In addition, participants, can also
get informed on alternative views and relevant narratives, especially
when the events focus on controversial issues, often raise new
questions.
Informal method to exchange face-to-face opinions. Inexpensive to plan
Benefits and run.
This method is suitable for ´Every subject under the sun!’
Risks
• Small tables
• Chairs for participants and presenters
• Flip chart paper
Required tools
• Markers
• Larger paper for harvesting collective knowledge
• Posters showing the “Cafè Etiquette”
Total session: 1 hour (maximum) including presentations by speakers.
Timeframe These could be around 5 minutes, even if some facilitators prefer
presentations without slides to encourage a more informal interaction.
Usually one expert speaker is needed, but there are also models with
Required skills multiple experts. A key ingredient is the presence of a moderator who
and resources should also train the experts to ensure there are lively and useful
discussions.

112
 Interesting to raise awareness and promote the understanding of the
Comments
risk assessment process.
Useful links Resources from EU-funded projects: SciCafe, Cafe 2.0
Examples https://inspiresproject.com/about-inspires/

Science shops
General outline
Method Science shops
Short Small entities carrying out scientific research in a wide range of
description disciplines on behalf of citizens and local civil society
Provide civil society with knowledge and skills through research and
Objective education; Enhance understanding among policy makers and education
and research institutions.
Target Civil society, policy makers and educational entities
Geographical
National, local
scope
Online/offline Offline
Impacts Involve
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Report
science’ stream
only)
Implementation
The core activities of universities are teaching and research, but many
have a third mission to transfer knowledge to society. The democratic
idea is that research should be accessible to everyone, including civil
society organisations and non-profits (complementary to curiosity
driven or commercial research). A Science Shop thus is a unit that
provides independent and participatory research support in response to
concerns experienced/expressed by civil society. Science Shops were
established in the 1970s in the Netherlands (with similar developments
Detailed
in e.g. Canada and USA), and are now active in many countries.
description
Civil society driven research leads to interesting research topics for staff
and students, and offers social and political learning for students, next
to developing problem-solving skills. It offers good PR for the
university. This is a win-win-win situation. Policy makers benefit from
additional knowledge to base decisions on. In an initial-meeting, the
research objectives and time frame are agreed, expectations managed,
and sources of knowledge identified. The CSO participates in the
sounding-board of the project. Results are made public. Through this

113
 co-operation, the research is both independent and participatory.
Further involvement of the CSO is possible, depending on the context
(Community Based Research, Citizen Science).
In university-based Science Shops, the university has final responsibility
for a product abiding by academic quality standards. Other Science
Shops are stand-alone organisations, who usually work in partnership
projects with CSOs and research institutes, or perform part of the
research themselves. Responsibilities are distributed within the team.
Because at universities’ Science Shops the research is mostly done in
the curricula, there are low costs involved. Mostly, bachelor or master
thesis research is used to perform research for a CSO. For professors,
supervising this research counts towards their teaching hours. At the
same time, working with students also has limitations, especially in time
planning. When additional funding is available, researchers can be
hired. The Science Shop, as infrastructure, offers an existing network of
CSOs in the region, in which trust relations have been established.
When starting from scratch, a needs survey among CSOs can be done,
to see if the expressed needs match research interest/capacity within
the institute, or the consortium submitting a research proposal. Science
Shop staff have good experience in process management of these co-
operative projects.
The method combines different types of knowledge, builds on issues
defined by civil society, and makes output usable to civil society.
Benefits
Additional benefits are co-creation of knowledge, empowered CSOs,
motivated students, and PR for the involved research institute.
Risks
Required tools
If infrastructure already exists, projects may be set up in a time frame
of 3-6 months, though availability of students may prolong the time
Timeframe frame with another 6-12 months. It takes 1-2 years to start a full
Science Shop as infrastructure. Maintaining contacts is a continuous
effort.
Required skills
Advanced project management and facilitation skills required.
and resources
Comments
Toolbox on the International Science Shop Network
Useful links
https://www.livingknowledge.org/resources/toolbox/#c997
Science Shop Austria - Austria, “Wissenschaftsladen”, the The Bonn
Examples Science Shop - Germany
https://inspiresproject.com/about-inspires/

Science week
General outline
Method Science week

114
Short Science Week is a method to communicate science to a wide target
description audience, especially students.
Create enthusiasm for science, technology and health among children
Objective and the youth, and to strengthen and develop interest in the science
curricula in primary, secondary and upper secondary schools.
Target Researchers, citizens
Geographical
National
scope
Online/offline Both
Impacts Inform, Involve
Engagement
stream(s)
Engagement
window
(‘Quality of
science’ stream
only)
Implementation
Science Week is an annual event. However, it can also be viewed as an
approach consisting of many methods which aim at communicating
science to students in primary, secondary and upper secondary schools.
Every year a different theme should be chosen.
The organizer (EFSA) produces a catalogue of ideas which comprises a
number of different activities that the participating schools can carry
out. The organizer coordinates these activities, and sees to it that the
Detailed
event is introduced to schools and the press. The activities are
description
developed in collaboration with many different stakeholders, for
instance universities and companies.
Overall, EFSA should provide the setting for the activities. In practice, it
is the teachers who organise the event at their local schools and apply
the methods with their pupils. There is no attendance fee, and the
individual participating school decides how many of the proposed
activities they want to carry out or participate in.
The method reaches a large audience.
The partnerships with universities, companies, etc. provide an
opportunity for children and young people to realise how science is used
Benefits in real life. The method contributes to building bridges between schools
and society.
The method is really flexible. The secretariat leaves it to the schools to
decide how, and to what extent, they want to participate in the event.
Science Week organiser isn't in control of the direct application of the
method(s). It is up to the local teachers and schools to make their
Risks Science Week a success.
It can be difficult to measure direct results of the application of the
method.

115
 Finances are a main concern when applying this method. Organisers will
have to spend a lot of time on fund-raising.
Required tools
The theme is planned three years ahead. It takes place on the same
Timeframe
week every year.
Subject-matter expertise: Intermediate
IT skills: Advanced
Required skills
Facilitation skills: Intermediate
and resources
Event organisation skills: Advanced
Project management skills: Advanced
Networking is a keyword. The successful application of the method
depends a great deal on establishing external partnerships with
universities, companies, etc.
Comments
It would be used to engage locally to raise awareness and promote the
understanding of the RA process. The feasibility of this method depends
on available resources (this is really demanding)
Useful links http://actioncatalogue.eu/method/7414
Examples Danish Government

Scoping study
General outline
Method Scoping study
Literature review of published and grey literature, followed by focus
group and interview consultations: scoping studies aim to map key
concepts underpinning a research area and the main sources and types
of evidence available. They include a literature review and consultation
Short phase that may be used to (a) examine the extent, range and nature of
description research activity, (b) determine the value of undertaking a full
systematic review, (c) summarize and disseminate research findings, or
(d) identify research gaps in the existing literature. For example, a
scoping study may start with a literature review followed by a series of
focus groups and key informant interviews to prioritize research.
The purpose of a scoping exercise is:
• To map a wide range of literature, and to envisage where gaps
and innovative approaches may lie.
• To examine the extent, range, and nature of research activity.
Objective
• To determine the value for undertaking a full systematic
review.
• To summarize and disseminate research findings.
• To identify research gaps in the existing literature.
Target Researchers, academia, risk assessors
Geographical
International
scope

116
Online/offline Online/offline
Impacts Inform, Consult
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Do
science’ stream
only)
Implementation
Scoping studies are concerned with contextualizing knowledge in terms
of identifying the current state of understanding; identifying the sorts of
things we know and do not know; and then setting this within policy
and practice contexts. Scoping reviews are exploratory projects that
systematically map the literature available on a topic, identifying the
key concepts, theories, sources of evidence, and gaps in the research.
They are often preliminary to full syntheses, undertaken when feasibility
Detailed
is a concern -- either because the potentially relevant literature is
description
thought to be especially vast and diverse (varying by method,
theoretical orientation or discipline) or there is suspicion that not
enough literature exists. These entail the systematic selection,
collection and summarization of existing knowledge in a broad thematic
area for the purpose of identifying where there is sufficient evidence to
conduct a full synthesis or where insufficient evidence exists and further
primary research is necessary.
• Provides overview of state of evidence in a field
• Includes published and unpublished literature
Benefits
• Includes a wide range of studies design and methodologies
• Tools for mapping broad and diverse topics
• Difficulty establishing boundaries with broad scope
• Lack of detailed methodological steps, guidance, standards
Risks
• Unclear how to interpret scoping evidence with lack of quality
appraisal
Required tools
Timeframe Months
Required skills
Rapporteur
and resources
https://implementationscience.biomedcentral.com/articles/10.1186/174
Comments
8-5908-5-69/tables/1
Useful links http://actioncatalogue.eu/method/7414
Journal of Clinical Epidemiology
Examples
https://www.jclinepi.com/article/S0895-4356(18)30754-6/fulltext

117
Stakeholder working groups
General outline
Method Stakeholder working groups
The method is designed as a workshop that enables focused discussions
between different groups of stakeholders. The method consists of five
Short
steps (information, selecting topic, discussion, deliberation, and vote) of
description
which some can be repeated if more than one research scenario is to be
enriched by each group.
• Bring together stakeholders with different points of views on the
selected research scenarios and prompt them to answer a series
of questions in order to generate rich group discussions that can
feed into the process of producing a final research programme.
Objective • To have stakeholders to react to each other and bring up
questions, knowledge and insights into a specific research
scenario for each group.
• To help prioritizing the most promising enriched research
scenarios.
Target Researchers, policy-makers, NGOs, industry, academia, risk assessors
Geographical
National, regional, local
scope
Online/offline Offline
Impacts Consult, Involve
Engagement
‘Preparadness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify
science’ stream
only)
Implementation
This method brings together stakeholders with different points of views
on the selected research scenarios and prompt them to answer a series
of questions in order to generate rich group discussions that can feed
Detailed into the process of producing a final research programme. The main
description objective is to have stakeholders to react to each other and bring up
questions, knowledge and insights into a specific research scenario for
each group. The working groups help prioritizing the most promising
enriched research scenarios.
• The method is flexible (1 day/half day or repeat the
consultations according to number of research scenarios) which
can be advantage for recruiting stakeholders.
Benefits
• The research scenarios can be (elaborately) enriched by the
diverse views and knowledge due to stakeholder’s expertise
• Stakeholders tend to be very productive

118
 • As any other method with stakeholders, it is more difficult to
motivate them to participate at the workshop.
• If table facilitators are not well prepared (familiar with the
method/process) the result does not have to be beneficial
Risks • It’s extremely difficult to convince a high number of
stakeholders to attend in a single day consultation. Even if they
tell you that they will attend, there is a high probability that
they will not attend. You must be prepared for this, by inviting a
higher number of people than the minimum you wish to achieve.
Required tools
Timeframe The workshop lasts 6 hours
Subject-matter expertise: Intermediate
IT skills: Basic
Required skills
Facilitation skills: Advanced
and resources
Event organisation skills: Advanced
Project management skills: Advanced
Comments Already in place at EFSA (Stakeholder Bureau, discussion groups)
Useful links http://actioncatalogue.eu/method/7446
Examples

Swot Analysis
General outline
Method Swot Analysis
Acronym for Strengths, Weaknesses, Opportunities and Threats in
relation to a situation, activity or programme. A SWOT analysis is a
well-known strategic planning tool to discover weaknesses and
strengths of an individual, group or organization, and to identify both
Short
potential opportunities and threats. A SWOT analysis in a multi-
description
stakeholder partnership can be an effective way to review strategies
participants are developing, or to evaluate an implemented activity. The
tool helps participants be realistic about what they can achieve and
where they should focus.
• To carry out an environmental scan
• To make an internal assessment
Objective
• For strategic planning and organizational development
• To incorporate different perspectives on a situation.
Target Researchers, academia, users, industry, consumers, risk assessors
Geographical
Global, European, Local
scope
Online/offline Offline (also online SWOT is possible)
Impacts Consult, Involve

119
Engagement
‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Verify
science’ stream
only)
Implementation
A SWOT analysis can be done as a brainstorm in a small group or
workshop setting. Make sure your SWOT is definable, measurable, and
clear from the beginning.
How to use this method:
• Design four series of SWOT analysis questions to explore
internal external factors that may affect a case, situation or
challenge. Internal factors that generate strengths and
weaknesses may include human resources, finances,
organizational structure and priorities, and institutional culture.
Common external factors that affect opportunities and threats
include the political, social and economic context, and
technological advances or limitations.
• Create a blank two-by-two table to capture factors having a
potential impact on the situation. Questions that guide the
participants may be included in this table, or provided in flip-
charts or handouts.

Detailed
description

• Create small working groups and instruct each group to appoint

a facilitator and a note-taker. The facilitator must guide the
discussion while remaining impartial and supportive.
• Rotate among the groups to check that they are focused, and to
provide assistance if required. A bell or other sound marking
each quarter of the allocated time can help participants to
manage their time. The note-taker will add key points into the
matrix as the group agrees on them.
• Conclude the working session and have the groups report back
in plenary.

120
 • Lead a collaborative discussion that identifies priority areas for
action. Record the key reflections and next steps.
• 8. Ensure appropriate follow-up to the activity.
Ideally the group should be composed of 8 - 12 participants
representing diverse relevant roles and ideally including
decisionmakers. Alternatively, up to 40 participants if using subgroups.
Online SWOT: If your participants have adequate internet connections,
you can convene a SWOT in a web conferencing tool (e.g. Skype for
Business, Adobe Connect, GoToMeeting, etc.). Use audio, not text chat,
to gather inputs, but prefer no video unless all participants have
excellent bandwidth. Do not exceed 10-12 participants. Check
periodically.
SWOT is an adaptable and flexible method, allowing for different
perceptions to be recorded, and it directs the attention of those
involved towards joint action. This method is useful to encourage many
people to share their inputs, helping them think about potential
Benefits solutions and constraints, for example, as part of a strategic planning
process. SWOT can also take past mistakes or weaknesses and
transform them into constructive learning processes. It can help make
complex problems easier to deal with, in the shortest time possible. It is
a useful starting point for a group self-evaluation.
The method should not be used as an end in itself. Its value lies in
using it as a step in a process.
The more subtle aspects of a scenario should not be neglected, as they
can sometimes be overlooked in favor of grand or dramatic SWOT
Risks indicators.
Ensure that the participants devote an appropriate amount of time to
each of the four areas to create a balanced picture of the issue.
There are limitations in SWOT, it will not fix anything unless you aim to
actually apply and utilize what you have defined.
Flip charts, paper hanging on the wall, a white- board, colored cards,
Required tools
copies of the two-by-two table, Pencils and markers
1 hour for quick SWOT; 2 hours for normal SWOT, or up to a half-day
SWOT workshop for major initiatives; preparation time must be added.
Suggestion on the break-down of activities for a 1h30 session:
Timeframe • Introduction to the method (5 minutes)
Individual reflection and preparation (15 minutes)
• Group based work on the four factors (45 minutes)
• Collective debriefing (25 minutes)
Required skills
Facilitator, rapporteur, supporting team
and resources
This method could be embedded in the evaluations made at the early
Comments
stages of a mandate (problem formulation)
http://acnudh.org/wp-content/uploads/2011/11/slitoolkit.pdf
Useful links
http://www.mspguide.org/tool/swot-analysis

121
 Payments for Forest Ecosystem Services SWOT Analysis and
Possibilities for Implementation
Examples
https://ec.europa.eu/jrc/en/publication/payments-forest-ecosystem-
services-swot-analysis-and-possibilities-implementation

TOP 100 list

General outline
Method TOP 100 list
The TOP 100 list is an approach for capturing a high level and large
number of ideas and a technique that calls for participants’
Short
contributions. The technique is very simple in principle: state your issue
description
or question in the top of a blank sheet of paper and come up with a list
of one hundred answers or solutions about it.
• To generate ideas
• To encourage creativity and thinking out of the box
Objective • To brainstorm with structure
• To break the ice
• To activate past learning.
Target Researchers, CSOs, Users, Industry, consumers, citizens
Geographical
Global, European, Local
scope
Online/offline Offline
Impacts Consult, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
• Begin by identifying the issue to be tackled with a list of possible
solutions and related ideas.
• Prepare an adequate space in the room to make the Top 100 list
accessible to all (with flip charts, paper hanging on the wall, a
white- board, coloured cards, or a collective notepad).
• Write the problem at the top of the working space, followed by
Detailed
the numbers 1 to 100. The high number is what makes the
description
method effective. It forces a profound level of reflection that
reaches all corners of the mind.
• Ask participants to come up with their ideas as quickly as
possible. All ideas should be recorded, even if they seem
obscure or irrational. The first 30 ideas tend to be the obvious
ones, as they stem from the recent memory or most-repeated

122
 experiences of participants. The next 40 ideas will begin to
demonstrate patterns and trends. These ideas tend to be the
most difficult to generate because they require diverging from
the habitual approach. The final 30 entries are often the most
imaginative and innovative, perhaps even absurd, because by
this time the most common options have already been
exhausted. This is the most profitable phase of the process,
where shifts in perspective are most likely to occur.
• Lead a reflection process once 100 ideas have been produced.
This should examine the general trends and patterns, as well as
the plausibility of the entries themselves.
• The information can then be used in a variety of complementary
exercises that analyze and use the information produced. One
approach is to cluster and then prioritize the points, as after a
brain- storming session.
• Reproduce the list in a reusable format in order to include it in
the end-of-activity report, or on the Intranet.
Almost any subject can be addressed with a Top 100 List. Whether you
do this on an individual or group level will depend on the objectives.
The Top 100 List can also be used as a “background activity” during a
learning event. Start the Top 100 list on a flipchart and make it
available for participants to write down ideas throughout the duration of
the activity. Participants may come up with ideas during the activity,
particularly during breaks, they can write these down on the list. This
ensures that loose ideas that would perhaps be lost, are captured.
Instruct participants to use acronyms and short forms, and to avoid full
sentences as they consume precious time and energy.
The List of 100 is a powerful technique that can be used to generate
ideas, clarify thoughts, uncover hidden problems or get solutions to any
specific questions. The technique is very simple in principle and is a
form of cooperation between the conscious and subconscious minds
Benefits tackling one single problem. The technique is based on the concept
of getting good ideas from lots of ideas. With a List of 100 method you
tend to get more unexpected ideas, because you catch your
subconscious off guard, not giving it any time for its behind-the-scenes
editing.
• Time constraints may hinder the completion of the list. Be sure
to provide enough time to complete the list, because it is only
effective when done in one sitting or in one specific, dedicated
period (for example, over a three-day training event).
• Distractions may affect the action. Rid the room of all
Risks
distractions. This includes turning off mobile devices and
finishing any drinks.
• Repeated ideas may provide clues to the thought processes of
participants. Therefore, address them only at the end of the
session.
Flip charts, paper hanging on the wall, a white- board, coloured cards,
Required tools
or a collective notepad

123
Timeframe It can be used in the course of a 1-3 day event
Required skills
Facilitator, personnel already engaged in an event
and resources
Comments
Useful links http://acnudh.org/wp-content/uploads/2011/11/slitoolkit.pdf
Examples http://itcilo.wordpress.com/2010/09/06/100-facilitation-tips/

TOPSY TURVY (or Reverse Brainstorming)

General outline
Method TOPSY TURVY (or Reverse Brainstorming)
Topsy Turvy (also known as reverse brainstorming) is a method for
generating creative, unconventional ideas and solutions, and for
mobilizing untapped energies. It can be particularly useful for
addressing recurring problems and issues. As the name suggests, the
process involves turning a discussion ‘upside down’ by posing a
Short
question that runs counter to your real objectives, and encouraging
description
participants to contribute negative or even anarchic ideas. These ideas
are then reversed to formulate positive solutions. Taking an initially
negative or ‘destructive’ approach to the discussion gives participants
free rein to explore the flip side of an issue or problem, and thus spurs
creativity.
• To Generate original and potentially effective approaches to
Objective recurring issues or problems.
• To grasp the causes of a problem openly
Target Researchers, Users, Industry, NGOs, CSOs
Geographical
Local
scope
Online/offline Offline
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate
science’ stream
only)
Implementation
Topsy Turvy is easy to implement and can be used to:
• Generate original and potentially effective approaches to
Detailed
recurring issues or problems. Instead of asking people familiar
description
questions that generate the usual responses, Topsy Turvy helps
establish a fresh perspective and encourages new ideas or

124
 thinking outside the box. By getting people to think of how
things can go wrong, the method allows them to grasp the
causes of a problem openly, without fear of being deemed
pessimistic, and to explore negative aspects which are usually
ignored – even though they may be important to consider. In
turn, this leads to renewed interest in an issue and encourages
participants to steer away from stale, uninspired solutions.
• Break up the monotony of group activities. Schedule Topsy
Turvy between group work sessions, or slot it in just before an
afternoon group activity to raise participants’ energy levels. But
don’t overuse it: Topsy Turvy works best as a way of adding
variety, not as a steady diet.
• Kick off a workshop. Use Topsy Turvy as an icebreaker to
discuss workshop rules in the first session.
How to apply the method:
• Clearly identify the problem or issue to be discussed and on that
basis, formulate a negative question.
• Write the question on the top of a flipchart board for participants
to see.
• Reading out the question and ask participants to share their
‘destructive’ ideas without censoring their thoughts.
• Encourage the feeling of chaos by prompting participants for
anarchic, disruptive suggestions. Tell them that any ideas, no
matter how wacky, are welcome.
• As people share ideas, write them down on the flipchart. You
can also use cards to collect ideas.
• When the ideas begin to dwindle, stop the process – normally
within 10 - 15 minutes.
• Next, turn all the negatives into positives: Take each negative
idea and rewrite it as its positive mirror opposite.
The method can be applied in a group of ideally 10 - 20 people.
Instead of asking people familiar questions that generate the usual
responses, Topsy Turvy helps establish a fresh perspective and
encourages new ideas or thinking outside the box. By getting people to
Benefits think of how things can go wrong, the method allows them to grasp the
causes of a problem openly, without fear of being deemed pessimistic,
and to explore negative aspects which are usually ignored – even
though they may be important to consider.
Risks The method may initially create some sense of confusion
Flipchart board
2 flipcharts with plenty of paper
Required tools
Rectangular cards, in two colours (VIPP cards)
Marker pens
Timeframe 20 - 30 minutes
Required skills Facilitator
and resources

125
 When deciding whether to use Topsy Turvy, be sure that there is
receptivity for truly innovative solutions and approaches that could
result in substantial change. If you are only looking for limited
Comments
improvements in a process, or small fixes to a problem, Topsy Turvy
may not be the best approach; but if you want to tackle something
difficult or find a broad-ranging solution, it may be just what you need.
https://www.unicef.org/knowledge-
Useful links
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Examples

User committee
General outline
Method User committee
Short This method involves users and other stakeholders in the formal
description monitoring and steering of the research and innovation process.
Involvement of stakeholders in the formal monitoring and steering of
Objective
the research and innovation process.
Target Researchers, CSOs, consumers, industry, risk assessors
Geographical
European, national, regional, local
scope
Online/offline Offline
Impacts Involve
Engagement
‘Preparedness’, ‘Quality of science’
stream(s)
Engagement
window
(‘Quality of Pre-mandate, Plan, Do, Verify, Report
science’ stream
only)
Implementation
This method involves users and other stakeholders in the formal
monitoring and steering of the research and innovation process. The
Dutch Responsible Innovation Program (NWO-MVI) has required
valorisation panels since 2009. In the 2014 call, the following
instructions for the user-committee were given: “Applicants must
always put together a valorisation panel and produce a valorisation
Detailed
plan. Besides representatives of the private partners, the valorisation
description
panel includes all other actual and potential users and/or user groups.
Relevant societal stakeholders can also be included in the valorisation
panel. Also, representatives from organisations that are willing to
disseminate the research results and to valorise these among the target
group that they represent can be included in the valorisation panel. The
valorisation panel is put together during the drawing up of the full

126
 proposal, is involved in writing the proposal, and remains involved in
the project throughout its entire duration. More specifically, the
valorisation panel's main task is to contribute its knowledge and
expertise, and to confront the researchers with the everyday user
practice, so that the researchers can incorporate this in their choices.
At the very least it has a supportive role in:
• Articulating the research question
• Drawing up the valorisation plan
• Reporting about the research
• Disseminating and communicating the research results
The valorisation plan is aimed at making the relevant research results
available for and usable by top sectors (research priority
conglomerates, ed.), societal partners and/or other interested parties
from inside and outside of the established scientific community. Besides
an overview of the costs associated with the valorisation, it also
describes the role of the valorisation panel. Applicants of research
proposals awarded funding are required to organise an initial
valorisation workshop immediately after the start of the project. The
results of the first workshop will be monitored by the MVI Steering
Group. Applicants from projects awarded funding will receive further
information about this with the funding decision. They will also be
informed about how the valorisation pathway will be monitored
throughout the course of the project”.
Instrumental (and democratic) value in making input to research and
Benefits innovations that are in-line with users’ (and societal) needs and
demands. Applicable in any (multi-disciplinary) field.
Strongly dependent on how the engagement process within the
committee works and who is represented. Typically, there is a kick-off,
a mid-term, and a final workshop. How these workshops are shaped is
still open. Also, the consulting process during the writing of the proposal
Risks
for the research is not defined. Representativeness of the committee
(typically, there are many representatives of industry as ‘users’, instead
of the end-user (the consumer); also, CSOs are underrepresented
(typically, most NGOs that participate represent a branch organisation).
Required tools
Required preparation time totally depends on pre-existing contacts with
relevant stakeholders. The committee engagement is continuous from
Timeframe
the writing of the proposal, throughout the research activity, and
through to the dissemination phase.
Subject-matter expertise: Advanced
IT skills: Basic
Required skills
Facilitation skills: Intermediate
and resources
Event organisation skills: Basic
Project management skills: Intermediate
Comments Similar to SEA’s Bureau and Forum
Useful links http://actioncatalogue.eu/method/7441

127
 http://responsibleinnovation.eu/research/mviproject_information
https://www.nwo.nl/en/about-nwo/organisation/nwo-
Examples
domains/ttw/industry+and+other+partners/what+does+nwo+offer/use
r+committees

VIPP Cards Collection Clustering

General outline
Method VIPP Cards Collection Clustering
VIPP (Visualisation in Participatory Programmes) Card Collection and
Clustering is a facilitated process that involves the use of cards to
quickly gather and organize ideas from a group of people. It is
especially useful in situations where you expect to get a diverse range
of answers or inputs, or for addressing potentially sensitive issues in a
Short
way that is both safe and equitable.
description
The process starts with a well-defined open-ended question.
Participants write ideas onto cards and then cluster them based on
similarities. The clusters can be prioritized as a basis for subsequent
action, or they can be used for further discussion and development, as
needed.

• To introduce a topic or theme at the beginning of a workshop/

multi-session event.
• To provide an alternative to a formal podium presentation or
keynote with experts.
Objective
• To elicit knowledge from resource persons without requiring
them to do a lot of preparation.
• To introduce new concepts or potentially ‘hot topics’ that
participants may be curious about but have little knowledge of.
Target Researchers, Users, Industry, NGOs, CSOs
Geographical
Global, European, Local
scope
Online/offline Offline
Impacts Inform, Consult
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
VIPP Card Collection and Clustering is best used in group settings to
Detailed generate ideas quickly, to organize and prioritize ideas based on the
description input of the group, and to gather opinions. It is a more interesting way
to accomplish these tasks than a routine plenary meeting, and usually

128
 generates significantly better results. It also levels the playing field and
ensures greater diversity of inputs, by preventing the discussion from
being dominated by only a few voices. All ideas and inputs are
considered through the same open process.
Card Collection and Clustering is suitable for groups of 10 - 20
participants. Larger groups should first be divided into smaller ones of
no more than 20 participants; each group does the exercise separately
with its own facilitator

Variants:
Multiple questions: You can use Card Collection and Clustering to
generate ideas for multiple questions. Assign a specific colour of card to
each question, and run the questions through the process one after
another.
Preassigned categories: If you know in advance that the responses
are likely to cover certain particular topics or categories of response,
then you can pin up cards with those categories before the session
begins. Proceed with card writing as normal; then ask people to come
up individually to pin their cards under the categories.
Non-anonymous responses: If your topic clearly does not require, or
benefit from, anonymous responses, then instead of collecting
responses in a box and reading them out, you can ask participants in
turn each to read out one of their cards and suggest a category for it.
Card Collection and Clustering is beneficial for:
• Generating creative ideas at an early stage of a workshop or a
planning process, and assigning priorities. Once ideas are
collected and clustered by theme, the clusters can be used as
inputs into subsequent group work.
• Combining and harmonizing diverging ideas and opinions, for
example if your group includes individuals from different
departments, or people with differing professional or cultural
Benefits backgrounds.
• Ensuring that more unusual or creative ideas are captured and
documented, particularly if those generating such ideas are
junior or ‘outsiders’ who may feel nervous in front of more
experienced colleagues in plenary sessions.
• Helping to overcome the inhibiting influence of hierarchies, when
input is needed from many people on a sensitive topic.
• Surfacing workplace issues. People can share their views
anonymously by writing on cards.
Risks
Pin board and pins
Flipchart paper
Required tools
Rectangular cards, in several colours (10 x 20 cm/ 4 x 8 inch)
Marker pens (one colour only)
Timeframe 45 - 60 minutes
Required skills Facilitator
and resources

129
 https://www.unicef.org/knowledge-
Comments
exchange/files/UNICEF_Knowledge_Exchange_Toolbox.pdf
Useful links http://actioncatalogue.eu/method/7441
http://responsibleinnovation.eu/research/mviproject_information
https://www.nwo.nl/en/about-nwo/organisation/nwo-
Examples
domains/ttw/industry+and+other+partners/what+does+nwo+offer/use
r+committees

Vision factory
General outline
Method Vision factory
A Vision Factory is a combination of World Cafés for the future
development of complex topics, e.g. urban development or
restructuring of companies. It starts with several target group specific
World Cafés in order to consider their specific requirements and
opinions related to the topic of interest.
Short
Based on the results of these World Cafés, guiding questions for the
description
“main event” (Vision Factory) are defined.
For this event, representatives from all target groups are invited to
discuss about the topic under consideration of the specific
requirements. Result is a vision of the future for the respective topic
with a longer time horizon (5-20 years).
• To develop a long-term strategy or vision of the future for the
respective topic with a longer time horizon (5-20 years).
Objective • To create engagement and consensus on a joint vision.
• To deal with complex topics involving the interests and needs of
different target groups.
Researchers, academia, industry, NGOs, consumers, citizens, risk
Target
assessors
Geographical
Local, national, European
scope
Online/offline offline
Impacts Inform, Consult, Involve
Engagement
‘Preparedness’
stream(s)
Engagement
window
(‘Quality of Not applicable
science’ stream
only)
Implementation
Detailed Pay early attention to the reason for organizing the Vision Factory.
description Based on this, it is possible to decide which kind of target groups are

130
relevant and which parameters are important to achieve your desired
result of the method. It is very important to find and frame topics or
questions that matter to the identified target groups. The Vision Factory
conversations are about discovering and exploring powerful
topics/questions. Usually a complex topic is being dealt. Thus, the
advice is to split it in sub-topics. To each sub-topic you assign one
discussion table at the main event. Define also what kind of results can
be obtained from the method. This has impact on the definition of
guiding questions for the target group specific events.
After the definition of the purpose, the target groups have to be
determined. In this context, it is necessary to decide on a maximum
number of participants for both the target group specific events and the
main event. The former ones should have 10-20 participants, the latter
one up to 100. The implementation of target group specific events
should consider the respective characteristics of the groups. Consider
different communication channels to invite target.
Implementation of the Vision Factory:
• The event starts with a welcome from the organizer and an
introduction by the moderator to the participants. In the first
round (ca. 30 minutes), the requirements of the topic are
discussed at each table. The goal is a poster with clustered
requirements and challenges related to the table topic. This is
the starting point and basis for the development of the vision.
Participants discuss and write down the requirements and
challenges. The co-moderator of the table explains the task,
moderates the discussion and clusters the written input.
• In the second round (ca. 1 hour), the participants make a
journey into the future. They discuss and write down
imaginations and concrete ideas about the ideal state in the
topic area. The achieved future will be illustrated by examples.
The participants should agree on 3 to max. 5 key messages
(vision cores). These vision cores summarize the ideas and
stand as guiding themes above them. If necessary, participants
may use glue dots for prioritization.
• In the next round (ca. 1 hour), the visions are concretized and
visualized by the participants. The participants transfer the
vision cores to a table poster. They discuss how these
participants transfer the vision cores to a table poster. They
discuss how these visions could be achieved. The participants
then make a selection for further processing, regarding the most
important points and actions. On the table poster, they should
clarify the areas where they want to visualize something. Then
the participants should divide into (two / three) groups. Each
group takes on a specific visualization task (by handicrafts,
drawings…). Then they prepare the results for plenary
presentation and discussion and select a participant to do so.
The result is a large table poster / collage with the vision cores,
on which the most important points and actions for the subject
area are to be seen, with reference to which needs / wishes are
met here as well. Possibly also connections between these

131
 actions, points of conflict and first recommendations for action
(in the sense of a back casting, i.e. based on the desired future
condition, think of ways and instruments that could lead there).
• Afterwards, during a larger break (ca. 1 hour), participants can
visit other topic tables, have a look at the clustered ideas and
vision cores and leave comments via post-its.
• After the break, a plenary presentation and discussion of the
tables´ results takes place (ca. 2 hours). The discussion covers
the identification of special features, overlaps and synergy
potentials as well as potential points of conflict. In addition,
further suggestions, concerns and criticisms should be collected.
The participants should also comment on how they would
contribute to achieve the vision cores.
• At the end of the event, the moderator gives an outlook on what
will happen to the results and what the future course of the
project looks like.
Benefits
There could be potential for conflict between different target groups at
Risks the main event.

Provide a kind of guidelines/screenplay for the moderator(s) and table

hosts. Also, templates for documenting the discussions are helpful.
Define guiding questions for the participants to support a logical
Required tools
progression of discovery throughout several discourse rounds. Well-
defined questions focus the attention of participants to what really
matters. Open-ended questions are recommended.
3-6 months for the implementation of the method. During this duration,
the target group specific World Cafés and the Vision Factory have to be
implemented. For the organization of a single event please refer to the
method World Café regarding timeline and logistics. The difference of
Timeframe
the Vision Factory compared to the single World Cafés is the longer
duration (4-6 hours). Depending on the target groups and the
organizer, the Vision Factory should be planned for a Saturday or
Sunday.
A facilitator, owing the know-how and the experience in the
Required skills
organization and moderation of the workshop. Social scientists are very
and resources
well suited for such events. The moderation of the event is crucial.
An event organizer should be involved. It is recommended to charge a
professional moderator (as the café host) with the overall moderation of
Comments the events (World Cafés + Vision Factory).
Could work in some specific reflexions involving stakeholders such as
strategy definition
https://www.user-participation.eu/planning-the-process/step-5-
Useful links participatory-methods/planning-the-future-visions-strategies-
projects/vision-factory

132
 https://www.user-participation.eu/planning-the-process/step-5-
Examples participatory-methods/planning-the-future-visions-strategies-
projects/vision-factory/showcase-vision-factory

133
ANNEX II - Best Practices from EU and International
entities

Stakeholder Engagement – EU Agencies

European Union Aviation Safety Agency (EASA)

Engagement method Advisory Board Collaborative Platform

The European Union Aviation Safety Agency or EASA is

an agency of the European Union (EU) with responsibility for civil
aviation safety. It carries out certification, regulation, and
Agency standardisation, and also performs investigation and monitoring.
It collects and analyses safety data, drafts and advises on safety
legislation, and coordinates with similar organisations in other
parts of the world

Name of the activity Advisory Board Collaborative Platform

An online communication platform was put in place to allow for

horizontal communication by Advisory Board members. A large
number of proposed Agency actions directly affect the Member
States and the Industry. So called advisory bodies provide the
Agency with a forum for consultation of interested parties and
national authorities on Agency priorities, both at strategic and
Description
technical level. Consultation covers all aspects of the rulemaking
process. They also advise the Agency on EU wide safety
priorities, strategic and horizontal issues. They are key
contributors to the Agency Rulemaking Programme and the
European Plan for Aviation Safety

Timeframe ongoing

Link https://www.eesc.europa.eu/en

Comments

European Chemicals Agency (ECHA)

Engagement method Network

The European Chemicals Agency (ECHA) is an agency of the

European Union which manages the technical and administrative
Agency aspects of the implementation of the European Union
regulation called Registration, Evaluation, Authorisation and
Restriction of Chemicals (REACH)

Name of the activity NeRSAP

NeRSAP has been set up to exchange on advances and review of

Description
concepts, methods and experiences focused on practical

134
 concepts for Socio-economic Analysis (SEA) and Analysis of
Alternatives (AoA) on EU-wide or national chemicals
management implementation. It is set up in collaboration
between ECHA, Member States and stakeholders from industry
and NGOs.
Given REACH SEAs and AoAs are in general prepared by SEA
and AoA practitioners for their clients being it industry, Member
State authorities, ECHA, the Commission, or others, the NeRSAP
network welcomes practitioners with regulatory, academic or
consultancy background with demonstrated experience in the
field of REACH and/or EU or national chemicals legislation.

Timeframe ongoing

https://echa.europa.eu/fr/support/socio-economic-analysis-in-
Link reach/network-of-reach-sea-and-analysis-of-alternatives-
practitioners

Comments

EIP-AGRI
Engagement method Focus groups

In February 2012, the European Commission launched the EIP

Agricultural Productivity and Sustainability – otherwise known as
the EIP-AGRI. The main objective of this particular European
Innovation Partnership is to bridge the gap between agricultural
researchers and practitioners. It encourages those involved in
Agency different segments of the agri-food system (farmers, businesses,
researchers and advisors) to share their ideas and experiences.
Working together, they are expected to devise innovative
responses to problems and to develop academic findings into
practical applications, thereby “delivering solutions that are well
adapted to circumstances and which are easier to implement.”

Name of the activity EIP-AGRI - Focus Group on Protein Crops

The Focus Group on Protein Crops of the European Innovation

Partnership addressed the challenge of improving the
profitability of protein crops in Europe in order to make it an
attractive crop for farmers while satisfying the requests of the
animal feed industry (and to some extent the food industry) and
Description
promoting more technically, economically and environmentally
sustainable European agricultural
production systems. With this as objective, a group of 20
experts from across Europe assessed the challenge and
identified possible solutions.

Timeframe 2014

135
 https://ec.europa.eu/eip/agriculture/sites/agri-
Link
eip/files/fg2_protein_crops_final_report_2014_en.pdf

Comments

European Citizen Science Association (ECSA)

Network of researchers and communicators - Citizens
Engagement method
science
The European Citizen Science Association (ECSA) is a non-profit
association set up to encourage the growth of the Citizen
Science movement in Europe in order to enhance the
Agency
participation of the general public in scientific processes, mainly
by initiating and supporting citizen science projects as well as
performing research on citizen science.

Name of the activity ECSA Network of researchers and communicators

ECSA is framing citizen science as an open and inclusive

approach, for example by supporting and being part of the
exploration, shaping and development the different aspects of
the citizen science movement, its better understanding and use
for the benefit of decision making. ECSA's thematic working
groups undertake the major part of the activities. They focus on
research, exchange of experience and capacity building and are
open to our members and supporters. Membership of working
groups requires a time commitment to attend meetings and
undertake tasks.
ECSA is framing citizen science as an open and inclusive
approach, for example by supporting and being part of the
Description exploration, shaping and development the different aspects of
the citizen science movement, its better understanding and use
for the benefit of decision making.
ECSA draws on +200 individual and organizational members
from over 28 countries across the European Union and beyond.
Launched during the EU GREEN WEEK in June 2013, ECSA has
grown from an informal network of researchers and
communicators interested in Citizen Science into the European
reference network of Citizen Science initiatives. ECSA offers the
opportunity to interact among groups and disciplines that
already have or want to build a relation to citizen science,
through activities in H2020 projects, contributing to policy
briefs, the open science policy platform and being part of the
development of principles for good practice in citizen science.

Timeframe ongoing

Link https://ecsa.citizen-science.net/

Comments

136
European Cooperative Programme for Plant Genetic Resources (ECPGR)
Engagement method Online Platform

The European Cooperative Programme for Plant Genetic

Resources (ECPGR) is a collaborative programme among most
Agency European countries aimed at ensuring the long-term
conservation and facilitating the increased utilization of plant
genetic resources in Europe.
European Search Catalogue for Plant Genetic
Name of the activity
Resources (EURISCO)
The European Search Catalogue for Plant Genetic Resources
(EURISCO) provides information about more than 2 million
accessions of crop plants and their wild relatives, preserved ex
situ by almost 400 institutes. It is based on a network of
National Inventories of 43 member countries and represents an
important effort for the preservation of world's agrobiological
diversity by providing information about the large genetic
diversity kept by the collaborating institutions.
Between 2003 and 2014, EURISCO was hosted and maintained
Description by Bioversity International, Rome, Italy. Since 2014, EURISCO is
being maintained at the Leibniz Institute of Plant Genetics and
Crop Plant Research (IPK), Gatersleben, Germany. The central
goal of EURISCO is to provide a one-stop-shop for information
for the scientific community and for plant breeders. EURISCO
contains both passport data and phenotypic data.
EURISCO is being maintained on behalf of the Secretariat of the
European Cooperative Programme for Plant Genetic
Resources (ECPGR), in collaboration with and on behalf of the
National Focal Points for the National Inventories.

Timeframe ongoing

Link https://eurisco.ipk-gatersleben.de/apex/f?p=103:1:0:::::

Comments

European Commission and the European Economic and Social Committee

(EESC)
Engagement method Platform + discussion forum

The European Circular Economy Stakeholder Platform was

launched as a joint initiative by the European Commission and
Agency
the European Economic and Social Committee (EESC) in March
2017.

Name of the activity European Circular Economy Stakeholder Platform

The two institutions are working closely together to develop the

Description
Platform as a space for the exchange of ideas and a growing

137
 body of information, and to make the circular economy happen
faster to the benefit of all. The Platform brings together
stakeholders active in the field of the circular economy in
Europe. On the virtual platform stakeholders can contribute by
submitting content for the website (good practice, publication,
event, network, etc.), engage with other stakeholders on the
discussion forum and stay abreast with all ongoing activities of
the Platform by subscribing to the newsletter.

Timeframe ongoing

Link https://circulareconomy.europa.eu/platform/

Comments

Stakeholder engagement – International Agencies

The International Center for Tropical Agriculture (CIAT)

Engagement method Gamification

The International Center for Tropical Agriculture - CIAT is a not-

for-profit research and development organization dedicated to
reducing poverty and hunger while protecting natural resources
in developing countries. In 2019, CIAT joined with Bioversity
Agency
International (as the Alliance of Bioversity International and
CIAT) to "deliver research-based solutions that harness
agricultural biodiversity and sustainably transform food systems
to improve people’s lives"
Gamification of farmer-participatory priority setting in plant
Name of the activity
breeding: Design and validation of “AgroDuos”
Participatory methods to characterize farmers’ needs and
preferences play an important role in plant breeding to ensure
that new varieties fulfill the needs and expectations of end
users. Different farmer-participatory methods for priority setting
exist, each one responding differently to trade-offs between
various requirements, such as replicability, simplicity, or
granularity of the results. All available methods, however,
require training, academic skills, and staff time of specially
qualified professionals. Breeding and variety replacement may
Description be accelerated by empowering non-academic organizations,
such as NGOs and farmer organizations, to carry out farmer-
participatory priority setting. But for this use context, currently
no suitable method is available. A new method is needed that
demands relatively low skill levels from enumerators and
respondents, engages farmers without the need for extrinsic
incentives, and gives statistically robust results. To achieve
these objectives, CIAT followed principles of “gamification” in the
design of AgroDuos, a choice experiment that resembles a card
game and that involves pairwise ranking of variety traits. CIAT

138
 tested the method in a pilot with 39 farmers in Honduras to
define their trait priorities for common bean (Phaseolus
vulgaris L.). To validate the results, CIAT independently carried
out conjoint analysis, an established method for priority setting
in plant breeding. AgroDuos produced valid and useful results
while enabling rapid, easy, and engaging data collection.
Challenges persist concerning local adaptation and data analysis
by non-specialist staff, which may be resolved in the future by
providing templates and online support.

Timeframe 2017

https://www.bioversityinternational.org/news/detail/participator
Link
y-research-is-a-serious-game/

Comments

Biodiversity International
Engagement method Crowdsourcing software

Biodiversity International is a global research-for-development

organization. Bioversity International delivers scientific evidence,
management practices and policy options to use and safeguard
agricultural and tree biodiversity to attain sustainable global
Agency food and nutrition security.
The agency works with partners in low-income countries in
different regions where agricultural and tree biodiversity can
contribute to improved nutrition, resilience, productivity and
climate change adaptation.

Name of the activity Climmob

Climmob is a software for crowdsourcing climate smart-

agriculture. Climmob, created by Jacob van Etten and developed
by Bioversity International, turns the research paradigm on its
head; instead of a few researchers designing complicated trials
Description
to compare several technologies in search of the best solutions,
it enables many farmers to carry out reasonably simple
experiments that taken together can offer even more
information.

Timeframe ongoing

Link https://climmob.net/

Comments

Consultative Group for International Agricultural Research (CGIAR)

Engagement method Data Platform + digital innovation process

139
 CGIAR (formerly the Consultative Group for International
Agricultural Research) is a global partnership that
unites international organizations engaged in research about
food security. CGIAR research aims to reduce rural poverty,
increase food security, improve human health and nutrition, and
Agency sustainable management of natural resources. It is carried out
at 15 centers (CGIAR Consortium of International Agricultural
Research Centers) that collaborate with partners from national
and regional research institutes, civil society organizations,
academia, development organizations, and the private sector
focusing on agricultural research for sustainable development.

Name of the activity Platform for Big Data in Agriculture

The Platform for Big Data in Agriculture harnesses the power of

big data for agricultural research and development. It is one of
three CGIAR research platforms and it is carried out with
support from the CGIAR Trust Fund, UKAID and through bilateral
funding agreements.
The platform hosts the ‘Inspire challenge: Demonstrating the
power of big data analytics through inspiring and innovative
projects’.
The Inspire Challenge is an initiative to challenge partners,
Description
universities, and others to use CGIAR data to create innovative
pilot projects that will scale. They look for novel approaches that
democratize data-driven insights to inform local, national,
regional, and global policies and applications in agriculture and
food security in real time; helping people–especially smallholder
farmers and producers–to lead happier and healthier lives.
The Inspire Challenge team seeks co-investors and thought
partners to strengthen the incubation to acceleration pipeline for
digital agriculture: co-invest, co-create, co-lead.

Timeframe ongoing

Link https://bigdata.cgiar.org/inspire/#

The Inspire Challenge has so far awarded more than US$2.5

Comments
million in grants to 18 projects.

Engagement method Software tool for surveys – open-access dataset

Agency CGIAR

Name of the activity The RHoMIS tool

RHoMIS was designed to improve the process of gathering

information from farming households in the rural developing
Description world. Household surveys are very widely carried out, but the
data is rarely comparable, and so the opportunities for learning
between individual projects are limited. For these reasons, the

140
 tool balances standardisation with flexibility.
RHoMIS was also designed to reduce costs, time requirements,
and reporting burdens for those who carry out household
surveys. Internationally recognized indicators are used, and
reflexive learning since 2015 has led to a smooth and rapid
questionnaire, which gathers considerable detail in a relatively
short amount of time. The RHoMIS tool is built using open
source software. The survey is delivered using Android mobile or
tablet devices and the ODK software suite. Indicators are
calculated and analyses returned using the R programming
language.

Timeframe ongoing

more than 30,000 interviews conducted in 33 countries (the

publication only analyzed interviews through mid-2018). Data
are collected on tablets or smartphones, which can be solar
Link powered for use in remote regions. Broad adoption of this
standardized survey tool may help guide international efforts to
address smallholder challenges related to climate change, food
security, nutrition, farm productivity and social inclusion.

Comments https://www.rhomis.org/

Center for International Forestry Research (CIFOR)

Engagement method Knowledge Fair - Open space – Peer Assist

The Center for International Forestry Research (CIFOR) is a non-

profit scientific research organization that conducts research on
the use and management of forests with a focus on tropical
Agency
forests in developing countries. CIFOR is the forestry research
center of the Consultative Group on International Agricultural
Research (CGIAR)

Name of the activity CIFOR’s annual meeting

In 2004 CIFOR piloted an innovative format for its annual

meeting. During the previous years staff member had
commented that the institution had become to much
compartmentalized, with scientists working mainly within the
orbits of their own programs.
The event lasted 5 days:
Description • On Monday a Knowledge Fair features workshops,
presentations, poster displays, and videos relating to
research conducted by CIFOR and its partners. Parallel
workshops in the morning cover three topics.
• Tuesday and Wednesday an Open Space.
• Thursday Peer Assist and closure.
• Friday Program meetings

Timeframe Pilot activity from 2004 to 2005

141
 At the end of the meeting participants were asked to complete
an evaluation form. 72 of the 100 participants did so. By the
participants the opportunities for engaging in discussion and
Link
knowledge sharing was especially appreciated. Participants
mentioned they would like to apply peer-assists’ method to
scientific issues.

Comments Sharing Solutions for a CGIAR without Boundaries

The Food and Agriculture Organization (FAO)

Engagement method Online platform

The Food and Agriculture Organization (FAO) is a specialized

agency of the United Nations that leads international efforts to
defeat hunger.
Agency Our goal is to achieve food security for all and make sure that
people have regular access to enough high-quality food to lead
active, healthy lives. With over 194 member states, FAO works
in over 130 countries worldwide.

Name of the activity AGRIS platform

AGRIS is the International System for Agricultural Science and

Technology, a multilingual bibliographic database that connects
users directly to a rich collection of research and worldwide
technical information on food and agriculture.
One of the world’s leading public information services that offers
11,714,169 bibliographic records produced by more than 500
data providers including research centers, academic institutions,
publishers, governmental bodies, development programmes,
international and national organizations from 148 countries.
More than 400,000 agricultural and research professionals
worldwide access AGRIS resources each month.
Maintained by the Food and Agriculture Organization of the
Description
United Nations (FAO), AGRIS has been serving users from
developed and developing countries through facilitating access
to knowledge in agriculture, science and technology since 1974.
While AGRIS is predominantly a bibliographic database, it also
provides full text links to about 3 million of its records. It
facilitates access to publications, journal articles, monographs,
book chapters, and grey literature - including unpublished
science and technical reports, theses, dissertations and
conference papers in the area of agriculture and related
sciences. Most of the AGRIS records are indexed by AGROVOC,
the FAO multilingual Thesaurus. AGRIS is also indexed and made
accessible via Google Scholar, thus extending the global access.

Timeframe Ongoing

Link http://agris.fao.org/agris-search/info.action

Comments Sharing Solutions for a CGIAR without Boundaries

142
Engagement method Online survey

Agency FAO

Global Forum on Food Security and Nutrition • FSN Forum online

Name of the activity
survey
In April-May 2015 the Global Forum on Food Security and
Nutrition (FSN Forum) conducted an online survey by using the
SurveyMonkey tool. The goal was to gain a deeper knowledge of
the Members’ expectations and level of satisfaction. In addition,
this survey was also an opportunity to collect inputs on ways to
Description further improve the service offered and to gather suggestions for
topics which our Members would be interested in.
The survey comprised 17, both closed and open-ended questions
and was available in English, French, Russian and Spanish and
received harvested 215 responses: 128 in English, 40 in French,
32 in Russian, and 15 in Spanish.

Timeframe April-May 2015

Link http://www.fao.org/fsnforum/activities/online-surveys

Comments http://agris.fao.org/agris-search/info.action

Engagement method Knowledge fair

Agency FAO

Name of the activity Knowledge Share Fair in Niger

By organising the Knowledge Share Fair in Niamey, the various

parties involved in FAO’s Knowledge Management and Gender
(KMG) programme proposed a new way of exchanging
knowledge on technical topics using a participatory approach.
The aim of the fair was not only to boost the profile of the KMG
programme, but also to initiate a process of exchanges of
experience, knowledge and networking on topics of common
interest to partners in West Africa. The event was attended by a
range of people and organisations selected for their potential
contribution to the exchanges, for the benefit they might derive
Description
from the presentations and for their interest in the programme’s
mission. Accordingly, producers’ organisations mixed with NGOs
and development projects, organisations interested in the topics
covered by the KMG programme, technical services and UN
agencies, students and researchers from universities or
specialised agricultural training centres, and representatives of
financial and technical partners and of FAO. The participants
discovered new methods for sharing knowledge, such as:
the maquis mondial (a French-language and African version of
the ‘world café’); the carrousel, a variant of the maquis mondial;

143
 peer assist; chat shows; proverbs; the tree of knowledge;
exchange visits; presentations with debates; and many more.

Timeframe November 2010

Link http://www.fao.org/3/am036e/am036e01.pdf

FAO knowledge fairs best practices: http://www.fao.org/3/a-

Comments
aq228e.pdf

Engagement method Community of Practice

Agency FAO

Name of the activity FAO - Community of Practice on food loss reduction

Members of the Community of Practice on food loss reduction

access the Forum, participate in online discussions, get in touch
Description with other practitioners, share and request relevant and updated
information, contribute in building up a worldwide community
aimed at reducing food losses and achieving food security.

Timeframe ongoing

Link http://www.fao.org/food-loss-reduction/en/

The CoP gathers practitioners and experts worldwide to share

and disseminate information and practices. It offers the
Comments
possibility to interact and collaborate, especially through
its Forum

Engagement method Capacity Building Workshop

Agency FAO

CAPFITOGEN Programme for the Strengthening of Capabilities in

Name of the activity
National Plant Genetic Resources Programmes
The workshops contributed to improve skills and capacity of the
technical staff in the area of conservation and sustainable use of
plant genetic resources for food and agriculture. They also
promoted the use and adoption of several data management
tools adapted to the needs of the national programs of this
region. Additionally, they help to gather information on the
Description needs to further develop new tools within the framework of the
Global Information System on Plant Genetic Resources referred
to in the Article 17 of the International Treaty.
These activities are funded by Spanish Government and have
been organized with the support of the National Plant Genetic
Resources Centre of the Spanish National Institute for
Agriculture and Food Research and Technology (CRF-INIA), the

144
 Spanish Ministry of Agriculture, Food and Environment, the
Polytechnic University of Madrid, the King Juan Carlos
University, the National University of Colombia and the
University of Santa Catarina in Brazil.
Thanks to the generous contribution of Spanish Agency for
International Development Cooperation (AECID), the Secretariat
funded travel and accommodation expenses of one participant
per Contracting Party.

Timeframe ongoing

Link http://www.fao.org/plant-treaty/tools/capfitogen/en/

The CoP gathers practitioners and experts worldwide to share

and disseminate information and practices. It offers the
Comments
possibility to interact and collaborate, especially through
its Forum

Engagement method e-Conference

Agency FAO

e-mail conference “Utilization of Food Loss and Waste as well as

Name of the activity
Non-Food Parts as Livestock Feed”
The electronic conference on ‘Utilization of Food Loss and Waste
as well as Non-Food Parts as Livestock Feed’ was held from 1
October to 30 October 2015. A background document was
distributed before the conference introducing the topics of the
conference and providing definitions and terms for a framework
and an action plan on ‘Food Loss and Waste plus Non-Food Part
to Livestock Feed’. The document highlighted the importance
and magnitude of food loss and waste and its impact on food
security and on local and national economies. Without
compromising animal health and welfare and animal product
safety and quality, and meeting legislative requirements plant
resources that are not used as food can be used as animal feed.
The aims of the electronic conference were to discuss the scope
Description
and boundaries of the framework including the definition of food
loss and waste; and explore the opportunities and possible
constraints in implementation of the framework.
During the first three weeks the conference was structured
through a set of leading questions. The leading questions for the
first week were about the definition of food loss and waste
related to feed and an inventory of plant resources used for food
and/or feed. The leading questions for the second week
addressed issues of feed safety and technologies to utilize food
loss and waste as animal feed including the roles of the food
industry and the feed manufacturing industry, while those for
the third week addressed the roles of researchers, Non-
Governmental Organizations (NGOs), civil societies, farmers and

145
 policy makers in making use of food loss and waste and non-
food parts of crops as feed. The fourth week provided
opportunity to again discuss topics of the first three weeks and
others not raised before. Most messages addressed either the
leading questions or responded to previous messages.

Timeframe from 1 October to 30 October 2015

http://www.fao.org/save-food/news-and-
Link
multimedia/events/detail-events/en/c/325893/
The conference generated considerable interest, as shown by
Comments the large number of subscribers (630) and 254 messages that
were received from 123 participants from 47 countries.

Engagement method Toolbox

Agency FAO

Toolbox for Sustainable Use of Plant Genetic Resources for Food

Name of the activity
and Agriculture (PGRFA)
Diversity in plant genetic resources for food and agriculture
(PGRFA) is essential to sustain food, nutrition and economic
security. The toolbox assists countries in designing and
implementing measures to promote sustainable use. The needs
of all stakeholders involved in different aspects of the global
PGRFA use system have been gathered and documented through
surveys and meetings and translated into an online system—the
Toolbox for Sustainable Use of PGRFA.
The Toolbox is for people seeking information or guidance on
policies, strategies and activities that can promote and enhance
the sustainable use of PGRFA, particularly at national and local
levels. Users may come from a wide range of stakeholder
groups, including those working in or associated with public
research institutes and gene banks, government agencies,
Description farmers’ associations, agro-NGOs, local and indigenous
community enterprises, seed networks, educational
establishments, international bodies, networks and services,
private plant breeding companies, and the commercial seed and
plant production industries, as well as independent plant
breeders, farmers and seed producers.
Using the Toolbox
The Toolbox has two primary components—web pages
highlighting and summarizing key areas of relevance in
promoting and enhancing the sustainable use of PGRFA, and a
database facilitating access to a selection of resources to aid
users in the development of sustainable use initiatives.
Resources are classified according to the relevant subject
area(s) and specific subject categories as well as by the primary
geographic area of relevance, language, publication date and

146
 format. Most resources are open access and freely available for
download from the Web.
Users can access resources via a search function in each subject
area web page (to find resources related to that area) or use the
‘Search for resources’ page to undertake free-text or structured
searches of the entire database.
To further aid users in navigating the database, the resources
are also classified according to eight types, as shown below.
Some resources are of more than one type—for example,
publications that include case studies, learning materials that
include tools, and case studies that are presented as
multimedia.

Timeframe ongoing

http://www.fao.org/plant-treaty/tools/toolbox-for-sustainable-
Link
use/overview/en/

Comments

Engagement method Co-development platform

FAO - International Treaty on Plant Genetic Resources for Food

Agency
and Agriculture
The Platform for the Co-Development and Transfer of
Name of the activity
Technologies
Stakeholders in the International Treaty are empowered to
utilise technologies for the conservation, characterization,
evaluation and use of plant genetic resources for food and
agriculture.
The Platform facilitates the discussion and coordination on issues
related to the identification of gaps and the development of
Description technology packets in the context of the International
Treaty. The Platform, with its objectives and operating
principles, was presented to the Fifth and Sixth Sessions of the
Governing Body in 2013 and 2015 and adopted as part of the
Programme of Work on Sustainable Use as a mechanism to build
partnerships for the co-development and transfer of technologies
relevant to PGRFA.

Timeframe ongoing

http://www.fao.org/plant-treaty/areas-of-work/sustainable-
Link
use/platform/en/

Comments

Engagement method Capacity building

147
 FAO - International Treaty on Plant Genetic Resources for Food
Agency
and Agriculture (PGRFA)

Name of the activity Training and Capacity Development Programme

Most of the activities have been implemented in partnership with

a wide range of stakeholders like universities and academic
organizations, large technical projects and research institutes
from developed and developing countries in different regions
and on a range of topics.
At its Sixth Session the Governing Body of the International
Treaty requested the Secretary to maintain the Capacity Building
Coordination Mechanism (CBCM) established in 2011; and to
identify existing centres which provide training relevant for the
implementation of the Treaty and that could function as partners
in an international network of centres of excellence with a
complementary curriculum.
The CBCM was established to coordinate existing scientific and
technical education and training programmes in areas of direct
relevance to the Treaty by building on existing capacities and
resources of recognised PGRFA institutions, universities and
centres of excellence; and to create synergies and opportunities
Description
among the interested stakeholders to develop capacity to carry
out interdisciplinary basic and applied research, particularly in
support of developing countries.
While the Secretariat of the International Treaty has played a
catalytic role, most activities are directly implemented through a
Network of Centres of Excellence. Each Centre operates as a
node at domestic level and coordinates a variable number of
institutions and organizations with complementary expertise.
The topics covered range from genomics, bioinformatics,
genebanks management, trade in PGRFA, information sharing,
governance of genetic resources, access and benefit-sharing law
and intellectual property rights. Some of the services offered by
the Centres are: 1) formal education and informal training
through postgraduate courses, scientific conferences and
technical events; 2) best practices to avoid risks and case
studies on various topics; 3) advice and support to users of the
Treaty systems

Timeframe ongoing

http://www.fao.org/plant-treaty/tools/training-and-capacity-
Link
development-programme/en/

Comments

Engagement method Learning modules for dissemination

FAO - International Treaty on Plant Genetic Resources for Food

Agency
and Agriculture (PGRFA)

148
Name of the activity Learning modules

The module is targeted to genebank managers and technical

advisers to policymakers and can also be used for education
The module contains background lecture notes, PowerPoint
presentations, practical exercises to reinforce an understanding
of the impact and working of International Treaty law in the
types of situations that professionals in plant genetic resources
can expect to face, references to the full-text of relevant laws
Description
and policies and a bibliography for further reading. The module
also contains complete Information for Trainers, with templates,
tips, and step-by-step guidelines for delivering a successful
training workshop.
The content is designed for a 2-day, face to face workshop, but
the materials may also be used separately for a shorter
workshop or to support classroom teaching.

Timeframe ongoing

Link http://treatylearningmodule.bioversityinternational.org/

Comments

United Nations (UN)

Engagement method Innovation Jam

UN-HABITAT the United Nations Programme for Human

Settlements, was mandated by the UN General Assembly in
Agency 1978 to address issues of urban growth. It collaborates with
governments and local partners to define the urban vision of
tomorrow.

Name of the activity Habitat Jam

In 2005, the Government of Canada and the UnitedNations

Human Settlements Programme (UN-HABITAT) organized a
three-day Habitat Jam. 19 Thousands of participants—from
urban specialists to government leaders to residents from
around the world—discussed issues of urban sustainability. Their
ideas shaped the agenda forthe UN World Urban Forum, held in
Description
June 2006. People from 158 countries registered for the jam and
shared their ideas for action to improve the environment, health,
safety, and quality of life in the world’s burgeoning cities. While
the Habitat Jam was a demonstration project, its success
indicates the promise of such approaches to engage many
citizens insolving important social issues and problems.

Timeframe 2005

Link https://en.wikipedia.org/wiki/Habitat_Jam

149
Comments

Engagement method Knowledge Portal

UN The United Nations Framework Convention on Climate

Change (UNFCCC) is an international environmental
treaty adopted on 9 May 1992. The framework sets non-binding
limits on greenhouse gas emissions for individual countries and
Agency
contains no enforcement mechanisms. Instead, the framework
outlines how specific international treaties (called "protocols" or
"Agreements") may be negotiated to specify further action
towards the objective of the UNFCCC.

Name of the activity UNFCCC ‘Adaptation Knowledge Portal’

UNFCCC ‘Adaptation Knowledge Portal’ (AKP), part of the Nairobi

work programme on impacts, vulnerability, and adaptation to
climate change (NWP)47. The UNFCCC AKP aims at facilitating
the sharing of good practices and lessons learned by offering an
exchange platform to all adaptation practitioners and
researchers, including partner organisations. The AKP provides
access to a curated database of adaptation knowledge resources
Description
such as case studies, methods and tools, publications and
technical documents, as well as other materials, and provides a
platform to share the latest news and resources on adaptation
under the UNFCCC process and from NWP partner organisations.
Users are welcome to contribute information to the
database or apply to become a NWP partner
organization through the submission forms on the AKP.

Timeframe ongoing

Link https://www4.unfccc.int/sites/NWPStaging/Pages/Home.aspx

Comments

Engagement method Online platform

World Health Organization, UN Environment Programme, the

Agency
World Bank, climate and clean air coalition

Name of the activity Breathlife

Breathlife is a platform that combines public health and climate

change expertise with guidance on implementing solutions to air
pollution in support of global development goals.
Description Objectives:
• Connect cities: Provide a platform for cities to share best
practices and demonstrate progress in their journey to
meeting WHO air quality targets by 2030

150
 • Increase monitoring: Work with municipalities to expand
monitoring efforts that can keep citizens informed and
facilitate more sustainable urban development
• Accelerate solutions: Build demand for new solutions that
are working and support municipalities in effectively
implementing them in their own cities
• Empower individuals: Educate people about the burden air
pollution poses to our health and our climate and provide
meaningful ways to take action both locally and globally

Timeframe ongoing

Link https://breathelife2030.org/

Comments

Engagement method Online Platform

The UN Environment Programme – UNEP is a programme of the

United Nations that coordinates the organization's environmental
Agency
activities and assists developing countries in
implementing environmentally sound policies and practices.

Name of the activity The World Environment Situation Room

The World Environment Situation Room implements the Big Data

Initiative. The project is global with overarching environmental
policy relevance and impact. It includes geo-referenced, remote-
sensing and earth observation information integrated with
statistics and data on the environmental dimension of
sustainable development. The themes of this Global platform
Description cover complementary dimensions for Global Green Solutions for
the Environment. It targets country policy makers, top
environmental policy makers, the environmental scientific
community, business and interested citizens. The platform is
essential as a knowledge instrument to support progress on
delivering the environmental dimension of Agenda 2030 for
Sustainable Development.

Timeframe ongoing

Link https://environmentlive.unep.org/home/aboutus

Comments

Engagement method Platform and citizens science

Agency UNEP

Name of the activity UNEP LIVE programme

151
 Includes also a citizens science portal “Environment Live’s
Description Citizen Science portal” that includes many examples of citizens
science initiatives on different topics.

Timeframe ongoing

Link http://uneplive.unep.org/citizen

Comments

Engagement method Data Hub

Agency UN environment

Name of the activity UN Environment SDG Data Hub

As custodian agency for 26 indicators of the Sustainable

Development Goals (SDGs), UN Environment is actively involved
in the development and refinement of data collection
methodologies, particularly for those indicators that remain
without internationally established methodologies and standards
Description (Tiers II and III).
In this effort, UN Environment periodically organizes Expert
Group Meetings, Capacity Building and Pilot Testing to seek
outside expertise from colleagues in academia, civil society, the
private sector, and other intergovernmental agencies.

Timeframe ongoing

Link http://sdgs-uneplive.opendata.arcgis.com/

Comments

Engagement method Online Platform

Agency UNDP and UN Environment

Name of the activity UNBiodeiversity Lab

The UN Biodiversity Lab is an online platform that allows

policymakers and other partners to access global data layers,
upload and manipulate their own datasets, and query multiple
datasets to provide key information on the Aichi Biodiversity
Targets and nature-based Sustainable Development Goals. The
core mission of the UN Biodiversity Lab is three-fold: to build
Description
spatial literacy to enable better decisions, to use spatial data as
a vehicle for improved transparency and accountability, and to
apply insights from spatial data across sectors to deliver on the
Convention on Biological Diversity and the 2030 Agenda for
Sustainable Development. The UN Biodiversity Lab data is
powered by MapX, the only UN-backed geospatial mapping

152
 software which collects and verifies scientific data, brokers
knowledge exchange for people in need of spatial solutions to
environmental and natural resource challenges, and offers a
customizable toolkit for analysis, visualization, and sharing. By
creating a collaborative, open-source environment, the UN
Biodiversity Lab is an inclusive and scalable data platform

Timeframe ongoing

Link https://www.unbiodiversitylab.org/about.html

Comments

Engagement method Information Portal and online e-courses

Agency UN

Name of the activity InforMEA and InforMEA learning

InforMEA is the United Nations Information Portal on Multilateral

Environmental Agreements, an online portal that provides
information about the Multilateral Environmental Agreements
(MEA's) to the public. The InforMEA initiative is facilitated by
the United Nations Environment Programme and supported by
the European Union. It seeks to develop Inter-operable
information systems for the benefit of the (MEA)Parties and the
environment community at large.
Since the launch of the InforMEA platform in 2011 the services
of InforMEA have expanded from providing a Thesaurus on
Environmental Law and Conventions to enabling access to MEA
related information in the form of decisions, resolutions, news,
Description calendars of events, a glossary, lists of Parties, national focal
points, national reports and strategies. InforMEA also provides
access to 28 free online e-learning courses related to the MEA's,
which are used in University curriculum's including that
of Macquarie University in Sydney, the University of Eastern
Finland and the UN system staff college (UNSSC).
The InforMEA Initiative consists of 20 Multilateral Environmental
Agreements hosted with four United Nations bodies. MEAs
include: Basel, Rotterdam, Stockholm and Minamata (Chemicals
and Wastes); CBD(Biodiversity); CITES (Trade in Wildlife); ITPR-
FA (Plants); Vienna (Ozone); Ramsar (Wetlands); UNESCO-WHC
(World Heritage); UNCCD (Deserts); UNFCCC (Climate Change);
5 UNECE Conventions and a number of regional conventions.

Timeframe ongoing

Link https://elearning.informea.org/course/view.php?id=40

Comments

153
World Vegetable Center (WorldVeg)
Engagement method International Improvement Network

The World Vegetable Center is the only international agricultural

research center with “development” in its mandate. Center
scientists have applied research in collaboration with farmers to
breed well-adapted cultivars and develop technologies to
increase yields and incomes in developing countries. Millions of
farmers today grow vegetable crops using seed or technologies
developed by the Center. The Center encourages the
participation of smallholders and other actors along the supply
chain in all research and development activities, and promotes
Agency consumption of diverse and safe vegetables with enhanced
nutritional qualities and nutraceutical potential to improve the
health of rural and urban poor consumers. Using this holistic
approach, AVRDC has generated a vast array of varieties that
are international public goods, developed technologies that
address economic and nutritional needs of the poor, and
empowered farmers, national agricultural research and
extension systems (NARES), nongovernmental organizations
(NGOs) and private sector personnel to engage in vegetable
production, marketing, and nutrition issues.

Name of the activity International Mungbean Improvement Network

Average global grain yields of mungbean are quite low and there
is much potential to develop better performing varieties.
International collaboration has been a key feature of mungbean
breeding research for decades. Many of the cultivated varieties
are based on breeding work coordinated by the World Vegetable
Center. To strengthen such international collaboration in the
light of future global challenges of nutrition security and climate
change, the World Vegetable Center established
the International Mungbean Improvement Network (IMIN) in
2016 with funding support from the Australian Centre for
International Agricultural Research (ACIAR). This network aims
to connect mungbean researchers from around the world to
openly share experiences, knowledge and technologies based on
Description
common principles of cooperation as laid down in a
Memorandum of Agreement. To date, IMIN has led to the
development of mungbean core and mini-core collections for
breeding to better exploit the potential of the available
mungbean genetic resources. This collection has been tested in
various countries in Asia and Africa, which led to the discovery
of many novel plant traits such as new sources of mungbean
yellow mosaic virus resistance, salt and heat tolerance, and
variations in iron and protein content. Parties interested to join
the network are invited to sign a Memorandum of Agreement
with the World Vegetable. The agreement specifies general
Principles of Cooperation to promote the open sharing of
knowledge, experiences and technologies. Membership is free of

154
 charge. BENEFITS include a regular newsletter with updates on
mungbean research, participation in an annual mungbean
workshop, and the potential to develop new projects and
collaborations with like-minded researchers. Improved
mungbean breeding lines and genebank accessions are available
from WorldVeg for members and non-members.

Timeframe Since 2016 - ongoing

Link https://avrdc.org/intl-mungbean-network/

Comments

World Fish Center

Engagement method Participatory Market Chain Approach (PMCA)

WorldFish is an international, nonprofit research organization

with headquarters in Penang, Malaysia, and offices in Asia, Africa
and the Pacific. WorldFish’s mission is to harness the potential of
fisheries and aquaculture to reduce poverty and hunger in
Agency
developing countries. WorldFish uses its experience in fisheries
and aquaculture to comply with Sustainable Development Goals.
WorldFish is one of the 15 specialized research centers of the
Consortium on International Agricultural Research (CGIAR).
PMCA applied to aquaculture value chain development in
Name of the activity
Bangladesh and Nepal
Technology and knowledge transfer has been a great challenge
for many developing countries in South Asia due to poor
information and communications technology networks. A limited
number of institutions focused on science and technology, weak
linkages among private and public institutions and political
instability. These factors have hampered the successful transfer
and diffusion of new and proven technologies between countries.
WorldFish, through the Agriculture and Nutrition Extension
Project (ANEP), supported the transfer of new technologies and
information between Bangladesh and Nepal in order to facilitate
aquaculture development. A number of activities were conducted
Description in order to facilitate this process. These included: meetings with
fish farmers to identify problems affecting producers;
stakeholder meetings and value chain analysis to identify
problems within the sector; events, including expert visits,
technical staff training, expert consultation using information
technology, hands-on training and exchange visits for farmers
and private sector entrepreneurs; and meetings to disseminate
information on new technology among fish farming stakeholders.
ANEP used the participatory market chain approach (PMCA) to
identify sector-specific problems and technological solutions
capable of transforming upstream and downstream segments of
the chain.

155
Timeframe 2014-2016

https://www.worldfishcenter.org/content/making-sense-market-
assessing-participatory-market-chain-approach-aquaculture-
value-chain
Link
https://www.worldfishcenter.org/content/aquaculture-
technology-exchange-between-bangladesh-and-nepal-
agriculture-and-nutrition

Comments

World Agroforestry
Engagement method Decision Hub

World Agroforestry (a brand name used by the International

Centre for Research in Agroforestry, ICRAF) is a centre of
science and development excellence that harnesses the benefits
of trees for people and the environment. Leveraging the world’s
Agency
largest repository of agroforestry science and information, the
centre develops knowledge practices, from farmers’ fields to the
global sphere, to ensure food security and environmental
sustainability.
Decision Hub based on SHARED (Stakeholder Approach to Risk
Name of the activity
Informed and Evidence-based Decision-making) methodology
The Decision Hub is a collective of stakeholder engagement
specialists, transdisciplinary scientists, and behavioral specialists
who together apply a tailored method for stakeholder
engagement, managing relationships and brokering multi-
stakeholder partnerships. These interactions are founded on a
principle of fostering evidence-based decision making. ICRAF
works with development partners, government agencies and
departments, private sector and research institutions. The
Decision Hub offers tailored facilitation and technical support to
stakeholder ecosystems. Through managing relationships,
catalyzing partnerships and sequencing interactions ICRAF aims,
through focusing on a human-centred process, to help address
complex development challenges.
Description The hub has been developed with to the SHARED (Stakeholder
Approach to Risk Informed and Evidence-based Decision-
making) methodology, which provides a comprehensive
framework, tailored to specific decision contexts, to bring
together processes, evidence, and tools, and shift the decision
paradigm towards more inclusive, inter-sectoral and inter-
institutional integration to tackle complex decisions and to
achieve desired outcomes. The SHARED approach includes four
inter-related phases using comprehensive facilitation to support
interaction with evidence, enhance co-learning, building long-
term relationships and ensure that evidence can be critically
interpreted, queried, and evaluated. This approach ensures
cohesive communication across multiple institutions, political

156
 levels and knowledge systems to build capacity and the evidence
base as a continuously linked process, within the same
development outcome pathway. The SHARED team works
through projects and consultancies in 15 countries in Africa and
2 in South Asia to date.

Indicative service offerings

• Leading structured change events
• Designing and implementing structured stakeholder
engagement processes
• Thematic and integrated research synthesis
• Science communications - Making science accessible
• Catalysing networks and partnerships
• Building capacity for data visualisation and interpretation for
integration into actionable decision making
• Research to enhance the impact of data visualisation into
decision making processes

Timeframe 2012 - ongoing

Link http://www.worldagroforestry.org/shared

Comments Contact: Mieke Bourne [email protected]

International Potato Center

Engagement method Competition

The International Potato Center (known as CIP from its Spanish-

language name Centro Internacional de la Papa) is a research
facility based in Lima, Peru, that seeks to reduce poverty and
achieve food security on a sustained basis in developing
through scientific research and related activities on potato,
sweet potato, other root and tuber crops, and on the improved
Agency
management of natural resources in the Andes and other
mountain areas. It was established in 1971 by decree of the
Peruvian government. CIP is one of the 15 specialized research
centers of the Consultative Group on International Agricultural
Research, an international consortium of agricultural research
organizations, having joined in 1972

Name of the activity CIP Open Access Competition

Following the success of the Data Sprint 2016 and the Open
Access Competition 2018, CIP is launching our third Open Access
Competition, this time focusing mainly on data documentation
and datasets that have been used to publish journal articles. The
Description goal is to have 100 new, quality assured and properly annotated
datasets published in CIP’s Dataverse repository
The Open Access competition will run until July 30, 2020
• All datasets published by CIP staff during the period of
1st August 2019 to 31th July 2020 will be eligible

157
 • Datasets need to be quality checked, include complete
metadata and well-defined variables annotated with
ontologies where possible
Competition Rules:
• Authors will receive one point for every qualified dataset
submitted to CIP’s Dataverse. For datasets with multiple data
authors, each author receives one point.
• Authors with datasets that have both the metadata and data
variables annotated with ontology terms will receive an extra
point.
• Authors of datasets that are part of an ISI Journal paper will
also receive an extra point.
Prizes:
• The individual author with the most score will receive funding
to cover the cost of one Open Access article and the cost of
his/her participation in one scientific conference
• The Program with most datasets will receive funding for two
Open Access articles.
• The Program in second place will receive funding for one
Open Access article.

Timeframe Ongoing (previous editions 2016 and 2018)

https://cipotato.org/open-access/cip-open-access-competition-
2019-2020/
Link https://cipotato.org/open-access/cip-open-access-competition-
2018/
https://cipotato.org/open-access/cip-open-access-data-sprint/

Comments

International Maize and Wheat Improvement Center (CIMMYT)

Engagement method Crowdsourcing

The International Maize and Wheat Improvement Center (known

by its Spanish acronym CIMMYT for Centro Internacional de
Mejoramiento de Maíz y Trigo) is a non-profit research and
training institution dedicated to both the development of
improved varieties of wheat and maize with the aim of
Agency contributing to food security, and the introduction of improved
agricultural practices to smallholder farmers to help boost
production, prevent crop disease and improve their
livelihoods. It is also one of the 15 non-profit, research and
training institutions affiliated with the CGIAR, formerly known as
the Consultative Group on International Agricultural Research.

Name of the activity Fall Armyworm Monitor

The Fall Armyworm Monitor is a web-based application that

collects population, incidence and severity data, and guides pest
Description
management decisions. The web tool relies on information
gathered by farmers using smartphones in their fields.

158
 Crowdsourced information on the movement of fall armyworm is
essential for effectively monitoring its spread and is a pivotal
step in its management. It was developed by CIMMYT in
cooperation with Bangladesh’s Department of Agricultural
Extension, through the Fighting Back Against Fall
Armyworm project, supported by USAID and Michigan State
University. Over 450 representatives from government,
nonprofits and the private sector participated in three-day
training to learn how to identify, monitor and apply integrated
pest management approaches.

Timeframe Ongoing

https://www.cimmyt.org/news/crowdsourced-data-feeds-fall-
Link armyworm-surveillance-in-bangladesh/
https://faw-monitor.firebaseapp.com/

Comments

International Livestock Research Institute (ILRI)

Engagement method Research Hub

ILRI's research is directed to improving food and nutrition

security through increased production and access to animal-
source foods; stimulating economic development and poverty
reduction through enhanced livestock value chains and increased
productivity; improving human health through improved access
Agency to animal-source foods and a reduction in the burden of zoonotic
and food-borne diseases; and managing the adaptation of
livestock systems to climate change and mitigating the impact of
livestock on the environment. ILRI's strategy 2013–2022 was
approved in December 2012. It emerged from a wide process of
consultation and engagement.
The Biosciences eastern and central Africa-International
Name of the activity
Livestock Research Institute (BecA-ILRI) Hub
The Biosciences eastern and central Africa (BecA)-ILRI Hub is a
strategic biosciences platform that, through increased use of
bioscience-based technologies, aims to improve the livelihoods
of millions of resource poor people in Africa. The program’s
portfolio comprises research projects on crop and livestock
improvement, nutrition and food safety, technology platforms
and capacity building.
Description Recent accomplishments
Over 1,000 scientists from more than 20 countries benefited
from short skill-enhancement training courses between 2015
and 2017.
Establishment of 13 communities of practice, linking African
scientists of different disciplines with similar interests. Support
to grant writing has resulted in proposals worth about USD13
million.

159
 Over 6,000 dairy farmers in Kenya and Rwanda received seeds
of a climate-smart Brachiaria forage. Due to the resulting extra
forage, they reported increases in milk production of up to 40%
and 50% higher live-weight gains in young cattle

Timeframe Ongoing

Link https://www.ilri.org/research/programs/beca-ilri-hub

Comments

Global Investment Facility (GEF)

Engagement method Breakout sessions

The Global Environment Facility (GEF) was established on the

eve of the 1992 Rio Earth Summit to help tackle our planet’s
most pressing environmental problems. Since then, the GEF has
provided close to $20 billion in grants and mobilized an
Agency
additional $107 billion in co-financing for more than 4,700
projects in 170 countries. Through its Small Grants Programme,
the GEF has provided support to nearly 24,000 civil society and
community initiatives in 128 countries.
Interactive Breakout Sessions - Civil Society Forum at the Sixth
Name of the activity
GEF Assembly
The opening session was followed by an interactive session
where 18 CSOs and IPLC made a pitch about their projects and
all attendees were invited to participate in breakout groups
to showcase the “real” story behind successful examples
of systems change including examples of interventions that have
involved different stakeholders from civil society/Indigenous
Peoples and Local Communities (IPLC), governments and the
private sector at local, national or regional levels. In the
afternoon participants attended four breakout sessions as
follows:
1. Effective collaboration for Systems Change
This breakout session showcased examples of effective
collaboration between civil society/IPLC, governments and the
Description
private sector, and will highlight innovative multi-stakeholder
partnerships at local, national or regional levels that have
contributed to sustainable energy, food production or other
areas with positive impact in terms of accountability and good
governance for system change.
2. Innovative environmental action and policy advocacy
Advocacy and innovative thinking and practice from civil society
is needed to achieve transformation of the unsustainable
economic systems that are driving the planet’s environmental
degradation. This session showcased examples of innovative
tools and mechanisms used by civil society and indigenous
peoples and local communities in achieving these
transformations.

160
 3. Pathways for inclusive GEF Projects and Programs:
Operationalizing GEF’s Stakeholder Engagement and
Gender-responsive Approach in GEF-7
The session engaged representatives from civil society,
grassroot and women’s organizations, government and the GEF
partnership in an informal discussion on how to ensure effective
stakeholder engagement and promotion of gender equality in
GEF programs and projects, and ways to practically
operationalize GEF’s new Policies on stakeholder engagement
and gender equality in GEF-7 and beyond GEF’s policies and
guidelines into practice.
4. Views of the Civil Society Network on Effective
Collaboration for Sustainable Transformation
In making a transformational change of the GEF CSO Network
(GCN) to function as a strong and active civil society network in
the GEF systems, there is a progressive need to engage the
network within the context of multi-focal dimensions. The GCN
aims to strengthen the members at country levels, empower the
national and regional elected Regional Focal Points and IPLCs
representatives along with a well-functioning GCN secretariat.
The presentation talked about experiences and effective
collaboration between available community mechanisms
between GCN and the greater civil society, IPCLs, governments,
private sector and line actors at local, national, regional and
global levels to accelerate the achievement of the global
environmental benefits.

Timeframe June 2018

https://assembly.thegef.org/assembly/civil-society-forum-sixth-
gef-assembly
Link
https://assembly.thegef.org/documents/cso-forum-interactive-
session-outcome-sixth-gef-assembly (outcomes)

Comments

Stakeholder Engagement – other organisations

The University of Washington

Engagement method Gamification

The University of Washington, Center for Game Science, in

Agency
collaboration with the UW Department of Biochemistry

Name of the activity Foldit

Foldit is an online puzzle video game about protein folding. The

objective of Foldit is to fold the structures of selected proteins as
perfectly as possible, using tools provided in the game. The
Description highest scoring solutions are analyzed by researchers, who
determine whether or not there is a native structural
configuration (native state) that can be applied to relevant
proteins in the real world. Scientists can then use these

161
 solutions to target and eradicate diseases and create biological
innovations.

Timeframe ongoing

Link https://fold.it/

A 2010 paper in the science journal Nature credited Foldit's

Comments 57,000 players with providing useful results that matched or
outperformed algorithmically computed solutions.

Danish Government
Engagement method Science Week

Agency Danish Government

Survey on Good Practices in the Protection of Human Rights

Name of the activity
Defenders
One of the methods applied during Danish Science Week is the
"Mass Experiment". The purpose of this experiment is to give
children and youths insights into scientific methods through the
medium of an issue which relates to their everyday lives. Every
year, the Danish Science Factory develops a new experiment in
collaboration with one or more research institutions. Around
1000 school classes - or between 20,000 and30,000 children
and youth - take part in the experiment. The pupils conduct
experiments and collect data, and afterwards, they report their
results to Danish Science Factory and the scientists who analyse
the data and write a final report (for more on results, see
below).
Besides the "Mass Experiment", Danish Science Week offers a
number of different methods which communicate science. For
Description instance, one of these is, "Book a Lecture" where scientists visit
schools and share their experiences with science. During Science
Week, between 500 and600 lectures are given. Furthermore,
many external partners organise activities, so Science Week
does not only take place in the classroom.
In the course of a science week, the Danish government wanted
to formulate a new strategy to combat climate change while
driving new business growth. Rather than limiting the idea
generation process to key government agencies and
departments, the Danish government organized an Innovation
Jam (a series of workshops) that brought together government
representatives, businesses, and citizens as well as academics,
experts, and artists. Such an approach helped government
agencies to get innovative ideas and policy suggestions from a
diverse set of stakeholders and go beyond the usual agency turf
wars that typically dominate policy debate

Timeframe

162
Link https://naturvidenskabsfestival.dk/what-danish-scienceweek

Comments

IBM – Innovation Jam

Engagement method Innovation Jam

Agency IBM

Name of the activity Innovation Jam

IBM started its Innovation Jams initiative in the early 2000s as a

way to engage its employees in defining (or identifying) the core
values of the company. A Values Jam enables a collaborative
approach toward defining what the company should be all about.
The company soon learned that the same approach could be
used to find solutions to more complex issues and problems.
One such problem was identifying future technology investment
areas. Given a wide range of emerging technologies, the
company had to decide which were worth pursuing, and which
Description types of business opportunities (and commercial applications)
would be feasible in those areas. Traditionally, such decision-
making would have involved senior managers and technologists
within IBM (and, to a limited extent, input from external
consultants and experts). However, IBM decided to throw open
the discussion to a broader audience. In 2006, the company
organized the Innovation Jam, aimed at identifying and
evaluating business opportunities associated with different
emerging technologies.

Timeframe 2006

The Innovation Jam—a massive limited-time online

brainstorming session—involved 150,000 people from 104
countries, including IBM employees, scientists and researchers
from universities, business partners, and customers. Over two
72-hour online sessions, participants posted more than 46,000
ideas as they explored IBM’s most advanced research
Link
technologies and considered their application to real-world
problems and emerging business opportunities. These ideas
were then carefully examined and pruned to identify the 10
most promising business opportunities. Over the years, IBM has
invested millions of dollars in new business creation based on
the output of the Innovation Jam.

Comments https://www.ibm.com/products/innovation-jam

“Wissenschaftsladen”, the The Bonn Science Shop - Germany

Engagement method Science shop - making research accessible to citizens

163
 WILA Bonn focuses on key social challenges, such as the energy
transition and social justice. They encourage scientists to take
Agency
up complex topics, whilst also ensuring that these complex
challenges are also understood by citizens.

Name of the activity Wissenschaftsladen

The largest science shop globally. Generates a turnover of ca 3

Description
million euros. Won multiple awards.

Timeframe Founded in 1984

Link https://www.wilabonn.de/en/

Comments

wer-weiss-was.de - Germany
Engagement method Information sharing network

Agency wer-weiss-was.de

Name of the activity wer-weiss-was

An online platform where users can ask and reply to questions

pertaining to e.g. different areas of science. 450 000 registered
Description
users, 1.6 millions of questions asked with over 6.7 millions
answers. Free registration

Timeframe Established in 1996

Link https://www.wer-weiss-was.de

The platform is mainly meant for experts in a specific area.

Comments However, everyone can join, meaning not every user will be an
expert.

Gute Frage - Germany

Engagement method Information sharing network

Agency gutefrage.net GmbH

Name of the activity Gute Frage

An online platform where users can ask and reply to questions

and share knowledge. 1.9 million active users, 15 million new
Description
users per month, 25 000 daily replies to questions. Free
registration

Timeframe Established in 1996

Link https://www.gutefrage.net

164
 The platform is mainly meant for experts in a specific area.
Comments However, everyone can join, meaning not every user will be an
expert.

Wellcome – United Kingdom

Engagement method Campaigns, partnerships, collaborative initiatives

Wellcome is a politically and financially independent

foundation supporting researchers, taking on big health
Agency
challenges, campaigning for better science, and helping
everyone get involved with science and health research

Name of the activity Wellcome

Combatting health-related issues by collaborating with

academia, philanthropy, businesses, governments and civil
society around the world. Approaches include vaccine
development, academical research into various health risks, and
raising public awareness of health risks through e.g. public
engagement projects. Examples of collaborating with scientist
include the largest ever survey into experiences of research
Description culture (4000 scientist participants), online forums and café
culture discussions where scientist can raise their opinions and
concerns. They also host e.g. the Reimagine Research Solutions
Summit 2020, where they bring together advocates and experts
from across the research community to reflect on the findings
and opinions. They basically facilitate the collaboration of
scientist from various fields. Won multiple awards for e.g.
innovation and collaboration activities.

Timeframe 1936 to date

Link https://wellcome.ac.uk

Projects funded by an investment portfolio, which in September

Comments
2019 was worth £26.8 billion.

Expert optix – United Kingdom

Engagement method Expert assessment tool

Eldeman BioScience Communications. Edelman is a global

communications firm that partners with businesses and
Agency
organizations to evolve, promote and protect their brands and
reputations.

Name of the activity Expert optix

“expert optix employs an integrated and holistic approach to

assessing experts through both quantitative and qualitative
Description assessment, incorporating dimensions such as sentiment and
social media visibility into expert profiles. And it presents it all in
a visually compelling, at-a-glance infographic.”

165
Timeframe Since 2015

https://www.edelman.com/expertise/medical-
Link
communications/expert-engagement

Comments

Aalto University - Finland

Engagement method Science collaboration platform

Agency Aalto University

Science collaboration platforms: provide events, networking,

Name of the activity
seed funding, and visibility for cross-sectional research
The university has various platforms specialising in topics such
as digital innovation, health, energy, sustainability… and
Description multidisciplinary collaboration. Connecting artist and technical
and scientific researchers both inhouse and with various
industries. Events include monthly brainstorming breakfasts.

Timeframe ongoing

Link https://www.aalto.fi/en/research-art/what-are-aalto-platforms

Aalto also facilitates coordination with various research

Comments infrastrustures: https://www.aalto.fi/en/research-and-learning-
infrastructures

Business Finland - Finland

Co-creation and co-innovation initiatives between
Engagement method
researchers and business

Agency Aalto University

Name of the activity Business Finland

By funding and encouraging:

• research organizations to carry out high-level public
Description research, strengthening their own expertise and at the
same time cooperate with companies
• companies to renew their business by working closely
together with research organizations.

Timeframe ongoing

https://www.businessfinland.fi/en/for-finnish-
Link customers/services/funding/cooperation-between-companies-
and-research-organizations/

Comments

166
Donau-Universität Krems und Bundesministerium für Kunst, Kultur,
öffentlichen Dienst und Sport - Austria
Engagement method Innovation Lab

Donau-Universität Krems und Bundesministerium für Kunst,

Agency
Kultur, öffentlichen Dienst und Sport

Name of the activity GovLabAustria

An innovation lab addressing key challenges in the public sector

in an open and interdisciplinary space. Methods utilised in the
experiments include co-leadership, co-creation and
collaboration, personas, gameification, design-thinking, as well
Description
as agile development/rapid prototyping. They also mediate
collaboration between experts from other innovation labs. They
also offer a physical place where creation and collaboration can
take place (ImpactHUB Vienna, see below).

Timeframe ongoing

Link http://www.govlabaustria.gv.at/ueber-uns/

Comments

Impact Hub - Austria

Engagement method Innovation Lab

Impact Hub Vienna GmbH

Agency

Name of the activity Impact Hub Vienna

By bringing sustainable businesses together to innovate. They

bring founders, creatives, investors, established companies and
NGOs together to innovate and develop entrepreneurial ideas
Description combining sustainability and profitability. They offer co-working,
event spaces, as well as accelerator programs for start-ups. A
global network of over 110 impact hub locations in over 50
countries, with 16 000 members worldwide.

Timeframe ongoing

Link https://vienna.impacthub.net

Comments

Science Shop Austria - Austria

Engagement method Science Shop

Agency Science Shop Austria

167
Name of the activity Science Shop Austria

Combines science, technology and the civil society with current

themes in the context of emerging technologies. It acts as a
mediator between the civil society, researchers, and experts in
Description
the fields of ambient assisted living, artificial intelligence, as well
as the Internet of things in order to solve societally relevant
issues within these fields.

Timeframe ongoing

Link https://www.scienceshop.at/en/

Comments

Science Shop Vienna - Austria

Engagement method Science Shop

Wissenschaftsladen Wien - Science Shop Vienna is an

independent research institute that performs research in
response to needs and demands of non-profit organisations,
Agency
such as human rights organisations, non-profit service
organisations, local authorities and social or environmental
initiatives

Name of the activity Wissenschaftsladen Wien - Science Shop Vienna

By facilitating research projects, organising conferences and

workshops, offering expert opinions in research.
Research topics are mainly derived from requests directed to the
Description institute or are developed together with NGOs/civil society
organisations. However, the Science shop can also reach a wide
number of cooperation partners from areas such as natural and
technical sciences, when necessary.

Timeframe Established in 1991

Link https://wilawien.ac.at/index_en.html

Comments

LISAvienna - Austria
Engagement method Life science platform

LISAvienna is a joint life science platform operated by Austria

Wirtschaftsservice and the Vienna Business Agency. On behalf of
the Austrian Federal Ministry for Digital and Economic
Agency Affairs and the City of Vienna, it contributes to the advancement
of life sciences in Vienna. LISAvienna supports innovative
biotech, pharmaceutical and medical technology companies in
Vienna that develop and market new products, services and

168
 processes. The platform links these companies with development
partners and key customers. As a central knowledge carrier,
LISAvienna provides input for decisions to advance the life
sciences in Vienna and contributes to positioning the city of
Vienna as one of the leading European innovation centres.

Name of the activity LISAVienna

A joint life science platform operated by Austria

Wirtschaftsservice and the Vienna Business Agency.
They support biotech, pharmaceutical and medical technology
companies in Vienna by linking them with key customers and
Description development partners. Their free events in Vienna, the
"Business Treffs", enable networking for an increased exchange
of experiences and knowledge transfer within the Vienna life
sciences community. They offer free services (only) for start-ups
in Vienna

Timeframe Established 18 years ago

Link https://www.lisavienna.at

Comments

European Science Engagement Association (EUSEA) - Austria

Engagement method Science Engagement platform

EUSEA is an international knowledge-sharing platform and

accelerator of innovation in the fields of public engagement. The
Agency association addresses experts involved in the design,
organisation and implementation of public engagement
activities across Europe

Name of the activity EUSEA European Science Engagement platform

The EUSEA Science Engagement Platform was established to

serve public engagement professionals across Europe in their
needs to find inspiration, resources, methods and tools for
running participatory, dialogue-oriented engagement activities.
The platform unites and showcases inspiring and innovative
ways to engage different publics with science. A special focus of
the resources provided on this platform lies on actively involving
citizens and stakeholders in dialogue-oriented research and
Description
innovation processes.
It has started by uniting science festival organisers with science
communication practitioners from cities, universities, science
centres, and cultural institutions across Europe. It “has evolved
from a meeting-place for science festival organisers to a
collaborative international community for public engagement
practitioners. Today, Eusea encourages and supports innovative
formats of science-society dialogues across Europe – ranging

169
 from Researchers’ Nights to Science Parliaments, from Science
Cafés to Maker Fairs, from public debates to local strategies
uniting scientists with policy makers.”

Timeframe Founded in 2001

Link https://eusea.info/platform/about-this-platform/about/

Comments

RAND corporation - US
Online platform for stakeholder engagement and expert
Engagement method
elicitation

Agency RAND corporation

Name of the activity ExpertLens

A three-to four round online process in which participants

provide input and engage in an online discussion. Participants
can also reply to other users’ comments. Information about a
Description
research or project are provided together with background
information to facilitate decision making and giving feedback.
Each expert round typically lasts for one week.

Timeframe ongoing

Link https://www.rand.org/pubs/tools/expertlens/about.html

Mainly applicable for public policy, health care, finance and

Comments marketing, or where expert panels are typically used to help
solve complex issues.

Innocentive - US
Engagement method Open innovation platform

Agency InnoCentive, Inc.

Name of the activity Innocentive

One of the best -known open innovation platforms. Allows

companies and experts (from various industries) to interact.
Description Only winning solutions are paid for. Awards vary greatly
depending on the complexity of the task or problem. A network
of over 400 000 experts

Timeframe ongoing

Link https://www.innocentive.com

Comments

170
Productivity Commission - Australian Government’s independent
research and advisory body
Engagement method Public Consultation

The Productivity Commission is the Australian Government’s

independent research and advisory body on a range of
economic, social and environmental issues affecting the welfare
of Australians. Its activities cover all levels of government and
encompass all sectors of the economy, as well as social and
environmental issues. One of its core functions is to conduct
Agency
public inquiries at the request of the Australian Government on
key policy or regulatory issues bearing on Australia’s economic
performance and community wellbeing. The Commission is often
required to provide the Government with policy options
representing alternative means of addressing the issues, as well
as a preferred option.

Name of the activity Public Inquiries

As part of the Commission processes, consultations give the

opportunity for different points of view in the community to be
heard and considered. They are used to gather relevant
information on policy issues and their impacts on different
groups within society. The Commission’s legislation requires
impacts to be assessed on a community-wide basis. Hence, the
inquiries help ensure that all costs and benefits are considered
regardless of sectoral or other particular interests. The process
can also help build momentum for reform.
Public inquiries usually involve two stages of consultation: Input
from interested parties and the general public is sought at an
initial stage on an issues paper to focus attention on the matters
it considers relevant, as well as at a later stage on a draft
report. Interested parties can generally provide written
submissions to the Productivity Commission at both stages. In
addition, public hearings and/or other consultative forums are
Description
held to give interested parties the opportunity to provide
feedback and input to the Commission’s inquiries. Final
Commission reports, including documentation on the analysis
and public consultation as well as research findings and policy
recommendations, are tabled in the Australian Parliament, and
are publicly available to the wider community. While
governments are not obliged to follow the Commission’s advice,
Commission recommendations are often accepted. A key
strength of the inquiries according to the Productivity
Commission is the consultation process, with a public draft
report for consultation allowing for: 1. Meaningful feedback in
consultations – interested parties are reacting to draft
recommendations and can challenge the logic and the evidence
2. The scope to revise if needed in the light of new information
3. Testing of the public reaction to the draft recommendations,
which can help identify issues that require further consideration

171
 including in respect of supporting information and
implementation issues. There is a risk according to the
Commission of recommendations being ruled out at an early
stage of investigating policy solutions without them having full
consideration.
A key strength of the inquiries according to the Productivity
Commission is the consultation process, with a public draft
report for consultation allowing for:
1. Meaningful feedback in consultations – interested parties
are reacting to draft recommendations and can challenge
the logic and the evidence
2. The scope to revise if needed in the light of new information
3. Testing of the public reaction to the draft recommendations,
which can help identify issues that require further
consideration including in respect of supporting information
and implementation issues.

Timeframe Ongoing practice

https://www.oecd.org/gov/regulatory-policy/AUS-Productivity-
Commission-Inquiries.pdf
Link
https://www.pc.gov.au/inquiries?collection=productivity-
commission-web&form=inquiries&gscope1=21 (outcomes)

Comments

U.S. Department of Health & Human Services - National Institute on

Minority Health and Health Disparities
Engagement method Community based participatory research

U.S. Department of Health & Human Services - National

Agency
Institute on Minority Health and Health Disparities
The NIMHD Community-Based Participatory Research Program
Name of the activity
(CBPR)
The NIMHD Community-Based Participatory Research Program
(CBPR) supports collaborative interventions that involve
scientific researchers and community members to address
diseases and conditions disproportionately affecting health
disparity populations. Recognizing the strength of each partner,
scientific researchers across multiple disciplines and community
Description members collaborate on all aspects of the project, which may
include a needs assessment, planning, research intervention
design, implementation, evaluation, and dissemination of
community-level interventions. The community is involved in the
CBPR program as an equal partner with the scientists. This helps
ensure that interventions created are responsive to the
community’s needs.
https://www.nimhd.nih.gov/programs/extramural/community-
Timeframe
based-participatory.html

Link

172
Comments

U.S. Government
Engagement method Crowdsourcing

Agency U.S. Government

Survey on Good Practices in the Protection of Human Rights

Name of the activity
Defenders
Challenge.gov is a web platform that assists federal agencies
with inviting ideas and solutions directly from the public, or
“crowd” (crowdsourcing). The website enables the U.S.
government to engage citizen-solvers in prize competitions for
top ideas and concepts as well as breakthrough software,
scientific and technology solutions that help achieve their agency
missions.
This site also provides a comprehensive toolkit, a robust
repository of considerations, best practices, and case studies on
Description running public-sector prize competitions as developed with
insights from prize experts across government.

Some examples: Department of Agriculture (USDA) challenged

the public to create an electronic application for the National
School Lunch Program.
The Environmental Protection Agency (EPA) is looking for
technologies to recycle nutrients from livestock manure.
The Bureau of Reclamation is looking for solutions to count food
for critical fish species.

Timeframe https://www.challenge.gov/

Link

Comments

USAID United States Agency for International Development

Engagement method Challenge Prizes

The United States Agency for International Development USAID

is an independent agency of the United States federal
Agency
government that is primarily responsible for administering
civilian foreign aid and development assistance.

Name of the activity Data Driven Farming Prize

For the Data Driven Farming Prize, delivered on behalf of the

United States Agency for International Development (USAID),
Description
Nesta invited innovators to create smart tools to support farmers
in producing more food in Nepal.

173
Timeframe ongoing

Link https://datadrivenfarming.challenges.org/

More than 140 teams applied from around the world, with 13
finalists given additional support to develop their product
Comments
further, with exciting results for plant diagnostics and moisture
management.

174
Annex III | Target audience identification methods

Overview

List of methods and their goals

Methods Goal

Identifying stakeholders to engage them in transdisciplinary,

Method for
sustainability research projects and generate knowledge.
Transdisciplinary
E.g. find academic experts to identify potential risks of a new
Research
biotechnology on human health.
Identifying stakeholders to help understand their involvement
Method to map in a range of societal challenges; prioritising engagement with
stakeholders stakeholders who can offer input on the challenges and influence
associated with policy and research priorities in a region.
societal challenges E.g. identify consumers or CSO organisations to discuss animal
testing.
Identifying stakeholders within a research context requiring
Stakeholder
practical input from the field.
identification in concept
E.g. find industrial stakeholders to discuss best practices in using
mapping
food packaging.
Identifying stakeholders within a project management context
Stakeholder Circle to develop and monitor a communication plan.
Methodology E.g. identifying diabetes patients to develop a communication plan
around a food additive.
Identifying stakeholders to engage them as part of a balanced
audience with different degrees of influence, avoiding missing
Prospex-CQI: Criteria-
out any stakeholder group.
Quota-Individual
E.g. engage a diverse group of stakeholders to exchange
perceptions on endocrine disruptive properties of pesticides.
Identifying top experts on a subject matter, with the objective of
engaging with analogous fields and find transferable
Pyramid Research knowledge and creative solutions to an issue.
E.g. identify experts from the pharmaceutical industry on a new
biotechnology not yet present in the food sector.
Identifying and engaging stakeholders to foster responsible
research & innovation and increase potential impact of policy
Identification Public-
making by assessing levels of interest and influence.
Public Partnerships
E.g. identify MEPs whose work is relevant to the Farm2Fork
strategy.
Identifying real world parties in line with the situations of
Stakeholder
stakeholders in innovation projects.
identification beyond
E.g. identify a group of farmers impacted by the authorisation of a
classification
new GMO cultivation.

175
 Methods Goal

Identifying stakeholders on social media to improve online

Mapping Stakeholders engagement.
from Social Media E.g. identify scientists active on Twitter or Linkedin on new food
and feed production technologies to engage with them.
Visualising data about stakeholders to improve/develop
Smart Sheet targeted communication.
Stakeholder Mapping E.g. identify all actors of the food packaging supply chain for
communication around nano-plastic hazards.

Comparative table to weigh advantages and disadvantages

The table below compares the above-mentioned methods against a set of criteria. A colour
code is set from green (recommended) to red (warning) in the following order: green,
yellow, orange, red. The criteria are defined as follows:
▪ Time investment: estimated time between kick-off until the result.
▪ Green: can be done in less than 2 weeks.
▪ Yellow: may be done in 2 weeks, but likely more (e.g. depending on available
data or skills).
▪ Orange: can be done between 2 to 4 weeks.
▪ Red: more than 4 weeks, OR difficult to assess due to dependency with external
factors (e.g. need for data from experts and lack of control on their willingness
to support).
▪ Flexibility: adaptability of the method to a variety of topics or contexts.
▪ Green: easily applicable to a wide variety of topics, and contexts.
▪ Yellow: flexible to a certain extent, with a minimum number of unchangeable
factors only (e.g. geographical scope).
▪ Orange: not highly flexible, mostly applied in restricted contexts.
▪ Red: little to no flexibility.
▪ Expertise required: amount of experience and skills required within EFSA to apply
the methodology.
▪ Green: can be performed by staff with little skills or experience in identification
methods.
▪ Yellow: achievable without experience but easier with some skills.
▪ Orange: difficult to perform without experience, requires some skills.
▪ Red: requires high seniority and/or specialist skills.
▪ Internal/external: balance between amount of data available internally and data
to gather from external sources (which can considerably delay the process).
▪ Green: can be done with internal knowledge.
▪ Yellow: requires minimal involvement of external experts.
▪ Orange: requires moderate involvement of external experts.
▪ Red: relies heavily or almost only on external experts to provide data.

176
 ▪ Neutrality: assessing the risk of identification bias from researchers carrying out
the method.
▪ Green: No or extremely unlikely risk of identification bias by the researcher.
▪ Yellow: Minor risk of identification bias.
▪ Orange: Sizable risk of identification bias.
▪ Red: Major risk of identification bias.
The line ‘Link to purposes’ relates to the methods presented in Annex 1 and highlights
which engagement purposes are best applicable to each target audience identification
method.
NOTE: if n/a is indicated in the table below, it means that the information is not available
or cannot be estimated as the methodology is patented or IP-protected.
Expertise Internal/ Link to
Methods Time Flexibility
required external
Neutrality
purposes

Method for Generate

Transdisciplinary ideas, learn &
consult
Research
Mapping stakeholders Generate
ideas, learn &
associated with consult, share
societal challenges lessons

Stakeholder
Learn &
identification in concept consult
mapping
Generate
Stakeholder Circle ideas, learn &
n/a
Methodology consult,
inform

Prospex-CQI: Criteria- Generate

ideas, share
Quota-Individual lessons

Generate
Pyramid Research ideas, learn &
consult

Identification Public-
All purposes
Public Partnerships

Stakeholder Learn &

consult,
identification beyond n/a n/a n/a
inform, gather
classification data

Mapping Stakeholders
n/a n/a Inform
from Social Media

Smart Sheet
Inform
Stakeholder Mapping

177
Description of the methods

Method for Transdisciplinary Research

This method3 consists of a snowball sampling approach, easily applicable by researchers
with no prior experience in stakeholder research.
It is a 2-phase process of design and implementation of an identification questionnaire:
1. collective identification: a provisional group of stakeholders is identified by the
researchers based on the knowledge they already possess;
2. researcher immersion - researchers focus on identification before the first data
generation by immersing in a research problem - this is done through 1) desk-
based research and 2) exploratory/pilot study in case study areas to supplement
this knowledge.
This process is typically followed by a stakeholder engagement activity or event, where
knowledge is generated.
Goal: identifying stakeholders with the aim of engaging with them in transdisciplinary,
sustainability research projects, to generate further knowledge

Researchers (no experience with

Target users
stakeholder identification needed)

Geographical scale From local to international

Online/Offline Online and offline

Example: this methodology was used across 17 European case studies for
transdisciplinary research in the EU-funded project RECARE for the prevention and
remediation of degradation of soils.

Estimated resources

Relative time investment Medium

Timeframe 2 to 3 weeks

FTEs and profiles 2 or 3 people. Ideally one social scientist.

None for the identification as such, but the objective is to have

Direct costs
an engagement activity ultimately

3
Julia Leventon, Luuk Fleskens, Heleen Claringbould, Gudrun Schwilch & Rudi Hessel, 2016: An applied
methodology for stakeholder identification in transdisciplinary research. Sustainability 11 763-775.
https://link.springer.com/article/10.1007/s11625-016-0385-1

178
 Pros Cons

▪ A flexible methodology, which can be ▪ Requires social scientists for case study
applied to different research contexts. exploration.
Some steps can be skipped or combined ▪ Relies on case studies – not always
differently. available for EFSA.
▪ Combines two approaches (collective ▪ Potentially time-consuming as
identification and researcher immersion), researchers first rely on existing
minimising resources. knowledge before supplementing it.
▪ More reliable thanks to the combined ▪ Risk of bias by the implementing
approaches: less chances of missing key researcher(s) - individual values or
stakeholders. existing networks can affect the
identification process.

Comparative table
Between 2-3 weeks Relies heavily on
Time investment (depending on Internal/external external actors =
external inputs) risk of delays
Easily adjustable to Minor risk of
Flexibility Neutrality
different contexts. identification bias
Better with social Generate new ideas,
science expertise but Learn from and
Expertise required Link to purpose
doable with no consult experts and
experience stakeholders

Mapping stakeholders associated with societal challenges

This method4 was originally providing guidance on identifying key stakeholders to South
Asian partners in the EU-funded CASCADE project. The identification of stakeholders was
performed by means of:
1- a detailed policy and trend analysis of societal challenges in relevant countries.
The analysis focuses on several pre-selected societal challenges. It consists in a desk
research setting out existing statistics, trends, assessing policy availability in the
chosen geographical areas and identifying key informants with knowledge or
responsibility in developing policies in those areas.
2- interviews and focus groups with knowledgeable experts with experience in one or
more areas of the societal challenges.
▪ Previously identified key informants support the identification of relevant
respondents.
▪ Semi-structured interviews are used to gain a better understanding of the
challenge, their impact on society and stakeholders.

4
Kanchana Ginige, Dilanthi Amaratunga & Richard Haigh, 2018: Mapping stakeholders associated with societal
challenges: A Methodological Framework. Procedia Engineering 212 1195-1202.
https://www.sciencedirect.com/science/article/pii/S1877705818301802#!

179
Interview data are analysed and further developed using focus groups. Focus groups
help refine the overall perspective, get consensus on the understanding of the challenge
and identify further stakeholders.
Goal: identify stakeholders to help understand their relationship to a range of societal
challenges; prioritise engagement with stakeholders who can offer input on the challenges
and influence policy and research priorities in the region.

Policy researchers (applicable to advocacy

Target users
groups, NGOs, consumers)

Geographical scale National (but scalable)

Online/Offline Online and offline

Example: this methodology was used offline in the EU-funded CASCADE project
(Collaborative Action towards Societal Challenges through Awareness, Development and
Education) for 7 societal challenges, including food security. This led to 348 interviews and
135 focus groups (about 50 interviews and 20 focus groups per challenge).

Estimated resources

Relative time investment High

Between 4 and 8 weeks approx. depending on availability of

Timeframe respondents and of number of interviewers/focus group
moderators
4 to 5 people, with experience conducting interviews and focus
FTEs and profiles
groups
Offline: international travel costs to interviews and focus
groups
Direct costs
Online: phone bills and membership to a digital meeting tool
(for focus groups)

Pros Cons

▪ Can apply to a wide range of topics ▪ Potentially too narrow - can be used in
such as food security, sustainable the context of societal challenges,
agriculture, health, climate etc. unclear whether this framework can be
▪ For narrower topics, possibility to replicated in other contexts.
reduce the number of interviews ▪ Highly time-consuming (50
▪ Originally designed for national interviews/20 focus groups) – require
research but adaptable to different either a large team of interviewers or a
geographical scopes. team of scientific officers.
▪ This method is already an engagement ▪ High direct costs if conducted offline.
tool in itself (interviews and focus
groups can help generate data).

Comparative table

180
 Between 4-8 weeks
(depending on Relies heavily on
Time investment external inputs and Internal/external external actors =
availability of risk of delays
interviewers)
Geographically
flexible, but unclear Very limited risks of
Flexibility Neutrality
if applicable beyond bias
societal challenges
Generate new ideas,
Experience in Learn and share
conducting lessons within
Expertise required Link to purpose
interviews/focus groups, Learn from
groups and consult experts
and stakeholders

Stakeholder identification in concept mapping

This methodology5 is a three-step creation process:
1- framework identification: define a relevant framework of stakeholder categories.
An iterative search in the scientific research is used to identify existing frameworks
and stakeholder categories used in related research. This lays the basis of the
stakeholder categories.
2- identification of specific stakeholder groups: this step starts by identifying
research disciplines relevant to the topic. Stakeholder groups related to these
disciplines are then added to the list of categories. This is then supplemented with
collaborative networks by searching for relevant working networks and groups online.
3- feedback solicitation: a few experts (less than 5) are consulted to ensure that the
framework obtained is an accurate reflection of the reality on the ground. Based on
the data gathered, a visual concept mapping method is used to illustrate the link
between stakeholder categories (see here and here).
Goal: identifying stakeholders within a research context requiring practical input from
the field

Target users Researchers

Geographical scale Any

Online/Offline Online

Example: this methodology was used in health research as a novel process applied to the
study of older adult mobility and the built environment. It was created to improve
methodological rigour in the selection of health research participants.

5
Claire Schiller, Meghan Winters, Heather M Hanson & Maureen C Ashe, 2013: A Framework for stakeholder
identification in concept mapping and health research: a novel process and its application older adult mobility
and the built environment. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3653754/

181
 Estimated resources

Relative time investment Low

Timeframe 1 to 2 weeks

FTEs and profiles 2 people, no expertise required

Direct costs Fee for access to scientific literature

Pros Cons

▪ Applicable to other public health topics. ▪ Requires broad access to the relevant
▪ Visual. scientific literature.
▪ Ultimately provides a set of categories
rather than actual stakeholders to
engage.
▪ Risk of identification bias in the
literature review and searches.

Comparative table
Between 1 and 2 External inputs only
Time investment Internal/external
weeks for validation
Easily adjustable to Minor risk of
Flexibility Neutrality
different contexts. identification bias
Simple Learn from and
Expertise required understanding of the Link to purpose consult experts and
topic stakeholders

Stakeholder Circle Methodology

WARNING: This method6 is federally registered with the US Patent and Trademark office.
This method works together with a tool called Stakeholder Circle®. It is used in project
management and consists of 5 steps:
1 – Identify stakeholders and understand their needs through an internal brainstorm
with the core research team. Stakeholders identified are categorised into:
▪ Upwards - senior managers of the organisation;
▪ Downwards - part of the project team;
▪ Outwards - stakeholders outside the project, such as end-users, government,
unions, shareholders; and
▪ Sidewards - peers of the project manager, such as other project managers.
2 – Prioritise stakeholders, based on their perceived power, proximity and “urgency”.
3 – Visualise the 15 first key stakeholders using the Stakeholder Circle® diagram.

6
https://www.stakeholdermapping.com/stakeholder-circle-methodology/

182
4 – Engage with the stakeholders by building and implementing a communication plan.
5 – Monitor changes over time to assess the effectiveness of the communication plan.
Goal: identifying stakeholders within a project management context with the aim of
developing and monitoring a communication plan.

Target users Project managers

Geographical scale Any

Online/Offline Online

Example: this methodology was used in health research as a novel process applied to the
study of older adult mobility and the built environment. It was created as a way to improve
methodological rigour in the selection of health research participants.

Estimated resources

Relative time investment Medium

Timeframe 2-3 weeks for steps 1 to 3

FTEs and profiles 3 or 4 people for a brainstorm.

Direct costs Access to the full Stakeholder Circle® toolbox for $38.50 AUD

Pros Cons

▪ Easy to use with guidance available. ▪ A business-oriented tool dedicated to

▪ Visual. communication. Cannot apply to all
▪ It already includes engagement as well phases of EFSA work.
as result monitoring. ▪ Lack of data on the flexibility and
▪ Internal process, doesn’t require adaptability of the tool.
external support. ▪ The method is patented and requires
buying the toolbox.

Comparative table
Between 2 and 3
Time investment Internal/external Internal only
weeks (until step 3)
Major risk of
Flexibility n/a Neutrality
identification bias
Generate new ideas,
Easy to use, Learn and share
Expertise required Link to purpose
guidance provided lessons within
groups, Inform

183
Prospex-CQI: Criteria-Quota-Individual
PLEASE NOTE: This method is IP protected.
This method7 is part of the Stakeholder Integrated Research approach (STIR) to
stakeholder engagement in research projects. The CQI is applied through a three-step
stakeholder mapping. CQI stands for:
▪ C- Criteria: Defining a set of criteria and categories for stakeholder groups that
are or could either be affecting the topic, be affected by it, or both;
▪ Q- Quota: Setting specific minimum quotas for all categories;
▪ I- Individuals: Identifying individuals that fit the categories, with the overall
selection fitting the quotas set.
The mapping is carried out with the objective of setting up stakeholder engagement
activities.
Goal: Identifying stakeholder with the aim of engaging with them as part of a balanced
audience with different levels of influence; avoiding missing on any stakeholder group

Target users Researchers

Geographical scale Any

Online/Offline Online and offline

Example: this methodology was developed in the EU-funded CLIMSAVE project for an
integrated assessment of climate change impacts, vulnerability and adaptation. It was
used to engage stakeholders in the development of an online tool supporting this
assessment in a range of sectors and highlighting interactions and side effects between
sectors.

Estimated resources

Relative time investment Low

Timeframe 1-2 weeks

FTEs and profiles 1 or 2 staff. No specific expertise required

Direct costs None

7
Marc Gramberger, Katharina Zellmer, Kasper Kok & Marc J. Metzger, 2014: Stakeholder integrated research
(STIR): a new approach tested in climate change adaptation research. Climatic Change 128(3) 201-214.
https://www.researchgate.net/publication/271738357_Stakeholder_integrated_research_STIR_a_new_approac
h_tested_in_climate_change_adaptation_research

184
 Pros Cons

▪ No expertise required. ▪ IP protection on the overall STIR

▪ Transparent, designed to avoid method.
identification bias. ▪ Stakeholder mapping rather than
▪ Applicable to various scientific contexts audience identification.
▪ Already part of a wider engagement ▪ Mostly applied in research context for
approach. engagement. Might not apply to
▪ Internal process, does not require communication.
external support.

Comparative table
Between 1 and 2
Time investment Internal/external Internal only
weeks
Usually applied to
Flexibility Neutrality Minor risk of bias
scientific research
Generate new ideas,
Easy to use, Learn and share
Expertise required Link to purpose
guidance provided lessons within
groups

Pyramid Research
This method8 uses a crowd-sourcing approach to knowledge-sharing. To conduct a
pyramid research on a given topic, one needs to:
▪ Identify the people who might have knowledge or interest in a given topic area
▪ Interview them and ask them to provide references of a person with a greater
expertise
After each interview, it is advised to assess how much knowledge was gathered to check
if one is climbing the right “pyramid”.
The process is then repeated until sufficiently high-positioned stakeholders have been
identified. It has been noticed that people in highest positions are then more likely to give
reference to experts in analogous fields.
Goal: identifying top experts on a subject matter, with the objective of engaging with
analogous fields and find transferable knowledge; gathering data on a topic with the
objective of finding creative solutions to an issue.

Target users Anyone

Geographical scale Any

Online/Offline Online

Example: An article from the Harvard Business Review uses the example of a forklifting
company looking for a solution to make unmounting forklifts safer. The company started

8
Marion Poetz and Reinhard Prugl, 2015: Find the Right Expert for Any Problem, Harvard Business Review.
https://hbr.org/2014/12/find-the-right-expert-for-any-problem

185
by contacting an intensive user of truck-mounted forklifts, who referred them to a maker
of machinery-mounting systems for farm tractors. The references went on until they talked
to a person in the entertainment industry specialised in unmounting stage equipment:
from an analogous field, the company gained transferable knowledge for their own issue.

Estimated resources

Relative time investment Potentially very high

Timeframe Difficult to assess, but probably up to 2 months

1 or 2 people with minimal expertise on the topic. Some

FTEs and profiles
experience with interviews is necessary.

Direct costs None

Pros Cons

▪ Can help build up a network. ▪ Difficult to assess time investment as

▪ Flexible and adjustable to many topics this relies greatly on responses.
and contexts. ▪ Risk of bias by the implementing
▪ This is an engagement method itself researcher(s) - individual values or
(for knowledge gathering). existing networks can affect the
▪ Good to use for creative problem- identification process.
solving. ▪ Only works with stakeholders who have
expertise on the topic.

Comparative table
Relies heavily on
Extremely variable
Time investment Internal/external external actors =
and hard to plan
risk of delays
Easily adjustable to Major risk of
Flexibility Neutrality
different contexts. identification bias
Generate new ideas,
Need experience Learn and consult
Expertise required Link to purpose
with interviews experts and
stakeholders

Identification Public-Public Partnerships

This method9 works on the assumption that the purpose of the engagement and the
desired level of engagement must be set up before identifying the target audience, as
this will determine the identification framework and tools used. In this context, two
methods are suggested:

9
Michael Dinges, Anna Wang and Anja Kongeter, 2015: Policy Brief on Stakeholder Engagement in Joint
Programming Initiatives ERA Learn 2020 1-15. https://www.era-learn.eu/documents/era-learn-
publications/policy_brief_stakeholder_engagement.pdf

186
1- Stakeholder analysis, in which information to determine whose interests should be
considered for an activity is gathered and assessed. This analysis leads to:
▪ A stakeholder register/repository which collects all stakeholder information,
categorises the stakeholders and gathers information such as stakeholders’
interests, involvement, expectations, importance, influence or impact. Tools such
as rating scales, influence diagrams or charts are useful to visualise and compare
levels of power, influence, interest etc.
▪ A stakeholder management strategy which lays down the approach for
increasing stakeholder support and reducing negative impacts represented in a
stakeholder analysis matrix - the tool requires resources including human
resources.
2- Expert judgment: external expert technical and/or managerial judgment.
The outcomes of this identification, including data about each stakeholder, will determine
the level of engagement required (lower levels = informative engagement, high levels =
empowering, decision-making engagement).
Goal: identifying and engaging stakeholders to foster responsible research & innovation
and increase potential impact of policy-making by assessing levels of interest and
influence.

Target users Anyone

Geographical scale Any

Online/Offline Online and offline

Example: The EU-funded project ERA LEARN 2020 which support research organisations
with information about Public-Public-Partnerships (P2Ps) applies this methodology to
improve the involvement of a wider community of actors in Joint Programmes Initiatives
(JPIs) and P2Ps.

Estimated resources

Relative time investment Medium

Timeframe Between 2 and 3 weeks

Requires good knowledge of the stakeholder landscape to lead

FTEs and profiles
the task (not a junior person) - 2 or 3 people
Might need to purchase a membership to a data management
Direct costs
or repository software

187
 Pros Cons

▪ Engagement-oriented. ▪ Risk of bias by the implementing

▪ Designed to improve impact. researcher(s) - individual values or
▪ Oriented towards civil society, assures existing networks can affect the
the societal relevance of research identification process and push smaller
output. stakeholder organisations out of the
▪ Applied to R&I impact discussions, very picture.
close to EFSA risk assessment ▪ Not used in a scientific setting.
processes. ▪ Might be a bit too analytical for a
simple target audience identification.

Comparative table
Between 2 and 3
Time investment Internal/external Mostly internal
weeks
Easily adjustable to Sizable risk of
Flexibility Neutrality
different contexts. identification bias
Requires some
seniority and good
Expertise required knowledge of the Link to purpose All purposes
stakeholder
landscape

Stakeholder identification beyond classification

PLEASE NOTE: This method is IP protected.
This method10 supplements usual stakeholder classification models with an identification
procedure for identifying real world parties. The method focuses on identifying
stakeholders in the context of innovation projects and answers the specific questions:
▪ what specific stakeholder fits within what specific category?
▪ how can a specific category be bounded in a justifiable way?
The identification method incorporates an identification procedure that explains how the
model is to be used. It facilitates actual identification through brainstorming with
experts on the innovation project. The identification method should be aligned with the
situations of the stakeholders in the innovation project.
Goal: identifying real world parties in line with the situations of stakeholders in innovation
projects.

Target users Researchers and project managers

Geographical scale Any

Online/Offline Online and offline

10
Janita F.J Vos and Marjolein C. Achterkamp, Staekholder identification in innovation projects: Going beyond
classification, European Journal of Innovation Management 9 161-178.
https://doi.org/10.1108/14601060610663550

188
Example: used in four cases in innovation projects.

Estimated resources

Relative time investment N/A

Timeframe N/A

FTEs and profiles N/A

Direct costs Access to the full methodology

Pros Cons

▪ Applicable to innovation projects but ▪ Very limited information on the

flexible for other contexts. methodology available.
▪ Aims at involving more “real world” ▪ No assessment of time investment and
stakeholders. expertise required.

Comparative table
Minor involvement of
Time investment N/A Internal/external
external experts
Sizable risk of
Flexibility N/A Neutrality
identification bias
Inform, Learn from
and consult experts
Expertise required N/A Link to purpose
and stakeholders,
Gather data

Mapping Stakeholders from Social Media

PLEASE NOTE: This method is IP protected.
This method11 is a conceptual framework of how organisations could use stakeholder
theory and Social Network Analysis (SNA) to pick out and prioritise stakeholders in social
media. It proposes an integrated model of stakeholder mapping that can be used to find
and prioritise offline and online stakeholders. A combination of Stakeholder Salience Model
(SSM) and SNA provides a direction and conceptual solution. The objective is to identify
“unknown” yet important stakeholder on social media. Two aspects of a stakeholder’s
online presence are assessed (connectivity and content) based on which social media
audiences are classified and integrated into the SSM model.
Goal: identifying stakeholders on social media with the goal of improving online
engagement.

Target users Industry and other organisations

11
Kristina Sedereviciute and Chiara Valentini, 2011:Towards a more Holistic Stakeholder Analysis Approach
Mapping Known and Undiscovered Stakeholders from social media International Journal of Strategic
Communication 5 221-239. https://www.tandfonline.com/doi/abs/10.1080/1553118X.2011.592170

189
 Geographical scale Any

Online/Offline Online and offline

Example: the methodology has been applied to public relations settings.

Estimated resources

Relative time investment N/A

Timeframe N/A

FTEs and profiles Requires an understanding of social media

39EUR for 24h access to the article with full detail of the
Direct costs
methodology

Pros Cons

▪ One of the only methods purely social- ▪ Very limited information on the
media-based. methodology available.
▪ Oriented towards online engagement. ▪ No assessment of time investment and
expertise required.
▪ Oriented towards public relations rather
than engagement.

Comparative table
Time investment N/A Internal/external Mostly internal
Possible risk of
Flexibility N/A Neutrality
identification bias
Inform, Learn from
Understanding of and consult experts
Expertise required Link to purpose
social media and stakeholders,
Gather data

Smart Sheet Stakeholder Mapping

This method12 uses pre-set templates for stakeholder mapping to help visualise the
stakeholder landscape. Three steps are suggested in this process:
1- Identifying stakeholders, individuals or organisations who have a stake in the project,
by brainstorming internally.
▪ It is advised to determine who are the relevant individuals within stakeholder
organisations to better target communication in the future. It is also advised, for
large lists of stakeholders, to group them by interest, and assess their importance
within their interest group.

12
https://www.smartsheet.com/free-stakeholder-analysis-templates

190
2- Prioritising them based on their interest and influence over the project using the
matrix provided.
▪ Identifying which stakeholders are likely to support or be critical of the project
might be useful at that point to think about early engagement.
3- Understanding stakeholders to better engage with them and adapt the
communication. Analysing the list of stakeholders, questions to answer here are:
▪ What motivates the stakeholders?
▪ What resources do you need from them?
▪ How will you deal with opposition from critical stakeholders?
▪ What is the best method for communication or engagement for different
stakeholders?
The method proposes different templates including easy-to-use analysis matrix (to
visualise levels of influence VS interest), stakeholder management spreadsheets but also
for strategy planning, communication plans, or stakeholder management.
Goal: visualising and organising better data about stakeholders to improve/develop
targeted communication.

Target users Project managers

Geographical scale Any

Online/Offline Online

Example: this methodology is used mostly for project management.

Estimated resources

Relative time investment Low

Timeframe Between 1 and 2 weeks

About 2 people with good knowledge of the topic and basic IT

FTEs and profiles
skills

Direct costs None

Pros Cons

▪ Visual. ▪ Risk of bias by the implementing

▪ Templates are free of charge. researcher(s) - individual values or
▪ Thought for communication plan existing networks can affect the
development. identification process.
▪ Flexible tool, can be applied to many ▪ Not used in a scientific setting
different settings. ▪ Targeted towards communication
rather than engagement.
▪ Limited guidance provided in the
methodology.

191
Comparative table
Between 1 and 2
Time investment Internal/external Mostly internal
weeks
Easily adjustable to Major risk of
Flexibility Neutrality
different contexts identification bias
Good knowledge of
Expertise required the stakeholder Link to purpose Inform
landscape

Expertise finding tools

Expertise finding refers to the use of tools to evaluate and identify individuals with relevant
experience in a field13.

Social Networking
One of the ways that this can be done is through social networking. An example of some
website/applications that can be used for expertise finding through social networking are:
LinkedIn, Academia.edu, ResearchGate, Facebook and Twitter.
One of the advantages of using social networking as a tool for finding expertise is that
there are many features available to consider, namely:
▪ webpages linked to users’ profiles.
▪ social relationships such as: Facebook friendships, Twitter mutual following
relationships, LinkedIn connections.
▪ resource containers: groups, Facebook pages, linked pages, the users that a given
user follows are also resource containers.
▪ Resources: publications in resource containers14.
A disadvantage is that profile information on certain social networks, e.g. Facebook may
be limited, as many members give the smallest amount of information that is needed to
register and do not explicitly state their interests and skills. However, this seems to be
inapplicable to LinkedIn, where most users maintain and update their profiles15.
Research shows that Twitter is the most effective social network, consistently
outperforming other social networks; it is particularly effective in fields such as science,
technology or computer engineering. When assessing people’s expertise, the information
about resources that they created themselves, own or annotated has shown to be more
effective than profile information. In addition to this, resources that have been created by
others, e.g. posts that appear on a person’s Facebook page, or material published by a
user’s Twitter follower/LinkedIn connection enhances the precision of expertise
assessment. Facebook seems to be the most effective tool in domains such as locations,

13
Jing Zhang, Jie Tang, Juanzi Li, 2007: Expert Finding in a Social Network. DASFAA 4443, p. 1066- 1069
https://link.springer.com/chapter/10.1007/978-3-540-71703-4_106
14
Alessandro Bozzon, Marco Brambilla, Stefano Ceri, Matteo Silvestri, Giuliano Vesci, 2013: Choosing the Right
Crowd: Expert Finding in Social Networks. Proceedings of the 16th International Conference on Extending
Database Technology, 638-648.
15
Ibid.

192
music, sport and movies and television. LinkedIn seemed to fair the worst in comparison
with the other social networks in all domains16.

Scientific literature
Another method of expertise finding would be through research of scientific literature.
There are many digital platforms that offer an array of publications, articles and papers
such as Academic Search, Analytical Sciences Digital Library, Index Copernicus, Science
Open, Semantic Scholar, Scientific Information Database and Google Scholar. Databases
such as: Science Direct, Web of Science, Wiley, IEEE Explore Springer Link, Scopus. The
advantage of using these websites is that they are usually free to use and offer a plethora
of different sources such as scientific journals, articles and publications.

Knowledge Base
Another expert finding tool is knowledge base. Experts can be found via databases such
as: Elsevier Expert LookUp, ExpertiseFinder, Authoratory, CoffeeChat.App.
These databases are often available at low cost and easily accessible. Elsevier Expert
Lookup, for instance helps identify scientific experts, find experts that fulfil funding
priorities and locate reviewers for papers and grant applications. Furthermore Elsevier
Expert Lookup can also be used to check whether there any potential conflicts of interest
with regard to funding streams and co-authorship17. Authoratory, another database, was
designed to find experts in the field of life sciences, including biology, chemistry and
medicine18.
Both of these databases also use mining techniques to provide accurate results. Elsevier
Expert Lookup employs the Elsevier Fingerprint Engine algorithms an inhouse created text-
mining technique19, while Authoratory uses software data-mining techniques to discover
new information about the authors and bring the researcher up-to-date20.
A potential disadvantage is that some of these databases offer limited results. For instance,
Expertise Finder is primarily designed for journalists to connect them with faculty experts
and academic experts. CoffeeChat.App adopted a unique way of receiving expertise-
through sharing your own, you are entitled to a phone call with an expert in a field of your
choosing21. While it is innovative it can also be time-consuming, and there is no guarantee
that an expert in a given field is actually available.

(Community) Threaded Discussions and Community Questions and

Answers
Community threaded discussions are convenient for locating community expertise. Up and
running forums and distribution lists for employees with expertise in a given field can
already be used and in the event that they are not a thought leader, activist or an individual
that is able to create and develop such a forum22. Some examples of community threaded

16
Ibid.
17
https://www.elsevier.com/solutions/expert-lookup
18
https://www.authoratory.com/
19
https://www.elsevier.com/solutions/expert-lookup
20
https://www.authoratory.com/
21
https://www.coffeechat.app/#how-it-works
22
Stan Garfield 2018, Expertise Locators Ask the Expert, Medium https://medium.com/@stangarfield/expertise-
locators-and-ask-the-expert-f273db1e227c

193
discussion forums include: Live Science Forum, Science Forums, International Food Safety
and Quality Network.
Similar to this, is Community Question Answering (CQA), which are web applications where
knowledge can be exchanged between users by asking and answering questions. Quora,
WikiAnswers, Yahoo!Answers and Answerbag are a few examples of CQA. A large amount
of users participate online, huge amounts of data is generated23 and elaborate answers
may also be provided to the posed questions, unlike traditional search engines that
retrieve information from pre-existent information repositories using keywords and
phrases24. Some of the disadvantages of CQA is that there may be thousands of questions
posted on a regular basis, making it difficult for an answerer to find and address questions
in connection to their field of expertise. Another downside is that the process can be quite
time-consuming, in addition to the time it takes to find the question, providing an elaborate
answer may also take ample time. However, research shows that CQA websites such as
Quora, comprise a set of highly dedicated domain experts who strive to fulfil the needs of
the user posting the query but also provide answers that have a high lasting value to a
larger audience25.

23
Chaoran Huang, Lina Yao, Xianzhi Wang, Boualem Benatallah, Xiang Zhang 2020, Software expert discovery
via knowledge domain embeddings in a collaborative network 130 Elsevier 46-53
https://www.sciencedirect.com/science/article/abs/pii/S0167865518308596
24
Chaoran Huang, Lina Yao, Xianzhi Wang & Manging Dong, 2018, A survey on expert recommendation in
community question answering 33 Journal of Computer Science and Technology 625-653
25
Ibid.

194