Milani Evidence-Based Policy Research PDF

Management of Social Transformations (MOST)
Policy is the priority

While it still promotes international, comparative and policy-relevant research
Evidence-based policy
on contemporary social transformations, MOST is now emphasizing the research
and policy interface as its major raison d’être. Tackling the sustainability of social
research: critical review
transformations is the programme’s main task, which implies action at normative,
analytical and strategic/political levels. It must concentrate on research of direct use
of some international
to policy makers and groups involved in advocacy.
MOST’s emphasis is thus on establishing and interconnecting international
programmes on
policy networks with renowned social science researchers to facilitate the use of
social science research in policy-making. This means bringing together basic research
relationships between
with those entrusted with policy formulation in governments, institutions, actors and
in UNESCO itself. social science research and
Tools for policy-making policy-making
The Policy Papers, dedicated to social transformations and based on policy-relevant
research results of work carried out by MOST and by other sections of the Social
and Human Sciences Sector (SHS), are intended for policy makers, advocacy groups, Carlos R. S. Milani
business and media.
SHS is seeking new ways of distributing knowledge to target groups, such as
ministers of social development, advocacy groups, UNESCO National Commissions
and local authorities. It has launched a tool for online knowledge management
and meta-networking for decision-making and strategy. This knowledge repository
will use innovative and refined search tools to facilitate access and intelligibility of
Management
complex research data for all potential users.
of Social
Transformations
www.unesco.org/shs/most Policy Papers / 18

Evidence-based
policy research:
critical review of
some international
programmes on
social science research
and policy-making
Carlos R. S. Milani
Federal University of Bahia, Brazil
Policy Papers / 18
MOST-2 Policy Papers series uses a novel methodology aimed at enhanced
dissemination and usability of research results for policy-making. Designed
according to scientific policy analysis principles, this methodology is based on
a generic structure for producing documents.
The generic structure first enables different types of documents to be produced

from the same original content. For instance, collections of the summaries of
the various sections from the Papers produce Policy Briefs (5 pages condensed
versions). Both Papers and Briefs are available in print and in electronic
versions.
The structure also gives all documents the same appearance,

so ease of reading improves with familiarity of the format.
A better indentation of the text further improves the location and utility
of the information: the content of each section in the document becomes a
fully-fledged knowledge item that's easy to spot, extract to be better studied,
compared and put into perspectives.
This logic serves as the foundation for the interactive Policy Research Tool that
MOST is currently developing. The online tool will provide free and speedy access
to policy-relevant comparative information, giving users the ability to create
research profiles based on subject categories, produce customized reports with
select content from the original documents, and easily compare cases and assess
the relevance of the policy options available.
MOST-2 methodology helps respond more efficiently to different types of

information needs and facilitates knowledge feedback and analysis, thus
improving the use of research results for policy-making.
Carlos R. S. Milani ([email protected]) is Professor of International Relations at

the Institute of Humanities, Arts and Sciences of the Federal University of Bahia
(UFBA, Brazil). He is also a CNPq research fellow, and the coordinator of the
World Political Analysis Laboratory (LABMUNDO, www.labmundo.org). His
recent publications include articles and books on transnational social move-
ments, discourses and practices of international cooperation agencies, as well
as globalization and world politics theory.
The ideas and opinions expressed in this publication are those of the author and do not necessarily
represent the views of UNESCO.
The designations employed and the presentation of material throughout the publication do not
imply the expression of any opinion whatsoever on the part of UNESCO concerning the legal status
of any country, city or area or of its authorities, or concerning its frontiers or boundaries.
Published in 2005 by the United Nations Educational,

Scientific and Cultural Organisation
1, rue Miollis
75732 Paris Cedex 15
France
SHS-2009/WS/8 – // CLD 1676.9
© UNESCO 2009
Printed in France
Contents
Executive summary 5
Foreword 7
Introduction 9
Part 1: Social science and policy-making:
the work of some multilateral
and bilateral organizations 13
Case 1: The United Nations University (UNU) 14
Case 2: The European Union and the European Research Area 20
Case 3: The World Bank 25
Case 4: The action of the Organisation for Economic
Co-operation and Development in the field of education 33
Part 2: Sketching a series of critical questions

on the relationships between
social sciences and policy-making 41
Some questions for the future

Research on other universalities
of the research-policy linkages 51
Work on what already exists within MOST:
explain the functions and objectives
of tools that are already available 52
Define MOST’s main partners and beneficiaries 53
Develop a “pedagogy of knowledge-
policy dialogue” in the field of social development 54
Boxes
Box 1: Excerpts from Policy Brief 3 (2005). Theme: Spatial
Disparities
Box 2: Excerpts from Policy Brief number 2 (2005). Theme: Peace
and global governance
Box 3: Green Paper. ERA Knowledge-Sharing: main highlights
Box 4: Policy Research Working Paper “Globalization and innova-
tion in emerging markets”, by Gorodnichenko, Yuriy; Svejnar, Jan;
Terrell, Katherine (2009).
Box 5: CERI Workshops on Evidence-Based Policy-Making in the
field of Education.
Executive Summary
The report has two main objectives. Firstly, to critically present current
programmes implemented by international organizations, which seek to
enhance the interconnections between social scientific research and policy.
Secondly, to highlight a series of critical issues with regard to the nature of
the linkages between the two domains. This double venture and the debate
it provokes ultimately aim to enable the MOST Secretariat to carve out its
own niche in relation to other major organizations working in this field.
The first part proceeds to offer an empirical overview of the differ-
ent ways that international multilateral organizations (United Nations
University, the European Union, the World Bank, and OECD) build bridges
between social scientific research and policy. Via this examination the
paper reaches the conclusion that what one largely deduces from these
ventures is that they are based on a rather positivist social scientific vision.
Hence they, more often than not, lead to (with varying degrees) a rather
conformist understanding of the link between the domains of research
and policy (i.e. the emphasis is on evidence based research – with a strong
quantitative dimension – providing solutions to problems).
In the second part, the MOST Policy Paper via an exploration of the
complex links between politics and social scientific knowledge convinc-
ingly argues that the aspiration to universal applicability (and by exten-
sion to an ultimately benevolent problem solving character) on the part of
evidence based policy making is problematic. Consequently, he suggests
Executive Summary
that alternative social scientific perspectives are legitimate and that the
accumulation of scientific evidence that, nevertheless, does not address
conditions of unequal distribution and/or disempowerment will not lead
to significant social improvements.
It is possible to have bad evidence-based policy making if the evidence

used is biased, flawed or incomplete. One could also say that, depending
on the purposes of data collection, evidence may serve unfair and unjust
policy objectives. Statistical data as well as cartography are “texts”, and
may be skillfully controlled and technically manipulated. (p. 45) 5
The text promotes a vision of a critical and engaged social science
that problematizes current policy practices and visions and, thus creates
the possibility for social transformation: “… research for policy is not so
much about providing answers as about changing the way questions are
understood, so that people (researchers and policy-makers, but other publics
Evidence-based policy research: critical review of some international programmes
too) can begin to think differently, thus critically building the contours and
contents of social problems”. (p. 49)
On the basis of this discussion, a series of research questions are
opened up which could guide the work of MOST and the methodology
it wishes to create and promote. These fall, mostly, in two categories:
Questions that deal with the historical trajectory of the institutionalisation
of instrumental social scientific research in international organisations and
national research-policy set ups. And questions that pertain to the policy
implications of a new, more critical, problem building research-policy
paradigm (e.g. a new role for politicians, the inclusion of civil society in
the making of decisions, creating a public sphere for the dissemination of
social scientific results and preoccupations which can thus reach a wider
public).
6
Foreword
Launched in 1994, the Management of Social Transformations – MOST

Programme, which is part of the Social and Human Sciences Sector of
UNESCO, was designed to steer reflection and action in the vast field of
Social Transformations. While its original mandate established a commit-
ment to the promotion of research that was comparative, international,
interdisciplinary and policy-relevant through the development of three
thematic fields, in its Second Phase (2004-2013) the Programme has been
reoriented, both thematically and in its modalities of operation.
The focus is now on building efficient bridges between social science
knowledge, public policies and practice. Here, knowledge means what is
produced within universities and academia, as well as non-academic expe-
rience and knowledge (for instance, within non-governmental organiza-
tions). At the same time, the making of public policies involves govern-
mental and non-governmental actors, which means that the conception of
public policy-making in this second phase of MOST presupposes complex
dynamics of politics, including issues relating to:
• recognition of identities (those social subjects and demands that
are included in the policy-formulation agenda);
• participation of actors (those actors who are invited to take an
active part in the decision-making process);
• the nature of norms (the different kinds of policy norms dealing
with universality and/or particularity, general objectives and/or
focused results); and
• co-responsibility in implementation (the monopoly of the State in
public action versus pluralistic approaches in public service provi-
sion and public-private partnerships).
Foreword
The aim of the venture (theoretical and methodological reflection on

the linkages connecting research and policy) is to make explicit the nature
of the link between social science research and policy-making, in developed
as well as developing countries. In other words, the MOST Secretariat is
working to establish critical analysis on what the world knows about the
7
theoretical and institutional underpinnings of both knowledge production
and its uptake by policy makers. The activity concentrates on four goals:
1. To determine an efficient methodology for social science research to
be optimally inserted in policy-making processes
2. To capitalize, promote and diffuse our knowledge and understanding
to three main categories of actors: social science researchers, policy

makers/senior advisers and members of civil society
3. To define the niche that will be developed by the MOST Programme
on the links between research and policy
4. To make recommendations for internal and external stakeholders.
The six-year course of activity has been split into three biennial
periods. One of the main fields of action in the current biennium (2008-
2009) includes mapping out the general, current state of Research-Policy
Links. The series of MOST Policy Papers will present some of the findings
of this endeavour.
GERMÁN SOLINÍS
8
Introduction
The MOST Programme, which is part of the Social and Human Sciences
Sector (SHS) of UNESCO, launched in March 1994, was designed to steer
reflection and action in the vast field of linkages between social science
knowledge and public policies. Knowledge here means what is produced
within universities and academia, but also non-scholar experience and
knowledge (for instance, within non-governmental organizations). At the
same time, the making of public policies involves governmental and non-
governmental actors, which means that the conception of public policy-
making during this phase of MOST, supposes complex dynamics of politics,
including issues relating to recognition of identities (those social subjects
and demands that are included in the policy-formulation agenda), par-
ticipation of actors (those actors who are invited to take an active part in
the decision-making process), the nature of norms (the different kinds of
policy norms dealing with universality and/or particularity, general objec-
tives and/or focused results), and co-responsibility in implementation (the
monopoly of the State in public action versus pluralistic approaches in
public service provision and public-private partnerships).
One can say that MOST’s primary purpose, since the beginning of
its activities, has been to transfer social research findings to public deci-
sion-makers. During its first eight years, the Programme’s mandate had
established a strong commitment to the promotion of research that should
be comparative, international, across social science disciplines and policy
relevant, mainly through international research networks. During this first
phase, the main instrument used by MOST for research dissemination is its
Clearing House – which functions basically as a portal containing informa-
tion on research networks, their results, proceedings, and “best practices”
9
on urban, migration, multiculturalism, global governance, as well as local
development policies.
According to the recommendations made by the Intergovernmental
Council in February 2003, there was a thematic reorientation of the
Programme, as well as a change in its modalities of operation. According
to information available on its website, in its second phase “MOST focuses

on building efficient bridges between research, policy and prac-
tice. The programme promotes a culture of evidence-based policy-
making – nationally, regionally and internationally. As the only UNESCO
programme that fosters and promotes social science research, it is placed
in a pivotal position in the overall promotion of UNESCO’s goals” (our
emphasis). It is unclear whether or not the modalities of action (for instance,
policy papers, briefings, organization of workshops and seminars) have
changed in the transfer of social science knowledge to decision-makers.
Internet MOST pages do not provide clear information on the four specific
tools that have been developed in its second phase: (i) the policy papers
series (which already existed before 2003); (ii) the digital library (which
does not seem to be functional); (iii) the conference reporting system
(which gives access to conference proceedings); and (iv) the origin and
the development of the policy research tool.
Having introduced these contextual remarks, this report has two
main objectives. First, to present and analyse current programmes
implemented by some multilateral and bilateral organizations,
trying to understand their philosophical foundations, ontology, meth-
odology, hierarchies and distinctions set up in the field of social science
research, as well as their institutional framework. This report focuses on
four major organizations: the United Nations University, the European
Union, the World Bank, and OECD showing thematic and interdiscipli-
nary case studies that illustrate their choices, priorities, mechanisms, and
procedures. Secondly, it aims to sketch a series of critical issues in
understanding the complex and paradoxical relationships between the
production of social science knowledge, and the politics of policy-making.
The idea of a politics of policy-making – which implies a life of potential con-
flict and a field of on-going negotiation around political cultures, actions
and practices – will be explained in detail in the second part of the report,
10 when we also outline some recommendations that could support the

MOST Secretariat in developing a more focused strategy in the coming
years – always trying to bear in mind MOST’s concrete investment possi-
bilities and its political niche in comparison with other major international
organizations working in this field.
Introduction
11
Part one
Social science and policy-making: the work of

some multilateral and bilateral organizations
Four major international organizations are analysed in this part of the
report: the United Nations University, the European Union, the World
Bank, and OECD. The basic criteria for selecting these organizations are
the following:
(a) The four organizations in focus have put into place a series of
projects and debates to foster the importance of policy social
research and evidence-based public policies. They have also pub-
lished documents, books, policy papers, etc. that reflect upon the
linkages between social science and policy-making.
(b) Their means, both human and financial, are of great relevance,
and their contribution to the construction of models based on
their own work seems to be considerable, both regionally and
worldwide.
(c) We also make this choice because we suppose that the four
selected organizations tend to express different cultural and
political perceptions and interpretative frameworks of how social
transformations, development, and governmental policies should
be globally thought of, decided upon and implemented. In other
words, we suppose that this variation in world visions should also
result in distinct conceptions of how social science research and
public policies relate to each other.
(d) The four selected organizations also announce in their programmes
that they adhere to diverse methods of work in the implementa-
tion of answers in order to promote closer ties between social
science knowledge and the making of public policies.
13
Methodologically speaking, it is important to assert that this report
is the result of information that was exclusively collected by means of the
international organizations’ Internet pages, since there has been no pos-
sibility to undertake interviews and/or produce primary data for this analy-
sis. This is a clear shortcoming of this report that must be acknowledged at
its very introduction. We do believe, however, that websites and Internet

reports show what these organizations intend to render visible to a larger
public audience, thus making the analysis of discourses and narratives that
we produce herein valid on the basis of the material that has been used
and consulted.
Case One: The United Nations University

(UNU)
UNU has five major roles according to its Charter: (a) UNU is an interna-
tional community of scholars; (b) it is a bridge between the United Nations
and the international academic community; (c) it serves as a think tank
for the United Nations system; (d) it contributes to capacity develop-
ment, particularly in developing countries; (e) it serves as a platform for dia-
logue and creative new ideas. Since its foundation in 1975, the University
has evolved into a global network of 13 UNU research and training centres
and programmes, and 14 associated institutions which carry out joint pro-
grammes with the University. Its two broad programme areas are “peace
and governance”, and “environment and development”. UNU’s principal
stakeholders include inter alia the UN system and the Bretton Woods insti-
tutions; regional organizations and national and local governments; civil
society networks and organizations; and private corporations (through
particular partnership and funding schemes).
UNU develops its mandate in the field of science-policy linkages focus-
ing on two major modalities: (i) building situational diagnoses on several
issue areas, and (ii) transforming multidisciplinary research and knowledge
into policy-relevant prescription and analysis. Within this second modality,
it also develops models and scenarios, and evaluates the implications of dif-
ferent policy options. In fact, UNU presents itself as a think-tank for the UN,
and attempts to give answers to complex global governance questions
14
and to communicate them to a diverse audience, including Member States
and academia, but also the private sector and NGO networks.
UNU affirms that its science-policy communication activities include,
for instance: (a) the involvement of practitioners in research by facilitat-
ing their participation in workshops and by bringing in mid-level manage-
ment from key organizations as visiting scholars for two to three months in
order to provide an opportunity for joint reflection; (b) quick publication
of policy/research briefs for each research project; (c) policy advocacy
(direct or through the media, particularly targeting national governments,
civil society organizations, and global institutions); (d) organization of
policy forums to disseminate the implications of UNU work; (e) undertak-
ing more rapid short-term projects or forums on urgent economic,
political and social problems. These short-term and urgent projects
require a particular methodology, since they suppose more consultation
with end-users in the formulation and implementation of such policy-
oriented projects.
In its Strategic Direction Report (2007-2010), UNU reaffirms the need
to disseminate information for policy-makers in a format that is accessible,
mainly through short, readable policy and research briefs. As a matter of
fact, UNU has been publishing “Policy Briefs” since 2005, mainly based
on the research done by four of its institutes: WIDER (World Institute for
Development Economics Research), whose major thematic interest has
so far been research on spatial disparities; CRISP (Center for Research
Innovation and Science Policy), working on issues relating to sciences and
technology policies; EHS (Institute for Environment and Human Security),
whose motto is: “Advancing human security through knowledge-based
approaches to reducing vulnerability and environmental risks”; and IAS
(Institute of Advanced Studies) based in Tokyo, whose thematic field
includes issues relating to global environmental governance and sustain-
able development.
Part one
It is interesting to analyse, as an illustration of the kind of dialogue

set up between social scientists and policy makers, WIDER Policy Brief
number 3 (2005) devoted to spatial disparities. This policy brief states that
there are some “appropriate” policies to be followed and implemented
(Box 1). Inequalities and social disparities are presented across individuals
and regions, but not across groups and social classes. The policy maker
15
who reads this information does not have access to the policy responses,
but only to the diagnoses. He/she needs to go further in reading the whole
UNU-WIDER research report. Nevertheless, one interesting aspect that can
be seen in the text relates to the contradictory nature of development that
is presented in the document, since spatial agglomeration and economic
concentration are shown to produce both benefits and adverse effects. It is

true, however, that the language used in this policy brief may be criticized
for being excessively politically correct.
Box 1
Excerpts from Policy Brief 3 (2005)
Theme: Spatial Disparities
“There are two reasons why policymakers should be concerned
about spatial inequality, defined as inequality in economic and
social indicators of well-being across geographical units within
a country. First, inequality between a nation’s regions is one
component of overall national inequality across individuals
(the other component being of course inequality across
individuals within each geographical unit or region). When
spatial inequality goes up then, other things being equal, so
does national inequality. Second, inequality between a nation’s
regions may be of concern in and of itself, especially when the
geographical regions align with political, ethnic, language,
or religion divisions. The ‘new economic geography’ has
emphasized that there are powerful forces of agglomeration
that tend to lead to a concentration of economic activity,
magnifying natural geographical advantages that a region may
enjoy. Thus spatial agglomeration brings the benefits of returns
to scale, and hence helps efficiency and growth. At the same
time, openness to the outside world, which is well recognized
as a long-term source of efficiency and growth, can also lead
to spatial concentration. The evidence presented in the UNU-
WIDER project is clear, spatial inequalities are high and rising.
What should be the policy response, bearing in mind the
tradeoffs involved?”
16
Another illustration of policy advice in the field of reform of global
political institutions can be found in Policy Brief number 2 (2005), written
by Andrew S. Thompson (University of Waterloo, Canada). Making a
summary of a book edited by John English, Ramesh Thakur and Andrew F.
Cooper, and published under the auspices of UNU, this policy brief sug-
gests the creation of a Leaders’ Summit of 20 (L20) that draws its inspira-
tion from both the current G7/8 leaders’ meetings and the G20 finance
ministers’ meetings. The purpose of such an informal organization would
be to bring together the leaders of key non-central States (i.e. other than
Western Europe, North America and Japan) in the hope that this type of
gathering of governmental peers (Presidents and Prime Ministers, minis-
ters and national secretaries) might lead to resolutions of issues over which
deadlock has so far been the norm. The policy brief analyses the benefits
and unfavourable aspects of such a non-formal setting, since it does not
follow the rules and procedures of intergovernmental bodies (UN, World
Bank, WTO, etc.). The author of the document supposes that through
non-formal institutions it is more feasible to reform and render the global
political architecture more democratic. The text formulation does not
eschew dealing with conflicts and politics – in this case, the global politics
involving institutional actors, economic operators and corporate media,
transnational social movements and activist networks – which are viewed
as crucial matters in the future framing of global governance mechanisms.
Neither does it announce any magic answers to global governance prob-
lems and shortcomings, thus avoiding the “best practice” formula
that is so frequently used (and abused) by UN Agencies in general.
Part one
17
Box 2
Excerpts from Policy Brief number 2 (2005)
Theme: Peace and global governance
“Whether the L20 ever gets off the ground is, in the end, up
to the world’s leaders. As outlined above, there are a number
of reasons why it could fail. Questions pertaining to the

membership and the scope of the agenda still need to be
answered, as do the questions about its potential relationships
with the current international architecture. But these challenges
are not impossible to overcome. Ultimately, the case in favor of
establishing an L20 is quite strong, if for no other reason than
the fact that maintaining the status quo no longer makes sense.
While reforming existing institutions is one solution, it is not
the only one: the international community must also decide
whether the time has come for new institutions such as the L20
to be created. If it chooses to take this route, Reforming from the
Top: a Leaders’ 20 Summit offers important guidance on how to
give shape to what remains merely an interesting idea”.
In the case of the UNU Institute for Environment and Human Security
(EHS), the strategy has been slightly different in terms of dissemination
and reaching out to policy-makers. UNU-EHS has been conceived of and
set up in order to improve the knowledge base for assessing the vulnerabil-
ity and coping capacity of societies facing natural and man-made hazards
in a changing and often deteriorating environment. The Institute aims to
improve the understanding of cause-and-effect relationships, and to offer
options to help reducing the vulnerabilities of societies. Interdisciplinary,
science-based and human-centred, EHS projects are said to support deci-
sion-makers by providing “authoritative research” and information within
its mandate. Starting this publication called “InterSecTions” (International,
interdisciplinary inSecurity ConnecTions) in 2005, the EHS aims to make
the results of its work known through publications in refereed scientific
journals, conference proceedings and books. However, this is only one part
of the story, particularly because those who read such journals and docu-
ments tend to constitute focused communities of interest and specialized
audiences. That is why EHS has also decided to launch this publication
series (both in paper and electronic versions), including short articles and
18 monographs. One question should, however, be put straight away: can

policy makers and public professionals be addressed optimally through
such kinds of scientific publications? Going through some issues published
under the “InterSecTions” series, a careful reader realizes that there must
still be an effort in terms of language adequacy and understanding
of complex issues relating to environmental change by potential policy-
maker readers. One could wonder how decision-makers can have access
to risk analysis and policy scenarios built on global environmental com-
plexities, particularly on topics that are very relevant to daily social and
economic realities, if the content is not accessible and intelligible in a more
simple fashion. It is true that Bonn’s EHS distributes these publications free
of charge; each issue is available for downloading on its respective series
webpage,1 but their content, it seems, still needs some sort of pedagogic
revision. Thinking of “a pedagogy of knowledge transfer” and the
rationale of research-policy linkages in concrete issue areas could be a hint
for MOST in its future development.
In addition, it is important to say that, like many other UN institutions
in the aftermath of the launching of the Global Compact by Kofi Annan at
Davos in 1999, UNU is also involved in promoting alliances with the corpo-
rate sector, through its “Partnership Initiative”. UNU acknowledges that it
is important to “engage private enterprises in the work of governments and
the United Nations, to build sustainable and equitable societies together,
and to mobilize comparative advantages in pursuit of a world free from
want and from fear”. It reaffirms that business companies recognize that
“addressing global problems such as poverty, environmental deterioration
and trade imbalances through a Corporate Social Responsibility program is
an investment opportunity”. Through this Partnership Initiative (and here
we cite the institutional rhetoric in the field of public-private partnerships),
UNU supports efforts for national governance programmes together with
long-term private business investment; corporate partners may enhance
their credibility, image and reputation; and corporate partners may gain
Part one
first-hand access to the University’s extensive international networks.

Moreover, private partners can have first-hand access to UNU research
that shapes global values and policies. What is the role for the corporate
1 We went through two issues of the series: Nothing begets nothing: the creeping disaster of
land degradation, by Paul L. G. Vlek (2005), and also Control, Adapt, or Flee: How to face
environmental migration? by Fabrice Renaud et al (2007). 19
sector in policy-making and social science research funding? What are the
dangers for the future development of democracy when corporations are
directly involved in public decision-making and funding of policy-relevant
social science research? By quoting such remarks from UNU Internet pages
and raising these questions we intend to suggest that some reflection is
needed on the actual results of public-private partnerships in the funding

of social science research, and the involvement of private companies in
public policy-making and implementation. We will come back to this issue
in the second part of this report.
Case 2: The European Union and

the European Research Area
In 2000, the EU decided to create the European Research Area (ERA). This
has meant starting the development of a future unified research area all
across Europe. This ERA should enable researchers (i) to move and interact,
benefit from high-level infrastructures and work with networks of different
European research institutions; (ii) share, teach, value and use knowledge
effectively for social, business and policy purposes; (iii) optimize European,
national and regional research programmes in order to support the best
research throughout Europe; (iv) develop strong links with partners around
the world so that Europe benefits from the worldwide progress of knowl-
edge, contributes to global development and takes a leading role in interna-
tional initiatives to solve global issues. Moreover, this research area should
inspire the best talents to enter research careers in Europe, incite industry
to invest more in European research – contributing to the EU objective to
devote 3% of GDP for research, and strongly contribute to the creation of
sustainable growth and jobs. It is true that such a huge strategic change
is still in the making in Europe, and that the reality of research is very
diverse across the European continent. The creation of ERA coincides with
budget cuts and reduction of posts in many European countries, which is
just one expression of contradictions between what Brussels announces
as its policy priorities and European national realities. With 80 per cent of
public sector research in Europe being conducted at national level, mainly
under national or regional research programmes, spending on science and
20 research in the European Research Area is still far too low (around 1.9% of
GDP) for the region to catch up with the United States (approximately
2.6% of GDP) or Japan (2.7%).2
According to information made available in EU Research Directorate
reports, seven years on the creation of ERA has become a central pillar of
the European Union for growth and jobs, together with the completion of
the Single Market, the European innovation strategy and the creation of
a European Higher Education Area. Today, there are still strong national
and institutional barriers which prevent ERA from becoming a reality.
Fragmentation remains a prevailing characteristic of the European public
research base. Researchers still see career opportunities curtailed by legal
and practical barriers hampering their mobility across institutions, sectors
and countries. Businesses often find it difficult to cooperate and enter into
partnerships with technological research institutions in Europe, particularly
across countries. National and regional research funding remains largely
uncoordinated. This leads to dispersion of resources and exces-
sive duplication. Reforms undertaken at national level often lack a true
European perspective and transnational coherence.
For these reasons, the European Commission has published a Green
Paper reviewing progress made with respect to the European Research
Area, raising questions for debate. The Commission sought answers to
these questions and solicited further new ideas in a public consultation
which lasted from May to August 2007. Following the public consulta-
tion results (over 800 written submissions), the Commission and Member
States launched a series of new initiatives to develop this research area,
called the “Ljubljana Process”, and five initiatives on specific areas of the
ERA Green Paper. As far as knowledge-use and science-policy linkages are
concerned, the Green Paper and the consultation process show some
interesting features. It acknowledges that access to knowledge generated
by the public research base and its use by business and policy makers lie
at the heart of the European Research Area, where knowledge should cir-
Part one
culate without barriers throughout the whole society. Another feature that
should draw our attention is that social sciences are very rarely referred to.
The Green Paper mentions the central relevance of effective knowledge-
2 See: Euractiv Foundation at www.euractiv.com 21

sharing notably between public research and industry, and in this case
exact sciences and engineering are also considered important.
In the consultation process, for instance, it is said in the Green
Paper (The ERA New Perspectives, Public Consultation Results, page 68)
that “regarding the main factors hindering efficient knowledge transfer
to industry, most of the 528 on-line respondents consider cultural dif-

ferences between the business and science communities to be a ‘very’
(293) or ‘fairly’ (146) important barrier”. On page 74, it reaffirms that
it is very important to increase the transparency of how scientific results
feed back into policy-making and ensure multidisciplinary expertise in
decision-making processes. Expertise and feedback are considered least
intrusive for scientific communities, those which demand the least trans-
formation of the status quo in the world of sciences. However, the Green
Paper Consultation Process shows that there is considerable debate over
other possible mechanisms, such as training scientists in societal issues or
expertise by civil society organizations. On page 75 of the Green Paper,
there is another mention of the issue of dialogue between researchers
and civil society. Responses during the consultation process highlight
that the major contribution of researchers to citizens’ better understanding
of the role of science has been the clarification of the social relevance of
research for policy-making. Here, the matter of an “advocacy of research-
policy linkage” seems to appear as an important issue. On page 78, one
can read that “scientifically generated knowledge must be readily available
for decision-making purposes. Of course, this requires the creation of fast
and easy mechanisms for contacts between decision-makers and research-
ers”. Box 3 below summarizes some highlights of the Green Paper as far as
knowledge-sharing is concerned.
22
Box 3
Green Paper
ERA Knowledge-Sharing: main highlights
“Developing communities of knowledge where the differing
worlds of research, industry and civil society can engage in
processes and networks of communication is deemed a sine
qua non for a well-grounded European Research Area. Cultural
differences between the business and scientific communities
and a lack of incentives for inventors or users remain major
obstacles to efficient knowledge transfer. The knowledge gap
between scientific communities and civil society, followed
by lack of incentives and the use of technical language, are
highlighted as the main factors hindering efficient knowledge
transfer to civil society. Beyond dissemination of scientific
knowledge, there is a broad agreement on deepening public
engagement in research with interactive approaches and
increased transparency on using scientific results in policy-
making.”
The European Union has also a particular portal for what they have
called “European Research in Action”, which focuses on the results of
European research into the understanding and better formulation of issues
of wide public interest. This impressive database is available in eleven lan-
guages, and presents information on several societal contemporary prob-
lems, including reports and information stemming from socio-economic
research.3 The topic devoted to socio-economic research, which we have
analysed in more detail, presents the following thematic priorities:
(a) Societal trends and structural changes: research carried out in this
area identifies societal trends such as changing family structures,
cultural patterns and value systems. It acknowledges problems
such as xenophobia and racism. The economic changes affect-
ing the labour market, and mechanisms for social protection,
Part one
inequality and discrimination are also investigated. This research

area aims to provide a better understanding of changing employ-
ment patterns such as flexible, part-time and temporary work,
and to provide information on changing gender stereotypes.
3 See: http://ec.europa.eu/research/leaflets/index_en.html. 23
Furthermore, research focuses on education and training, and the
role these two key variables can play in preparing individuals for
the changing political and socio-economic environment.
(b) Technology, society and employment: these research programmes
examine the ways in which suppliers, users, decision-makers
and public authorities work together in assessing technological

impact, and how public authorities, in particular, can then apply
the results. The second main research under this item assesses the
link between technological changes and employment. Here, work
is being undertaken on tele-working, work organization and skills
development. Another theme of this research area examines the
role of innovation in education, training and life-long learning.
(c) Governance and citizenship: research in this field aims to identify
and understand how multiple levels and new modes of govern-
ance can be developed, while maintaining accountability within
the context of the European integration process. The challenge
presented here is to ensure that democratic systems work prop-
erly under new conditions, such as the changing roles of public
and private sectors, and the changing relationships between rep-
resentative institutions and civil society organizations. At the same
time, it tries to understand the formation and coexistence of mul-
tiple actors, and participation of Europe’s citizens in public debate
– which is instrumental in formulating and implementing policy.
Research currently under way is presented on how the media
and cultural elements like language, history, gender, religion and
migration influence this.
(d) New development models fostering growth and employment: this
work explores new sustainable development models which foster
growth, job creation, equal opportunities, and reduce inequality
while improving the quality of life. Within this process, the role of
the public sector is explored and several indicators and method-
ologies are developed to assess the economic and social added-
value of production models. This helps to identify policies best
adapted to the European economic area, which take into account
regional and demographic differences. Lastly, research concen-
24 trates on the analysis of issues such as organizational innovation,

and new types of work and employment, including the working
potential of the older population.
This database is a powerful instrument of knowledge-transfer, and
MOST should reflect upon its financial means and human capaci-
ties to invest in the development of new databases as tools to
promote the dialogue with policy makers in general. Concerning other
knowledge-transfer mechanisms, the “European Research in Action” portal
mentions the “Dialogue workshops”, which bring together research-
ers, policy makers, NGOs and concerned citizens to share and debate
results of research, and assess their implications for policy-making. These
workshops are said to have gradually replaced the “top-down approach”,
where experts disseminated information after consulting the citizens. The
workshops have involved people who have practical knowledge and can
provide feedback to policy processes. To date, workshops have been held
on a range of topics including: employment funding, family and welfare,
European citizenship, migration and social cohesion of migrants, racism
and xenophobia. It could be of great interest for the MOST Programme to
analyse the construction of the bridges within such policy processes.
Case 3: The World Bank

Three main activities have been analysed within the broad spectrum of
programmes implemented by the World Bank in the field of social science
and public policy linkages: the Policy Research Reports (and related Policy
Research Working Papers), the Knowledge for Change Program, and
Modeling Tool to Monitor the MDGs (addressed to policy makers, and
based on the Development Data Platform). First, the Policy Research
Reports (PRRs) aim to bring to a broad audience the results of World Bank
research on development policies. These reports are designed to contrib-
ute to the debate on appropriate public policies for developing econo-
Part one
mies (our emphasis). PRRs are supposed to help policy makers take stock of
what is known and clearly identify what is not known, and they should thus
contribute to the debate in both the academic and policy communities on
adequate public policy objectives. Because they summarize research, the
PRRs are said to provoke further debate, both within the Bank and outside,
concerning the methods used and the conclusions drawn. With regard to
25
these Reports, the WB recalls that the Policy Research Working Papers are
more addressed to Bank researchers and the design of future Bank pro-
grammes (see example in Box 4).
In both types of World Bank reports, the production of empirical
evidence rooted in quantitative methods is considered more strategic
insofar as it should contribute to efficient public spending and thus

greater government accountability. Social impact analysis is another
example of policy-relevant research methodology that the World Bank
affirms using in support of beneficiary countries. According to the Bank’s
explanation on the objectives and use of such PRRs, evidence-based public
policies are those which have demonstrated benefits of focusing on what
works, and therefore resulting in more efficient spending of public funds.
It goes without saying that, in this case, there is a strategic approach to
creating knowledge and steering its usage by developing countries in
the policy-making process (our emphasis).
Box 4
Policy Research Working Paper “Globalization and
innovation in emerging markets”, by Gorodnichenko,
Yuriy; Svejnar, Jan; Terrell, Katherine (2009).
Short summary
“Globalization brings opportunities and pressures for domestic
firms in emerging markets to innovate and improve their
competitive position. Using data on firms in 27 transition
economies, the authors test for the effects of globalization
through the impact of increased competition and foreign direct
investment on domestic firms’ efforts to innovate (raise their
capability) by upgrading their technology, improving the quality
of their product or service, or acquiring certification. They find
that competition has a negative effect on innovation, especially
for firms further from the efficiency frontier, and we do not find
support for an inverted U effect of competition on innovation.
The authors show that the supply chain of multinational
enterprises and international trade are important channels for
domestic firms’ innovation. They detect no evidence that firms
in a more pro-business environment are more likely to display
a positive or inverted U relationship between competition
and innovation, or that they are more sensitive to foreign
presence.”
26
There is another question to be raised in relation to PRRs, their dis-
semination and reaching out to the policy-making community worldwide.
In a report published by the World Bank itself, it is said that the way the
Bank’s analytical work is disseminated and discussed is often inadequate.
Reports tend to be long and often are not fully read even by policy makers.
In many cases they are not translated into local languages or discussed
outside a limited group of government counterparts.4
Second, the Knowledge for Change Program (KCP) serves as
a very well-funded vehicle for the pooling of intellectual and financial
resources for data collection, analysis, and research supporting poverty
reduction and sustainable development. Launched in 2002 by the World
Bank’s Development Economics (DEC) Vice Presidency and its founding
donors, Finland and the United Kingdom, KCP aims to encourage and
facilitate the Bank’s dialogue with partner agencies, developing country
managers, and other interested parties. A subsidiary objective of this pro-
gramme is to give support to data collection and analysis, and improve
research capacities in the Bank’s client countries. The KCP funds are said
to have played an important role in influencing opinion-formation on
development policies mainly through the World Development Reports
(WDRs). The WDR 2008, for instance, has been cited by Nobel laureate
Norman Borlaug, the Gates Foundation, Kofi Annan, several leading econ-
omists and, of course, support for agriculture projects is being scaled up
significantly by the Bank Group itself as well as by many other donors.
Herein resides one major influence of knowledge produced by the World
Bank: it addresses its own constituencies direct (mainly donors and clients),
and sets development agendas in several world regions, particularly in less
developed Asian, African, Latin American and Caribbean countries.5
The methodological focus of the KCP, according to the World Bank,
is to move rapidly into areas and development issues where the creation
Part one
of new knowledge is likely to assist the formulation of better poli-

cies with a greater impact on poverty. The three trust funds estab-
lished under the KCP support activities relating to overarching themes of
4 See: Meeting the Challenges of Global Development: a Long-term Strategic Exercise for the
World Bank Group, October 2007, p. 64.
5 See: Knowledge for Change Program, Annual Report (2008). 27
(i) poverty dynamics and delivery of basic services; (ii) investment climate
and trade and integration; and (iii) global public goods. As far as the first
theme is concerned (poverty reduction), for instance, one of the main
activities refers to the development of impact assessments, which tend
to be considered a fundamental means to learn about the effectiveness
of development interventions in achieving results. With approximately

11.8 million US dollars of donor contributions for the develop-
ment of this activity (since its launching), the World Bank gives support
to countries in designing evaluation methodologies, and facilitates global
learning on development interventions based on such evaluations. WB is
also initiating a series of projects which aim to assess the development
impact of some new interventions in key areas such as education (school-
based management), infrastructure (slum upgrading), health (HIV/AIDS),
and rural development (land reform). Moreover, the Bank also uses poverty
measurement as a tool for monitoring, describing and forecasting income
poverty and inequality, including aggregate poverty measures, sharper
poverty profiles, and better household surveys.6
In the field of capacity building, the Bank’s efforts have been chan-
nelled through country-level support to academic and technical institutions
via its lending programme and associated technical assistance. The Bank
offers training and courses, mainly via the World Bank Institute (WBI),
and provides research grants to networks such as the African Economic
Research Consortium (AERC), the Economic Research Forum for the Arab
countries, Iran and Turkey (ERF), and the Economic Education and Research
Consortium (EERC) – for countries belonging to the Commonwealth of
Independent States in Eastern Europe and Central Asia. These networks are
in turn now part of the Global Development Network (GDN).
These capacity-building activities are monitored by researchers inside
the WB Group. One should recall that, with a significant central research
6 The procedures and criteria used under KCP are also worth describing. Proposals should
demonstrate relevance to the objectives of the programme and details of key aspects
such as innovation, partners, country participation, deliverables, and development im-
pact. A log frame-type matrix summarizing project objectives, inputs, outputs, outcome/
impact, performance indicators, risks and critical assumptions, is required as part of each
application for funding under the KCP, to facilitate evaluation. The criteria for assessing
proposals include the degree to which proposals are innovative, provide new knowledge
and/or pilot/demonstration impact; demonstrate country participation and ownership;
incorporate developing country capacity building; can apply to a different country or
28 region; and achieve results while remaining cost effective.
department and myriad research activities throughout the organization,
the WB Group constitutes one of the largest concentrations of development
researchers in the world. The Bank’s full-time researchers are mainly found
in the Development Research Group (DECRG) within the Development
Economics Vice-Presidency. These researchers in DECRG are embedded
within a vast development agency (with a global staff of about 8,500) that
is oriented towards the implementation of development projects in devel-
oping countries. What is the role of research in the development practice
at the Bank?7 Is the profile of researchers (disciplines, schools of thought,
background) who are recruited by the WB an influential factor in research
design and scientific options?
Ravaillon (2007), a professional with more than twenty years of
research experience with the WB, affirms that the Bank has mainly devel-
oped two kinds of research. First is evaluative research, which is broader
than impact evaluation, and attempts to assess whether development poli-
cies are effective, and under what circumstances they tend to be more
effective. It embraces both “micro” interventions in specific sectors and
policies, and it includes both ex ante and ex post evaluation. Evaluative
research must be driven by questions formulated by policy makers, and
not by preferences for certain types of data or certain methods. In evalua-
tive research design, policy questions should constitute the driving issue,
although the WB tends to focus more on methodological questions relat-
ing to data collection, software design, data analysis, etc. Policy should not
fit the methodology, but vice versa.
The second type of cross-cutting research developed by the WB can
be termed methodological research, which should help expand the
tool kit routinely employed by policy makers and analysts, including the
data collected and the methods used to analyse data. The Bank has become
a major producer of development data, and WB researchers have played a
crucial role. Nevertheless, Ravaillon (2007) recalls that “not every impor-
Part one
tant piece of development research has an immediate and clear policy

implication” (p. 1). Why? The author puts forward three main features
to explain this: (i) policy makers and practitioners must still understand
the potential for research to inform policy processes, and also be ready to
7 See: Ravaillon, Martin. 2007. Research in Practice at the World Bank. World Bank, Policy
Research Working Paper. 29
pay for the costs associated with research development; (ii) policy makers
should increase their perception of benefits once research projects address
what they consider to be “relevant questions”; (iii) research projects should
also result in credible answers, which can be based on evidence, but not
exclusively.
Of course, we should also say that this dialogue between research-

ers and policy makers is highly political, and involves micro background
features of the research team, as well as (and mostly) macro institutional
development factors. Schick (2002), for instance, through the analysis of
a government-sponsored study of race relations in New Zealand schools,
shows how political and institutional pressures and a positivist-empiricist
research culture further supported a mechanistic approach to social inclu-
sion. In her article she argues that a meaningful approach to difference and
voice in inclusive research requires critical attention to the conditions of
communication and the micro-politics of the day-to-day interactions that
shape the meaning of social categories in practice.8
That is why policy research should not be limited to research projects
on technical and methodological issues. Social science research may be
rooted in a diverse set of ontologies, epistemologies and methodologies,
including action-research, critical theory, and post-positivist research. As
Cox and Sinclair (1996) assert, the world of research tends to divide the
social reality into separate spheres, creating subdivisions that are dictated
by arbitrary (and often very conservative) intellectual conventions. There is
no such thing as the theory, since all theories are based on a context, and
can relate to either ontologies of problem-solving (taking the world and its
institutions as they are, seeking for solutions to concrete society problems)
or problem-building (looking for the historical background, complexities
and deep causes of social problems). In the first case, the number of inter-
vening variables is limited; research is based on a particular problem to be
analysed, thus producing laws and regularities. In the second case, there
is no divide between research, society and politics; research is ontologi-
cally normative and epistemologically critical, since it seeks to produce
social change according to a certain set of values. Critical social research
is problem-building insofar as it does not separate facts from values, prob-
8 See: Schick, Ruth S. 2002. When the Subject is Difference: Conditions of Voice in Policy-
30 oriented Qualitative Research, Qualitative Inquiry, 8 (5): 632-651.
lematizes society and social relations, and is not oriented towards adapta-
tion and maintenance of a given status quo.9
Choucri (2007) rightly complements this reasoning affirming that
there is a direct connection between knowledge and political factors, such
as power, influence, capability, war and peace. This connection is generally
acknowledged but seldom addressed head on by academia or interna-
tional agencies. If we accept the idea that knowledge is power, it is obvious
that its application is necessary for the actual realization of power. Power
is embedded in concrete social relationships; it is not an abstract attribute
of a single and isolated political actor. That is why knowledge may become
both instrumental (i.e. leading to change) and contextual (constrained by
conditions). Parenthetically, the formulation of evidence-based policy is
precisely the use of knowledge for the pursuit of policy, and the resort to
knowledge as a legitimization mechanism.10
Thirdly and in order to conclude on the Bank’s activities in the field of
research-policy linkages, under the programme “Modeling to monitor
the MDGs” in the Development Data Platform, one can find other policy
research tools available for decision-makers. The modelling methodolo-
gies being used by the Bank to understand the challenges of achieving
the MDGs at country as well as global level use software programmes to
deliver computable general equilibrium (CGE) models (neoclassic econom-
ics). These are used to analyse macroeconomic data, along with micro
data gathered in detailed household surveys. Policy makers using these
tools are supposed to be able to analyse various policy scenarios, while
also comparing the outcomes of actual policies and programmes, with the
counterfactuals. These computerized models can explain not only what
happened as a result of a given policy, programme or project, but what
would have happened had the policy, programme or project not been
implemented. This modelling draws on the Development Data Platform.
The Bank’s Development Data Platform (DDP) is a web-based data tool
Part one
that provides access to statistics from more than 75 key databases. Users
can also access record-level data and documentation from over three thou-
sand household surveys. The DDP Micro-data incorporates both innovative
9 See: Cox, Robert W.; Sinclair, T. J. 1996. Approaches to World Order. Cambridge: Cambridge
University Press (Chapter 6: Social forces, states, and world order, pp. 60-84).
10 See: Choucri, Nazly. 2007. The Politics of Knowledge Management. Paper prepared for the
UNESCO Forum on Higher Education, Research and Knowledge. 31
IT design and development and a substantial effort to locate and format
household survey data.
As a matter of fact, this brief analysis of the World Bank’s philosophi-
cal foundations in defining the role of knowledge in policy-making shows
its normative options in the use of social science methods, particularly
relating to neoclassical economics and functionalist sociology, but

also to econometric and statistical modelling. The appeal to quanti-
tative methods and the production of empirical evidence stems from the
fundamental notion that evidence-based public policy is rooted in research
that has undergone some form of quality assurance and scrutiny. This is
the kind of social science research that is supposed to bear the monopoly
of the scientific nature and the methodological rigor.
As a matter of fact, such “scientific” developments in the field of
policy research aim to “modernize” government by making greater use
of evidence, especially evidence from the social sciences. Evidence-based
policy-making strives to use only the best available evidence to inform
policy. This evidence is rated in terms of its quality and the use of a rel-
evant research design and is mainly quantitative. O’Dwyer (1994) says that
evidence is broadly defined as research conducted systematically using
scientific principles but there are differing interpretations of the strength
and quality of findings produced by different types of research methods.
Availability and validity are key issues.
What is not recalled in policy briefs or institutional reports is that
there is some disagreement in the literature about whether or not “evi-
dence-based” policy-making is better than other forms of policy-making.
Neither do they do recall that evidence-based research policy also reflects a
conception of the kind of linkage that can be set up between social science
research and public policies. Institutional research reports do not mention,
moreover, that it is generally recognized in the literature that evidence is
not the only factor influencing policy-making.11 Knowledge (and evidence
is also knowledge) also comes with diverse degrees of uncertainty that are
seldom acknowledged by international agencies. Policy-making, as politics,
is present everywhere. Its context and conditions are often very distinctive.
11 See: O’Dwyer, Lisel. 2004. A critical review of evidence-based policy making. Australian
Housing and Urban Research Institute, Southern Research Centre, AHURI Final Report,
32 No. 58.
As Choucri (2007) says, that is why knowledge management focusing on
policy issues can seldom assume that “one size fits all”.
Case 4: The action of the Organisation

for Economic Co-operation and
Development in the field of education
In 1968, OECD’s Directorate for Education’s Section on Research and
Knowledge Management set up the Centre for Educational Research and
Innovation (CERI), whose main mandate is to promote studies on research,
innovation and knowledge management. A new research focus emerged
in 2003, building on recommendations by the CERI Governing Board,
which stressed that “evidence-based policy research is a vital com-
plement to other practical and innovative processes in teaching
and learning and should not be neglected by policy makers as
a source of innovation”. The project has centred on a series of work-
shops (see box below) which brought together researchers, experts and
policy makers to exchange experiences and practices. These workshops
have reviewed main aspects of evidence-based policy research (methods,
transaction costs, and capacities), and have also discussed what constitutes
evidence for research in education, how that evidence can best be used,
and how to identify best practices in the field. As a result of these work-
shops, in June 2007 CERI released a book entitled “Evidence in Education:
Linking Research and Policy”. At its very introduction the book states the
following: it is crucial that educational policy decisions are made based on
the best evidence possible.
Part one
33
Box 5
CERI Workshops on Evidence-Based Policy-Making
in the field of Education
Workshop Where? Main questions

and focus
First held in April 2004 Washington D.C. What constitutes evidence?

Comparing advantages and
shortcomings of a range of
established social science research
methodologies.
Second held in January Stockholm How diverse are stakeholders

2005 (researchers, policy makers, practi-
tioners and the media)?
Communicating needs, priorities
and outputs.
Third held in September The Hague What are the effective mecha-
2005 nisms for mediating between
research and policy/practice?
Defining the roles and achieve-
ments of brokerage agencies
(good practices).
Fourth and final held in London How to implement evidence-

July 2006 based policy research? The focus
was on implementation, scaling
up and sustainability.
Several documents made available by CERI on the Internet reiter-

ate that evidence-based policy is defined as “the conscientious and
explicit use of current best evidence in making decisions and choosing
between policy options”. Evidence-based policy research (EbPR) is thus
defined as the research that is used to produce evidence-based policy.
This seems obvious, but it creates at the same time an important differ-
entiation, since EbPR is distinguished from “purely scientific research”
in that the former is oriented to informing action while the latter is ori-
ented towards developing theory and testing hypotheses. Both types of
research cannot be considered mutually exclusive; however, the formal
justification given in OECD’s documents for setting up such a distinc-
tion is the following: “burdens and standards of proof of causality are
34
very different and in many cases evidence-based policy is obliged to
use the best available evidence at a given moment in time”.12
Of course, one cannot separate this formal distinction from the
reasons why OECD’s Member States and CERI have decided to invest
(time, funds, expertise) in EbPR in the field of education. In the reports
and Internet material that we have consulted, we can read that OECD
shares some fundamental working principles, as follows:
(i) There is a growing concern with accountability, transpar-
ency and effectiveness of educational policies and educational
research in OECD countries.
(ii) The information readily available for policy-making is often
unsuitable, either because the rigorous research required for
policy needs has not been conducted, or because the research
that is available is contradictory and does not suggest a single
course of action.
(iii) OECD countries share a belief that education and knowledge
are increasingly important factors in innovation and economic
growth.
(iv) There is a need to set up a clearer definition of educational
research, and more consistent support for it.
(v) It is necessary to shift from a linear to an interactive model
of research utilization, for instance, by means of employing
teachers as researchers and identifying the knowledge needs
of stakeholders.
On the one hand, what such principles reaffirm is evidently relevant
from the viewpoint of micro relations between the world of educational
research: inter alia, the way research is conducted, who it involves in
its methodological development and scientific process, the scientific
opening to demands stemming from policy makers, policy epistemic
communities, and non-formal groupings. At the same time, no one
Part one
would currently dare say that policy makers need not be accountable
12 See: Burns, Tracey and Schuller, Tom. Evidence-based Policy in Education: new opportunities
and challenges. Paris, Centre for Educational Research and Development, Organisation
for Economic Co-operation and Development, draft version, 2006. See also: Evidence-
Based Policy Research: the next step. Report drafted by Olaf McDaniel, Lotte van der
Linden, and CBE Consultants BV (Amsterdam). Session organized by the Dutch Ministry
of Education, Culture and Science for the OCED Conference on Evidence-Based Policy
Research. The Hague, September 2005. 35
for their actions and decisions vis-à-vis civil society organizations and
citizens. On the other hand, what such tenets seem to ignore is the con-
textual reality within which we need to situate the dialectical, contra-
dictory and political relationships between social science research and
policy- making. These principles do not integrate questions of political
autonomy (of individuals, associations, NGOs), funding, institutional

development, training capacities and public-private tensions that are
fundamental variables in today’s analysis of science-policy relationships
under a mode of regulation and a discourse that tends to privilege an
ethics of the market over an ethics of the public good.
It is true that experts participating in the series of workshops
organized by CERI acknowledge that EbPR is a complex and potentially
contentious issue which generates serious political debate. They recog-
nize that it is unlikely for there to be a set of conclusions or recommen-
dations which sit comfortably with every OECD policy regime. Among
the most important reasons for the lack of success of EbPR in education,
OECD’s experts have pointed out the following: (i) short-term politics:
there is no time to include the present EBPR results in the practice of
the political decision process; (ii) there are no structural links between
research and schools; (iii) misinterpretation of research by media and
politicians; (iv) lack of interaction between research, policy and prac-
tice; (v) research results often do not fit into policy’s agendas or inter-
est; (vi) absence of suitable mechanisms or incentives at school level to
feed evidence into classroom practice; (vii) teachers have to respond
to immediate classroom needs and cannot wait for research results;
(viii) present EbPR does not provide sufficient classroom tools to play
an important role; (ix) researchers and practitioners do not speak the
same language and operate in isolation from each other; (x) nega-
tive evaluation results can be interpreted (by public and politicians) as
proof of bad policy; (xi) politicians often have their specific ideas and
are frustrated when evidence tells them that they are wrong.
These explanatory factors apply to the reality of OECD countries in
the field of educational policies, and demonstrate that there can be no
single best method or type of evidence-based policy research. National
contexts are variable, and the key is for research and policy communi-
36 ties to deploy appropriate and contextual combinations of approaches

and methodologies which match the characteristics of the policy issues
under consideration. They must also have the capacity to select, imple-
ment and evaluate these combinations. Governance (including types
of political arenas, backgrounds of political culture, the meaning and
importance of authority, and institutional mechanisms of policy dia-
logue) between these two epistemic communities (the science com-
munity on one side, and policy community, on the other) is central in
the definition of future scientific and political cooperation schemes.
How does OECD attempt to foster these dialogues? Of course,
through some traditional mechanisms embedded in international
cooperation (policy briefs, workshops, publications, which we have
already mentioned in the analysis of other multilateral organizations),
but also with the support of brokerage agencies, such as the Evidence
for Policy and Practice Information and Co-ordinating Centre (EPPI-
Centre) within the University of London,13 the Knowledge Chamber
of the Netherlands,14 the Danish Knowledge Clearinghouse, What
Works Clearinghouse (USA), the Canadian Council on Learning, and
the Social Care Institute of Excellence (United Kingdom). Many of
the agencies have been created as a result of the OECD/CERI work-
shop series (box 5). These brokerage agencies are also distinguished
in their goals and means, with New Zealand’s Best Evidence Synthesis
Programme providing an example of a brokerage programme within
the Ministry, whereas the Canadian Council on Learning, although also
federally funded, is separate from the provincial Ministries. The United
States’ What Works Clearinghouse functions in collaboration with a
number of other institutes and subcontractors, and also conducts con-
sumer surveys and questionnaires to ensure that the service it provides
13 Formed in 1993, EPPI-Centre works on health and education policy-relevant research.

Part one
For details, see: http://eppi.ioe.ac.uk. It is interesting to read EPPI-Centre’s glossary,

where we can find the following definition of evidence: “research evidence is knowl-
edge and understanding developed by empirical and conceptual research. There are
many types of research, all with their own methodology for creating and evaluating
evidence”.
14 Established in June 2006, the Knowledge Chamber of the Ministry of Education, Culture
and Science is a meeting point for the top of the ministry and knowledge institutes, such
as advisory councils and planning offices. It formulates subjects on which more knowl-
edge is needed. Knowledge questions here feed back into the process of evidence-based
policies. 37
is meeting the demands of the users (researchers, practitioners, policy
makers, etc).
With the support of relatively well-funded external agencies, CERI
has, for instance, been able to establish a series of criteria for what it
considers to be sound, rigorous and relevant EbPR. It is worth noting
that all these criteria deal with methodological issues only, as follows:
• Causality claim: to what extent does the research method
ascertain whether a causal effect happens or not?
• Explanation claim: to what extent does the research method
explain how or why the causal effect is happening?
• Transportability: how far is the evidence such that the results
can be applied to most (or all) of the relevant field in different
settings of time or space?
• Stability: will the evidence be reasonably stable over time in its
application?
• Validity: does the research use instruments which measure
what it is intended to measure?
• Variability: to what extent does the research method involve or
permit variation in the type of intervention?
We believe that such questions deserve further analysis by the
MOST Secretariat if and when it disseminates its policy briefs and
policy papers, since the answers eventually produced in each case may
be thought-provoking in the understanding of the research-policy link-
ages. CERI has set up these (methodological) criteria focusing on a
very particular field of action (educational policies), based on social
realities of OECD countries and available macro educational indicators
that allow for such a generalization. What is the reality of UNESCO
and MOST? What are the possibilities for MOST to focus both the-
matically and regionally? Moreover, experts involved in EbPR tend to
share a certain number of philosophical principles (the role of social
science research, the distinction between theoretical research and
EbPR, the acceptance of current governance structures that lead inter-
national relations, etc.) and training backgrounds (levels of expertise,
experience and practice in the field of EbPR), thus constituting a quite
homogeneous epistemic community. Again, who are the stakeholders
38 of MOST? What is their intellectual involvement and experience with

issues relating to the policy-research nexus? How do they formulate
their policy demands? Is there a degree of coherence and homogeneity
in the policy demands sent to the Secretariat of MOST? We intend to
come back to some of these issues in the final part of this report.15
15 In the analysis of international organizations, it is important to note that UNDP does

not have a particular programme on evidence-based public policies; however, it de-
velops many tools in which it presents good practices for decision-makers, guides for
action in the field of democratic governance, as well as development indicators
and methodologies for building indicators. One example is the “Guide to Civil Society
Organizations Working on Democratic Governance”, published by the Oslo Governance
Centre: this guide offers a snapshot of more than 300 civil society organizations working
Part one
on democratic governance at global and regional level, and it provides information on

CSOs across UNDP’s priority democratic governance areas. In the field of indicators, one
can recall the “Governance Indicators: A User’s Guide”, which is a joint collaboration
between the Oslo Centre and the European Commission, providing direction on how to
use and where to find sources of governance indicators. Intended for the non-specialist
user, this guide brings together information on how to use indicators and where to find
material on those sources in one easy-to-use package. Finally, one can also refer to the
“Governance and Conflict Prevention database”, which allows UNDP practitioners to
access information electronically on best practices and lessons learned in UNDP govern-
ance and conflict prevention activities for policy formulation efforts. 39
Part two
Sketching a series of critical questions

on the relationships between social
sciences and policy-making
In part one of this report, the aim was not a thorough evaluation of sci-
ence-policy linkage programmes being implemented by multilateral and
bilateral organizations; rather, our aim has been to introduce briefly some
practices and understand (both contextually and ontologically) the main
tools that are highlighted in their communication strategy as a means
to foster the dialogue between social science research and public policy
formulation. In sum, the first part of this report has shown a profound
sense of continuity and regularity in the way programmes and activities
are thought of and implemented by the four selected international organi-
zations. To a larger extent, irrespective of their diverse foundational phi-
losophies these organizations share the need to invest in evidence-based
policy research, and promote policy dialogues as an instrument to build
a broader consensus on the role of empirical evidence in policy research.
Regardless of their cultural differences and specific political mandates, they
all disseminate policy briefs, build databases and promote policy forums.
We can, of course, also pinpoint some particularities in their action as
individual organizations: for instance, UNU promotes short-term demand-
driven pilot projects and builds situational diagnoses on several issue areas,
the EU focus on research-industry relationships, the WB steers training in
creating and monitoring development databases, whereas OECD has a
particular (but not exclusive) interest in the field of educational policies.
In part two of this report, we intend to raise some critical issues on
the why such a broad consensus has been possible, and sketch a series of
41
questions that could hopefully support thinking and decision in the further
strategic development of the MOST Programme. We base our reasoning in
a series of arguments that we have developed in previous publications.1
To begin with, as Richard E. Lee et al (2005) affirm, it should be
recalled that social science has, since its institutional development in the
nineteenth century, had an “ambiguous relationship” with social policy.

Using the metaphor of a “tumultuous marriage in which the rules of con-
jugality were never fully established or agreed to by both parties”, the
authors recall that the linkages between social science and policies have
also gone through the myth that the accumulation of data (usually statisti-
cal data) would illuminate the directions in which the State might proceed,
by means of various new policies and reforms, in order to alleviate the
ills that the associations of information could perceive. This was also the
influence of a positivist and functionalist problem-solving spirit adopted
by natural science, which was transmitted to social sciences in their histori-
cal development. During the 1950s and 60s, this thinking began to get
institutionalized essentially in the more industrialized States (the United
Kingdom, the United States, later in Australia and Germany). This institu-
tionalization has also reached multilateral organizations, including through
the implementation of some programmes described in part one of this
report.
The idea behind the empiricist creed was that the promotion of social
policy was not politically neutral, thus it was more appropriate for social
scientists to play a role that would be “value-neutral” and “professional”.
This gives rise to the waves of programmes on “applied social science”, as
opposed to merely theorizing about social relations or merely undertak-
ing empirical research.2 To sum up, there were two contraditory positions:
Max Weber and Robert Merton, tenors of science as truth and the principle
of axiologic neutrality on the one hand, opposed to critical thinking and
1 For further details, see: Milani, Carlos R. S. 2005. Les relations entre les sciences et la
décision politique : le chercheur, les institutions politiques, les décideurs et la gouvernance.
In: Solinís, Germán. Construire des gouvernances: entre citoyens, décideurs et scienti-
fiques. Bruxelles: Peter Lang, 288 p.
2 See: Lee, Richard E.; William J. Martin; Heinz R. Sontag; Peter J. Taylor; Immanuel Waller-
stein; Michel Wieviorka. 2005. Social Science and Social Policy: from national dilemmas to
global opportunities. Reference Paper for the International Forum on the Social Science
– Policy Nexus, Buenos Aires (Argentina) and Montevideo (Uruguay), UNESCO, MOST 2,
42 30 p.
problem-building theories defended by thinkers such as Antonio Gramsci
and Karl Manheim, on the other. Nevertheless, even when research indi-
cates scenarios for practical action, it takes more than knowledge and
social science research to make policy. One should recall variables such as
social creativity and individual imagination to build workable proposals,
but also social mobilization and political support or contestation (i.e. the
politics of science-policy) to transform these proposals into policy (or avoid
their becoming policies).
The relationships between social science and policy-making can be
thought of on at least four levels of analysis: the researcher, his/her meth-
odological and ontological choices; the historical processes of institution-
alization of social sciences (nationally and, in more recent times, interna-
tionally); the motivations of decision-makers; and, finally, the structures
of political governance that organize the contexts where these relation-
ships and key actors evolve (State, international organizations, broker-
age agencies, social movements, NGOs, corporations). These four levels
are particularly relevant for a more acute analysis when one notes that
with globalization, decision-making tends to move beyond symbolic and
material frontiers of the State (from public to private, from national to
global). Questions such as “what is the utility of social science research?”,
“who funds research and why?”, “who are the decision-makers in policy-
making?”, “who defines the priorities?” or “how legitimate are influential
international agencies in setting up national priorities?” become central for
policy debates in democratic societies both in industrialized and develop-
ing countries.3
This is because globalization is not merely a competition for market
shares and well-timed economic growth initiatives; neither is it just a
matter of trade opportunities and liberalization. Globalization has also
evolved into a social and political struggle for imposing cultural values and
Part two
3 Speaking at UNESCO in 2006, Professor A. H. Zakri (Head of UNU Institute of Advanced

Studies) appealed for international help to foster relevant research programmes in the
developing world, where “the pressures are greatest, the need most acute and it is re-
ally a matter of life and death”. Nevertheless, instead of pinpointing the inequalities in
North-South relations in the field of university and scientific development, Professor Zakri
stated that many universities in developing countries are not relevant, affirming that a
universal characteristic of university success is “relevance” or “research utility”. “Univer-
sities and the research they undertake need to be relevant – to their government’s policy,
to their people’s educational needs and to their community’s needs” (quote). 43
individual preferences: the current global economic system optimizes the
values and criteria of performance, efficiency and productivity. Nowadays,
performance defines the new locus for the belonging of global subjects
that ought to thrive on the accomplishment of short-term responsibilities
at any cost. Being efficient and cultivating performance has become the
new global avatar for the myth of progress and development; global per-
formance provides a new sense of universality for national communities.
It goes without saying that such an over-estimation of economic perform-
ance, which in general one finds in the discourse of many global economic
players, has direct implications for democratic life in general, and for the
analysis of science-policy nexus in particular. According to these global
market ethics, political negotiations must also follow the pattern of effi-
ciency and, thus, fall into the market’s timetable; there should be no room
for doubt and long deliberation in a global risk society.4
As O’Dwyer (2004) asserts, political decisions about social policies
are rarely the direct outcome of social science research. They are more
usually the result of conflicting pressures by social actors – entrepreneurs,
workers’ organizations, religious authorities, special interest groups, and
the media. The author shows that there has recently been increasing inter-
est in Australia in evidence-based policy-making (EbPM), mainly stemming
from new policy developments in the UK. Multilateral organizations play
a major role in disseminating views and methods on how to think and act
in the field of social science-policy nexus, as we have described in the first
part of this report. Based on the Australian reality, the author mentions the
international success of an evidence-based approach to health, education,
criminology and social work, which has stimulated this change. While the
concept of empirical evidence should help to promote more and better use
of research findings and a more systematic use of knowledge, it is difficult
to produce the necessary kinds of evidence to inform other policy sectors
(such as housing and urban policy) in such a way as to label these policies
“evidence-based”. This is principally because of the difficulty in isolating
4 See: Milani, Carlos R. S. 2006. Globalização e contestação política na ordem mundial

contemporânea. In: Cadernos do CRH (UFBA), v. 19, p. 377-383. See also: Milani, Carlos
R. S. 2008. Bridging divides between environmental governance and transnational con-
testatory movements: culture of politics and institutional perspectives in the sustainable
development agenda. In: International Studies Association (ISA) 49th Annual Convention,
44 2008, S. Francisco (California), Archives of the 49th ISA Annual Convention.
the effects of interventions in housing and urban issues from wider social
processes and their geographic variations.
This distinguishes EbPM from public policy based on more conven-
tional policy development processes where intuitive appeal, tradition,
politics, or the extension of existing practice may set the policy agenda.
EbPM is not synonymous with good policy- making, but evidence-based
policy-making is more likely to be good policy-making in some particular
fields. Here, again, excessive generalization (both thematically and across
countries or cultural contexts) may be a perilous temptation. It is possible
to have bad evidence-based policy-making if the evidence used is biased,
flawed or incomplete. One could also say that, depending on the purposes
of data collection, evidence may serve unfair and unjust policy objectives.5
Statistical data as well as cartography are “texts”, and may be skilfully con-
trolled and technically manipulated.
Some factors influencing the use of an evidence-based approach in
policy-making that should also be taken into account are: prevailing public
opinion, organizational culture, incompatible time frames in policy-making
and research, values and ideology of both researchers and policy makers,
control of power, political goals, as well as institutional development and
degree of autonomy of the social science community. Carol Weiss (1979)
identifies “four I’s” which characterize policy-making in general: ideology
(people’s basic values – of policy makers and wider society); interests (per-
sonal or organizational, such as personal career aspirations or maximizing
budgets); institutional norms and practices ( for example, the US congress
works largely through face to face contact – reading is not part of the norm
and so written documents of research findings are likely to be ignored);
prior information (policy makers already have information from various
sources).6
Based on these four “I’s” and because “research utilization” is associ-
ated with a variety of different meanings and interpretations, Weiss (1986)
Part two
later identified seven different models of research-policy relationships: (i)

the knowledge-driven model (basic research highlights an opportunity →
5 See: O’Dwyer, Lisel. 2004. A critical review of evidence-based policy making. Australian
Housing and Urban Research Institute, Southern Research Centre, AHURI Final Report,
No. 58.
6 See: Weiss, Carol H. 1979. «The Many Meanings of Research Utilization», Public
Administration Review, 39 (5): 426-431. 45
applied research is conducted to define and test these findings → appro-
priate technologies are formulated → application occurs); (ii) the problem-
solving model (evidence is produced in a particular field in order to solve
or shed light on a particular policy problem); (iii) the interactive model
(information for policy-making is not only produced by researchers, but
also by planners, practitioners, interest groups, journalists, etc.); (iv) the

political model (research is used as a means of legitimization of policy deci-
sions); (v) the tactical model (research is an excuse for inaction); (vi) the
enlightenment model (cumulative research and information over time sen-
sitizes policy makers to new issues); (vii) and research as part of the intel-
lectual enterprise of society (research is not an independent variable affect-
ing the policy process, but rather policy interests often set the parameters
of research and the scientific agendas).7
The first two models are very linear in their nature, and presuppose
relationships between the world of social sciences and policy-making com-
munities that are spontaneous and free from pretension or political cal-
culation. Both share positivist principles in their epistemology and meth-
odology. They seem to inform many policy-research projects and EbPR
programmes analysed in part one of this report. The third model is rooted
in a phenomenological understanding of social reality, where human
interactions tend to erase conflictual situations, confrontation and abso-
lute opposition. The other four models take the politics of science-policy
relationships into account, and suppose different functions and uses of
research by policy makers.
An example of how to understand these models could come from
a UNU seminar organized in 2007 on the topic “Strengthening Linkages
between Science and Policy”. This seminar was held as part of the 21st
Pacific Science Congress in Okinawa, Japan, in June 2007. The two main
questions addressed during this seminar were:
• Why, despite recent advances in biotechnology, nanotechnology,
medicine and environmental sciences, do more than 1 billion
people around the world continue to live in extreme poverty
without access to proper nutrition, safe drinking water and basic
medical services, and survive on less than 2 USD a day?
7 See: Weiss, Carol H. 1986.The many meanings of research utilisation, in M. Bulmer (ed.),
46 Social Science and Policy. London: Allen Unwin.
• Why, despite unequivocal evidence that global warming will con-
tinue to cause dramatic changes in wind patterns, precipitation
and extreme weather that will negatively affect human popula-
tions, are policies mitigating these effects being debated?
The main conclusion of this 2007 seminar was that tenuous links
between science and policy can be seen as one of the primary reasons
why better technologies are not accessible to the poorest people on Earth.
The seminar report presents conclusions that may induce an uninformed
reader (an uninformed policy maker?) to think that simply through greater
investments in science policy the benefits of technological innovations
and economic growth would trickle down to all individual members of
a society, regardless of established patterns of domination, exclusion and
social reproduction of inequalities. It would be naïve to think of major
global ecological crises, world food security and climate change only as
a problem of access to technology and insufficient investment in sound
research-based policies. Building such a simplistic answer to the questions
raised during the seminar may also show how the “narrative of linkage”
can be used as an excuse not to tackle power relations that are inherent in
society, both nationally and internationally. It would also demonstrate how
difficult it may be to set aside the first three models introduced by Carol
Weiss. It seems there is no doubt that a paradigm shift is needed in the
way science-based advice (and social science-based advice in particular) is
translated into successful policy if we are to achieve environmental sustain-
ability without compromising social justice worldwide.
Based on Burrel and Morgan (1979), let us build a figure (see figure 1)
where we can find two analytical axes for understanding major premises
in building models on social-science and policy linkages. The horizontal
axis deals with objectivity and subjectivity, and refers to sources used in
social science research in order to construct an ideal of social reality. The
objectivist approach to social science is rooted in an ontology of realism, a
Part two
positivist epistemology, a deterministic conception of human nature, and

a nomothetic methodology. The subjective approach to social science is
based on an ontology of nominalism, an anti-positivist epistemology, vol-
untarism, and an ideographic methodology. The vertical axis presents the
normative and ontological perspective of social sciences before the idea
(or need) of social change: upwards one can situate perspectives of radical
47
change (a sociology of deep social transformations), whereas downwards
8
we find a sociology of regulation and adaptation within the status quo.
This second axis corresponds to the role that the researcher may resume in
defence of transformative social sciences, including through their relation-
ships with policy networks and non-scholar communities. In this sense,
the presentation of intellectual ideas through social science research may

threaten a certain notion of truth which has claimed a false appearance of
universality.
Figure 1: Paradigms and Models in the Analysis

of Social Science and Policy Linkages
Sociology of radical change

Sociology of radical change
Radical humanism Radical structuralism

models
Action research
models (popular
education, urban
action research)
Critical theory
models
Conflict theory
Subjective
Objective
Subjective
Objective
models
Social system
Phenomenological analysis and Weiss’s knowledge-
and interactive cybernetic driven model and
models models problem-solving
programmes
Knowledge-
Interpretative management and Functionalist
models EbPR approaches
Sociology of adaptation
Sociology of adaptation
This figure shows the legitimacy of alternative perspectives. It

entails a culture of debate, argumentation and dialogue in the analysis
of other models for understanding linkages between social science and
policy-making. Hopefully, it will also provide the means for an interna-
tional organization such as UNESCO to embrace other frames of reference
8 Burrel, Gibson; Morgan, G. Sociological Paradigms and Organisational Analysis: elements

48 of the sociology of corporate life. London: Heinemann, 1979.
and not worry too much about orthodoxy. Of course, the legitimacy of
non-functionalist paradigms is not assured today, and should not be taken
for granted. However, unveiling the ontology and epistemology of EbPR
models adopted by the large majority of international organizations today
may also allow for a more pluralistic debate on the issue of social sciences
and public policy linkages.
How can this figure support some strategic thinking for MOST in order
to craft an original analytical framework? What critical possibilities does it
offer in the understanding of the role of EbPR promoted by many multi-
lateral organizations? These are difficult but necessary questions. As Lather
(2008) affirms, the time has come to break out of the regulation, standard-
ization and surveillance of research and the speculation on what it would
mean to be a “mature” or “objective” science, and move beyond, towards
a more complex scientificity, where the empirical becomes more interpre-
tive, not less.9 There is therefore a need to refuse to concede science to
scientism, and reopen the debate on when, why and how research matters
in policy-making. Statements on how “rigor” in research is the most direct
route to better policies must also be discussed based on the actual policy-
making process, and the options made by models presented in figure 1.
There is a clear need to shift the discourse away from the focus on
“objective” and technical questions of research design and methods in
order to move towards the analysis of dialectical relationships between
research and policy. Questions on method and design are not irrelevant,
but they seem to have erased the key role of political behaviour of political
actors in current international programmes dealing with EbPR. As Lather
(2008) shows, the interest in research that informs policy and practice is
shared across disciplinary boundaries and methodologies and among the
“interdisciplines” that have long characterized the field of education. The
“alternative” models presented in figure 1 may also raise pertinent (and
different) questions to the understanding of the linkage problem. One of
Part two
the issues is how the theory/practice or basic/applied distinctions are to be

thought. Taking the complexity into account entails a more philosophical
and less instrumental (re)thinking of the research/policy nexus.
9 See: Lather Pathi. 2008. New Wave Utilization Research: (re)imagining the research/
policy nexus, Educational Researcher, 37 (6): 361-364. 49
Thus, it is useful to note that research for policy is not so much about
providing answers as about changing the way questions are understood, so
that people (researchers and policy makers, but other publics too) can begin
to think differently, thus critically building the contours and contents of
social problems. Working towards a more complex scientificity entails a sort
of “philosophy of negativity” (Lather, 2008), where modes of contestation

would be as constitutive of the very scientific field within which one locates
one’s research work. Of course, this would mean shifting the standard of
intelligibility for policy research away from the positivism that underpins
hegemonic understandings of evidence, objectivity, reason, measurement,
value-free facts, research utilization and responsible knowledge production.
As Pawson and Tilley (1996) affirm, in conducting research, spon-
sored research in particular, political considerations mitigate reflexive
impulses that might undermine the authority of the research. Apart from
structural factors building the politics of science-policy relations, what
goes on “behind the scenes” at both meso- and micro levels is also a
significant aspect of knowledge production. Factors such as professional,
situational, cultural and interpersonal relationships between researchers
are rarely addressed in methodological sections of research reports or in
texts relating to methods. Researchers’ attempts to situate themselves in
relation to their work signal an awareness of the centrality of research iden-
tities to the process of knowledge production but often do not address
the ways in which research relationships and settings shape research find-
ings, analyses and reporting. Research grounded in a commitment to
social change inevitably begins with at least a provisional presumption
that social difference, inequality and spatial disparity are both organized
and knowable.10 Contrary to the positivist and rationalistic creed, political
rationales need not be discarded as criteria for some research and policy
decisions.
Another final key aspect that we would like to highlight has to do
with the research process itself. Who participates in the definition of the
policy research agenda? Whose interests are taken into account? Contrary
to the common sense that may prevail in “participatory projects”, it
10 Pawson, R.; Tilley, N. 1996. How (and how not) to design research to inform policymak-
ing, In: C. Samson and N. South (eds.), The Social Construction of Social Policy: Methodolo-
50 gies, Racism, Citizenship and the Environment. London: MacMillan, pp. 34-52.
would be naive to think that a history of exclusion can be overcome by
“including” individuals already identified and selected because they are
disempowered by those very structures. Calling “them” empowered is
not enough. If we are concerned about issues of voice and exclusion in
the production of knowledge, then it is critical to recognize the situated
character of the research process itself.11 As Schick (2002) recalls, these
conditions of research are at least as influential in shaping the meanings
research produces, publicizes and legitimizes as the superficial markers of
identity foreseen in the research design. No matter who decides what cat-
egories will be used and how they will be defined in principle, in practice,
inclusiveness is produced in the micro-politics of day-to-day interaction.
Designing qualitative research practices that attend to these relationships
more self-consciously and reflexively may offer a more productive chal-
lenge to exclusion in knowledge production.12
Some questions for the future

To conclude this report, and steer debate within the MOST Secretariat,
we have organized a series of strategic suggestions and theoretical ques-
tions for the future development of MOST in promoting social sciences
and policy linkages. Based on many of the arguments presented above,
what follows should be read as a series of guiding topics for discus-
sion. An attempt to define an original approach and to establish some
definitions of how MOST conceives of linkages between social science
and policy-making could take into consideration, inter alia, the following
issues:
Research on other universalities

of the research-policy linkages
(1) Based on the assumption that empirical-evidence research policy has

Part two
turned out to be the hegemonic conception on linkages between
11 For a critical account of participatory experiences in Brazil, see: Milani, Carlos R. S. 2008.
O princípio da participação social na gestão de políticas públicas locais: uma análise de
experiências latino-americanas e européias. In: Revista Brasileira de Administração Pública,
v. 42, p. 551-579.
12 See: Schick, Ruth S. 2002. When the Subject is Difference: Conditions of Voice in Policy-
oriented Qualitative Research, Qualitative Inquiry, 8 (5): 632-651. 51
social science research and public policy formulation, two avenues of
work could pave the way for MOST in its future development:
(a) How, when and why has this hegemony been built? For instance,
MOST could foster research on the national trajectories of social
science institutional building where EERP has developed more
thoroughly (such as the United States of America, the United

Kingdom, among others). MOST could also deepen the analysis of
the roles played by international agencies, rooted in the first results
presented in this report. MOST could also analyse national strate-
gies of integration (from outside to inside) of EERP approaches, for
instance, in Latin America. How, why and when have we started
promoting and using EERP methodologies in developing countries
in general, and in Latin America in particular?
(b) Are there other problem-building approaches on R-P linkages
that could constitute a counter hegemony or other expressions
of universality? MOST could promote studies on national social
science institutionalization processes in Latin America aiming at
understanding how EERP methodologies may have contributed
to the withdrawal of endogenous approaches (action research,
popular education, etc.) from local research and policy agendas.
MOST could develop and fund national case studies trying
to analyse how R-P linkages may be conceived of in different
manners. Could MOST identify and disseminate different histori-
cal traditions and recent reform policies that may have an impact
on R-P linkages (in Argentina, Bolivia, Brazil, Mexico, Venezuela,
for instance)?
Work on what already exists within MOST:

explain the functions and objectives of tools
that are already available
(1) Are available tools different from what other agencies propose and
do? Do they reach a different audience? Do they convey particu-
larly different contents? In what sense are they relevant? Are they
52
complementary one with another? What is the specificity of the MOST
tool that is available on the Internet?13
(2) In the case of policy briefs, many international organizations suppose
that they are efficient in promoting dialogue between research and
policy, but how efficient are they? Who actually receives, reads and
uses them, and how? Who writes them? In the case of MOST, is it
possible to benefit from the expertise of its research networks and ask
project leaders themselves to prepare a policy brief based on some
of their fields of expertise? One might suppose that some researchers
would feel privileged to have a piece of their research work formatted
as a policy brief to be disseminated through UNESCO’s networks and
reach a larger audience.
(3) Based on the assumption that R-P linkages vary according to themes,
agendas, contexts and cases, could MOST interview policy makers,
researchers and other social actors on practices used in order to use
social science research in the decision-making process? Could these
interviews show improvements, contradictions and limits of these
practices? If we take the example of participatory budgeting in Brazil,
there is extensive research which has been undertaken, many dif-
ferent practices across the country, distinct social actors involved in
the political process, etc. What interesting and contradictory remarks
on these experiences could MOST analyse and discuss? What lessons
could be learned from these experiences in Brazil?
Define MOST’s main partners and beneficiaries
(1) Can MOST work efficiently with many diverse partners? Is partner-
ship diversity an asset for MOST? Or given the reality (financial,
13 It is interesting to note that the MOST Secretariat document “Policy Research Tool: Back-
ground, Design Methodology and Application” (November 2007) states that one of the
Part two
main goals of the tool would be to set up “a no-fee online policy research service which
is expected to further new modes of decision-making, based on actual evidence from re-
alities on the ground” (page 2). It also affirms that “the knowledge needed to design ef-
fective policy must thus be made available to decision-makers in forms they can use. The
currently fashionable call for evidence-based policy specifies what constitutes effective:
policy designed by reference not to common sense or to ideological preconceptions but
to prior practical experience” (page 3). We strongly believe that the MOST Programme
should steer some more analysis on how and why EERP may suppose that what it labels
as “practical experiences” are not inhabited by world visions, ideologies, theories of the
social and the political. 53
operational, human resources) of some multilateral agencies and
their current agenda, would there be a need for MOST to limit the
scope of partnerships in order to be more focused and efficient (both
in substantive matters and methodologically)?
(2) Assuming that funding mechanisms have an influence on the way
R-P linkages are built and practised, can MOST promote a critical
reflection on current public-private partnerships in the development
of social sciences, and also their impact on policy formulation and
implementation?
Develop a “pedagogy of knowledge-policy dialogue”

in the field of social development
(1) In the preparation and development of an analytical framework for

MOST, some of the research questions we have discussed in this
report should be tackled, such as:
(a) Who are the players and how do they interact? What is the quality
of communication between the different sets of agents involved
in commissioning, executing and implementing policy-relevant
research?
(b) What are the kinds of methodologies and epistemological para-
digms which dominate within policy and research communities?
(c) At what levels are policy goals set and policies fashioned and
implemented?
(d) Which forms of capacity are most in need of expansion or strength-
ening? How and by whom should this be done?
(e) What are the development opportunities for social science
researchers (refresh, retrain, updating)? What is the quality of
social science research infrastructures?
(f) What are the different roles played by mediation actors and bro-
kerage agencies in different countries? How effective are they?
(g) How do different structures of governance shape the meaning
of policy-relevant research? To what extent do different levels of
policy-making have adequate access to research?
(2) Once MOST completes a preliminary step of defining its own con-
54 ception and analytical framework, could it develop pedagogical tools

(guides, e-learning modules) working exactly on the critical contradic-
tions of the research-policy nexus that national trajectories analysed
(for instance, in Latin America) might reveal? Through a “pedagogy
of the knowledge-policy dialogue”, could MOST bring about concep-
tual clarification, as well as analyses of the limits of a cause-and-effect
linear reasoning?
Part two
55
Policy Papers
1. Searching for New Development Strategies – The Challenges of the Social

Summit, Ignacy Sachs, 1995.
2. From Social Exclusion to Social Cohesion: Towards a Policy Agenda,
Sophie Bessis, 1995.
3. Cybernetics of Global Change: Human Dimensions and Managing of
Complexity,
y M. Mesarovic, D. McGinnis and D. West, 1996.
4. Multiculturalism: New Policy Response to Diversity,
y Christine Inglis,
1996.
5. Démocratie et citoyenneté dans la ville du XXIe siècle, Céline Sachs-
Jeantet, 1997. (French only)
6. Sustainability: A Cross-Disciplinary Concept for Social Transformations,
Egon Becker, Thomas Jahn, Immanuel Stiess and Peter Wehling,
1997.
7. Nouvelles configurations villes-campagnes, Ricardo Abramovay and
Ignacy Sachs, 1999. (French only)
8. Fight Urban Poverty: A General Framework for Action, Denis Merklen,
2001.
9. Domestic Democracy and International Democracy: Deficits, David
Steele, 2002.
10. Rural labour migration in China: challenges for policies, Zhan Shaohua,
2005.
11. Intercommunality: The success story of CODENOBA, Argentina, Nicole
Maurice and Clara Braun, 2005.
12. Anti-poverty Policies and Citizenry: The “Chile Solidario” Experience,
Julieta Palma and Raúl Urzúa, 2005.
13. Illegal Drugs and Human Rights of Peasants and Indigenous
Communities: The Case of Peru, Carolina Navarrete-Frías and Francisco
E. Thoumi, 2005.
14. Illegal Drugs and Human Rights of Peasants and Indigenous Communities:
The Case of Bolivia, Carolina Navarrete-Frías and Francisco
E. Thoumi, 2005.
15. Illegal Drugs and Human Rights of Peasants and Indigenous Communities:
The Case of Colombia, Carolina Navarrete-Frías and Francisco
E. Thoumi, 2005.
16. El rol de las ONGs en la reducción de la pobreza en América Latina,
Ferrer, Marcela; Monje, Pable; Urzúa, Raúl, 2005.
17. The System for Evaluating and Monitoring Social Development Programs
and Policies: the case of the Ministry of Social Development and the
Fight against Hunger in Brazil, Vaitsman, Jeni; Rodrigues, Roberto W.S.;
Paez-Sousa, Rômulo, 2006.
Special Edition. Public Participation in Socially Sustainable Urban Development,

György Enyedi, 2004.
Special Edition. Social Science and Social Policy: From National Dilemmas to
Global Opportunities, Richard E. Lee, William J. Martin, Heinz R. Sonntag,
Peter J. Taylor, Immanuel Wallerstein and Michel Wieviorka, 2005.
58
Management of Social Transformations (MOST)
Policy is the priority

While it still promotes international, comparative and policy-relevant research
Evidence-based policy
on contemporary social transformations, MOST is now emphasizing the research
and policy interface as its major raison d’être. Tackling the sustainability of social
research: critical review
transformations is the programme’s main task, which implies action at normative,
analytical and strategic/political levels. It must concentrate on research of direct use
of some international
to policy makers and groups involved in advocacy.
MOST’s emphasis is thus on establishing and interconnecting international
programmes on
policy networks with renowned social science researchers to facilitate the use of
social science research in policy-making. This means bringing together basic research
with those entrusted with policy formulation in governments, institutions, actors and
in UNESCO itself. social science research and
Tools for policy-making policy-making
The Policy Papers, dedicated to social transformations and based on policy-relevant
research results of work carried out by MOST and by other sections of the Social
and Human Sciences Sector (SHS), are intended for policy makers, advocacy groups, Carlos R. S. Milani
business and media.
SHS is seeking new ways of distributing knowledge to target groups, such as
ministers of social development, advocacy groups, UNESCO National Commissions
and local authorities. It has launched a tool for online knowledge management
and meta-networking for decision-making and strategy. This knowledge repository
will use innovative and refined search tools to facilitate access and intelligibility of
Management
complex research data for all potential users.
of Social
Transformations
www.unesco.org/shs/most Policy Papers / 18

Milani Evidence-Based Policy Research PDF

Uploaded by

Copyright:

Available Formats

Milani Evidence-Based Policy Research PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Milani Evidence-Based Policy Research PDF

Uploaded by

Copyright:

Available Formats

Management of Social Transformations (MOST)

Policy is the priority

www.unesco.org/shs/most Policy Papers / 18

The generic structure ﬁrst enables different types of documents to be produced

The structure also gives all documents the same appearance,

MOST-2 methodology helps respond more efﬁciently to different types of

Carlos R. S. Milani ([email protected]) is Professor of International Relations at

Published in 2005 by the United Nations Educational,

SHS-2009/WS/8 – // CLD 1676.9

Part 2: Sketching a series of critical questions

Some questions for the future

It is possible to have bad evidence-based policy making if the evidence

Launched in 1994, the Management of Social Transformations – MOST

The aim of the venture (theoretical and methodological reﬂection on

to three main categories of actors: social science researchers, policy

to information available on its website, in its second phase “MOST focuses

10 when we also outline some recommendations that could support the

 Social science and policy-making: the work of

its very introduction. We do believe, however, that websites and Internet

 Case One: The United Nations University

It is interesting to analyse, as an illustration of the kind of dialogue

concentration are shown to produce both beneﬁts and adverse effects. It is

of reasons why it could fail. Questions pertaining to the

18 monographs. One question should, however, be put straight away: can

ﬁrst-hand access to the University’s extensive international networks.

needed on the actual results of public-private partnerships in the funding

 Case 2: The European Union and

2 See: Euractiv Foundation at www.euractiv.com 21

to industry, most of the 528 on-line respondents consider cultural dif-

inequality and discrimination are also investigated. This research

and public authorities work together in assessing technological

24 trates on the analysis of issues such as organizational innovation,

 Case 3: The World Bank

insofar as it should contribute to efﬁcient public spending and thus

of new knowledge is likely to assist the formulation of better poli-

of development interventions in achieving results. With approximately

tant piece of development research has an immediate and clear policy

Of course, we should also say that this dialogue between research-

relating to neoclassical economics and functionalist sociology, but

 Case 4: The action of the Organisation

Workshop Where? Main questions

First held in April 2004 Washington D.C. What constitutes evidence?

Second held in January Stockholm How diverse are stakeholders

Fourth and ﬁnal held in London How to implement evidence-

Several documents made available by CERI on the Internet reiter-

autonomy (of individuals, associations, NGOs), funding, institutional

36 ties to deploy appropriate and contextual combinations of approaches

13 Formed in 1993, EPPI-Centre works on health and education policy-relevant research.

For details, see: http://eppi.ioe.ac.uk. It is interesting to read EPPI-Centre’s glossary,

38 of MOST? What is their intellectual involvement and experience with

15 In the analysis of international organizations, it is important to note that UNDP does

on democratic governance at global and regional level, and it provides information on

 Sketching a series of critical questions

nineteenth century, had an “ambiguous relationship” with social policy.

3 Speaking at UNESCO in 2006, Professor A. H. Zakri (Head of UNU Institute of Advanced

4 See: Milani, Carlos R. S. 2006. Globalização e contestação política na ordem mundial

later identiﬁed seven different models of research-policy relationships: (i)

also by planners, practitioners, interest groups, journalists, etc.); (iv) the

positivist epistemology, a deterministic conception of human nature, and

Social science and policy-making: the work of

Case One: The United Nations University

Case 2: The European Union and

Case 3: The World Bank

Case 4: The action of the Organisation

Sketching a series of critical questions