Public Management Review

ISSN: 1471-9037 (Print) 1471-9045 (Online) Journal homepage: http://www.tandfonline.com/loi/rpxm20

A Framework for Assessing the Performance of

Policy Networks

Joris Voets , Wouter Van Dooren & Filip De Rynck

To cite this article: Joris Voets , Wouter Van Dooren & Filip De Rynck (2008) A Framework for
Assessing the Performance of Policy Networks, Public Management Review, 10:6, 773-790, DOI:
10.1080/14719030802423129

To link to this article: https://doi.org/10.1080/14719030802423129

Published online: 15 Dec 2008.

Submit your article to this journal

Article views: 476

View related articles

Citing articles: 29 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=rpxm20
Abstract
A FRAMEWORK FOR
This article deals with the question of how to
assess policy network performance. We ASSESSING THE
propose three dimensions of policy network
performance (production, process, and re- PERFORMANCE OF
gime) that are derived from both performance
and network literature. Each of these dimen-
sions can be linked to the costs of policy
POLICY NETWORKS
networks. The framework goes beyond the
one-dimensional new public management Joris Voets,
conceptualization of performance that Wouter Van Dooren and
focuses on efficiency and effectiveness. A
policy network assessment needs to take into Filip De Rynck
account other public principles such as
democratic quality and capacity building. Joris Voets
Public Management Institute
Katholieke Universiteit Leuven
Leuven, Belgium
E-mail: [email protected]
Key words
network performance, performance assess- Wouter Van Dooren
ment, policy network Department of Political Sciences
University of Antwerp
Antwerp, Belgium
and
Public Management Institute
Katholieke Universiteit Leuven
Leuven, Belgium
E-mail: [email protected]

Filip De Rynck
Department of Business Administration and
Public Administration
University College Ghent
Ghent, Belgium
E-mail: [email protected]

Vol. 10 Issue 6 2008 773–790

Public Management Review ISSN 1471-9037 print/ISSN 1471-9045 online
Ó 2008 Taylor & Francis
http://www.tandf.co.uk/journals
DOI: 10.1080/14719030802423129
774 Public Management Review

INTRODUCTION

It is a well-founded assumption that contemporary government is increasingly

networked. This phenomenon is characterized as the era of collaboration by Agranoff
and McGuire (2003) and the network society by Castells (1996). In particular, policy
networks (of which some are collaborative networks) have become more important in
public governance to deal with wicked issues. As a result, we need to comprehend the
performance of such networks (Agranoff and McGuire 2001a). In order to develop this
understanding, we need to leave the beaten track of the traditional performance
literature. This research tradition has conceptualized performance mainly at the macro
level (performance of a country) or at the micro level (performance of an organization
or individual). Policy networks however fall between macro and micro and therefore
traditional approaches do not simply work. Thinking about policy network performance
challenges our conventional understanding of performance (see also Gray 1989;
O’Toole 1997; Mandell 1999; Agranoff 2005; Provan and Kenis 2008). Public
administration increasingly takes place in settings of networked actors who necessarily
rely on each other and cannot compel compliance on the part of the rest. Yet
the standard writings to which most administrators turn for advice to improve
performance devote relatively little attention to acting effectively in such situations
(O’Toole 1997).
In this article, the leading research question is how to assess policy network
performance. In the first section, we start with the conventional definition of
performance, which relies heavily on the new public management (NPM) discourse
(Hood 1991). We discuss two limitations of this NPM definition of performance,
namely its narrow focus on efficiency and effectiveness, and its preoccupation with
organizational (micro) performance. We conclude the first section with some
suggestions to overcome these limitations. In the second section, the notions of
performance and policy networks are joined together. This is the core of the argument,
resulting in an assessment framework for network performance. The third section is a
discussion of how to put the framework to work as well as of the limitations of the
approach to policy network performance developed in this article.

BUILDING BLOCKS

Performance the NPM way

Performance is about intentional behaviour, what is done or what is believed to be

done (Dubnick 2005). The now dominant definition of performance is inspired by
new public management thinking. In response to fiscal and other crises, governments
pledged to do more with fewer resources – a government that works better and costs
less (Gore 1993). Government reformed in the name of performance. In particular in
 Voets et al.: A framework for assessing the performance of policy networks 775

the UK, this led to cutback management and a reduction of the size of government
(Dunleavy 1986). Other countries followed other trajectories (Pollitt and Bouckaert
2004).
The NPM interpretation of performance is focused on effectiveness and efficiency.
An organization is effective when it attains predetermined goals. This should be done in
an efficient way, which means maximizing outputs while minimizing inputs. Performing
organizations thus keep it lean and purposeful; they match resources to defined tasks.
Frugality of resource use in relation to given goals is the criterion of success, while
failure is counted in terms of instances of avoidable waste and incompetence (Hood
1991: 12). Performance in this view follows a production logic where inputs are
transformed to outputs and outcomes (Van Dooren 2006). In the remainder of the text,
we will refer to this interpretation of performance as production performance. Production
performance constitutes the first dimension of the framework to assess policy network
performance presented in this article.
The NPM interpretation of performance is one dimensional and restricted to the
micro level. In order to discuss performance of policy networks (but probably also to
improve discussions about the performance of single organizations, Van Dooren 2008),
we need to extend the performance concept and make it multidimensional and
multilevel.

From a narrow focus on efficiency and effectiveness, to a multidimensional

performance concept

Although production is an important aspect of public performance, other dimensions

exist. Unlike the profit principle in the private sector, the public sector has no
bottom line unless a vague one in terms of serving the public interest (Moore 1995).
Private value is expressed in monetary terms and determined through the invisible
hand of the market. In contrast, public value can seldom be expressed in monetary
terms only, and is determined through a complex and imperfect system of political
representation and bureaucracy. As a result, there is more to public performance than
setting targets and objectives, producing products and services, and evaluating their
achievement. Based on a seminal article by Hood (1991), two dimensions can be
added to the production logic, namely process performance (focused on values like
fairness, honesty, and mutuality) and regime performance (focused on values like
robustness and resilience).
An important second dimension of public performance is focused on values like
fairness, honesty, and mutuality. Government has to pursue honesty, fairness, and
mutuality through the prevention of distortion, inequity, bias, and abuse of office
(Hood 1991: 13). Government has to act in a democratic way, and democratic is
broadly defined as ‘with respect for the power of the people’ – as in the Greek
root word: demos (people) and kratos (rule or strength) (Graubard 2003).
776 Public Management Review

The concept of democracy thus covers more than the traditional institutions of
democracy. These values are institutionalized in appeal mechanisms, public reporting
requirements and ethical codes. Success and failure are measured by levels of trust
and entitlement (i.e. consent, legitimacy). This performance dimension is about the
processes of governance. This view strongly opposes the claim of production
performance that only results matter. We will label this dimension as process
performance.
A third dimension of performance is linked to robustness and resilience of the
regime. Government has to keep operating even in adverse ‘worst case’ conditions, and
should be able to adapt rapidly in a crisis (Hood 1991: 14). Reliability is often an
argument for choosing public production instead of private production. In times of
stress, governments cannot go broke and start over with another business plan. On the
contrary, government has to perform when other actors do not perform. Robustness
and resilience are not only about the capacity to resist a crisis, but also about the
capacity to deal with future challenges. For instance, will governments be able to deal
with ageing populations or unrelenting globalization? Success and failure in this
dimension are counted as survival and security. We label this performance dimension as
regime performance.

From a narrow focus on the micro level, to a multi-level performance concept

Micro and macro levels of analysis have different meanings in different disciplines
(Schelling 1978; Hedstrom and Swedberg 1998). Here, we use a crude distinction
where micro is the organizational level and macro is the jurisdiction-wide level. The
performance literature has traditionally focused on either macro or micro performance
(Van Dooren 2007). Macro performance is expressed in statistical series, often under
the banner of quality of life and social indicators. Micro performance, as we argued above,
has been a central discussion in the managerial reforms of the 1980s and 1990s and is
found in scorecards and management tools. Although in theory micro performance should
add up to macro performance, the two perspectives are seemingly decoupled. The
separation is not so much conceptual, but institutional. Macro performance is mainly
studied by highly specialized professionals in statistical bureaus and think tanks, while
micro performance is assessed by well-informed practitioners in a rough and ready
mode.
The question then is how to approach the intermediary network level (in terms of
not referring to a state system or a single organization within a state system). Two
seemingly evident strategies are to start from the macro or the micro perspective
respectively. To start from the macro level, we would have to identify some cross-
cutting, jurisdiction-wide issues (for instance poverty), then single out that segment of
the issue that is relevant for the network (for instance poverty in a particular region),
and finally attribute network activities to the issue (what is the impact of the network
 Voets et al.: A framework for assessing the performance of policy networks 777

activities on poverty reduction). Each step seems to be a daunting task, and such a
strategy might be criticized for being too bureaucratic and technocratic, or simply
unrealistic.
A more realistic and useful approach is to start from the micro level. The networks
we study are policy networks. Such networks commonly refer to set(s) of horizontal
relationships with a certain level of stability/endurance and of structuring, between
multiple actors that are relatively autonomous and faced with resource dependency,
involving processes of resource exchange through bargaining and negotiation, to achieve
public purpose (see Voets 2008 for an overview). As such, policy networks do not refer
to state systems, but to multi-organizational structures in which certain aspects of
public policy are dealt with. We contend that the assessment of policy network
performance is more closely related to micro performance assessments of organizations
than to the statistical series of the macro studies. Therefore, we believe that the main
step is to broaden the narrow micro view of organizational performance to include
performance of policy networks.

Multiple levels of assessment

While it is already challenging to evaluate performance of single organizations, policy

network performance is even more complex. According to Provan and Milward (2001:
416), ‘most problematic is the fact that multiple organizations require dealing with
multiple sets of constituencies’. The question ‘effectiveness for whom’ is very
prominent in policy network performance (Provan and Kenis 2008). In a policy
network, the private actors for instance might applaud the results of the network
because it takes over their agenda, while it might have negative effects for other
stakeholders or governmental actors.
For this reason, a policy network performance framework should not only include
different dimensions of performance, but also multiple levels of assessment. Provan and
Milward (2001) suggest that network performance can be assessed at three levels: the
level of the participants/organization; the level of the network; and the level of the
community.
The first level of assessment fits the micro level, in the sense that it focuses on the
performance of a policy network in relation to individual organizations. Policy network
performance is assessed from a single actor perspective, which assesses the value of the
network in relation to its organizational aims. Autonomous actors participate in a
network for their own purposes, based on an assessment of perceived costs and
benefits. The network performs as long as they feel network participation can serve
their interests (although participants can also be driven by external motivation, like
legislation (as part of the shadow of hierarchy) or the prospect of subsidies). This level
of assessment is very prominent in the network literature (Kickert et al. 1997; Provan
and Milward 2001; Mandell and Keast 2007).
778 Public Management Review

The policy network itself is the second level to evaluate performance. This level
refers to the network structure and formal members in the network; it is multi-
organizational and focuses on the network as a whole. Provan et al. (2007) argue that
this level is understudied in network and performance literature.
The community level as a third level takes the action range of the policy network as
the point of departure. The community level can be interpreted in terms of policy
sectors (e.g. welfare), target groups (e.g. drug users, children), and from a territorial
perspective (e.g. stakeholders resident in a geographical focus area). A policy network
focused on tackling drug abuse for instance involves a community made up of drug
users, care organizations, justice departments, police, and schools. These actors are part
of the community on which the network activities are focused, but community
members are not necessarily network members or represented directly in the policy
network. The focus on the community level reflects an increased need to get support
from a wide range of stakeholders (Stoker 2006: 47).
A policy network performance framework should take into account the three levels
because performance may differ substantially depending on the level that an analyst is
looking at.

PUTTING IT ALL TOGETHER: A FRAMEWORK FOR NETWORK PERFORMANCE

Based on the arguments developed in the previous section, this section develops a
general framework to assess policy network performance. It does so by building on
existing frameworks, dimensions and criteria present in the literature (e.g. Kickert et al.
1997; Provan and Milward 2001; Boyne 2002; Mandell and Keast 2007; Skelcher et al.
2008).
The framework is developed along three dimensions. The three dimensions
reiterate the types of performance discussed in the previous section, namely
production performance, process performance, and regime performance. These three
dimensions need to be set against the costs of policy networks. The framework
essentially structures the arguments for a proper performance assessment of policy
networks.

Production performance: Policy networks need to deliver

This dimension closely follows the NPM interpretation of performance where goal
attainment is a key criterion. In this view, only results matter (Hatry 1999). Policy
networks are only successful if they are effective. Through policy networks,
government has to be able to do more (e.g. more outcomes), preferably with fewer
resources. Goal attainment is important for policy networks. Research shows that
without goal attainment of some sort, policy networks lose trust and ultimately become
 Voets et al.: A framework for assessing the performance of policy networks 779

irrelevant (Huxham and Vangen 2005). Goal attainment is an issue at the level of the
community, the policy network, and the participating organizations.
At the level of participating organizations, the key criterion is the fulfilment of self-
interest (Khator and Brunson 2001: 156). This criterion can be defined more broadly as
partner satisfaction. This can be measured by asking actors to what extent they are
satisfied with the realized or intermediate solutions (Koppenjan and Klijn 2004: 126).
Actors are satisfied if they can reduce uncertainty in relation to their position in the
network environment or if they are able to acquire (access to) resources of others.
Goal attainment at the level of the policy network adds up the level of goal
attainment of participating organizations. The higher the number of participants that
feel that their self-interest is being served by the policy network, the higher the
network performance is. Following the production logic, even overall network
objectives should be seen as goals of individual organizations that are however
purposively shared by other organizations.
At the level of the community, a criterion for goal attainment in the eyes of the
stakeholders is defined by Provan and Milward (2001) in terms of public perceptions
that problems are being solved.
Goal attainment at one level does not imply goal attainment at another. The network
members can consider the policy network to perform because it fully serves their self-
interest (e.g. iron triangles1), while community members outside the network might
feel the opposite or are confronted with negative side-effects (de Rynck and Voets
2006). The role of government in this respect is to ensure a balance of performance on
the different levels. If we accept representative democracy as the cornerstone of
government, the evaluation of goal attainment by network or community members
should not take precedence over the evaluation of the elected bodies of the competent
governments. We will discuss this democratic aspect in more detail below.
Production performance in a network context does not imply that goals have to be
set a priori. More likely than not, objectives will change throughout a process (Klijn and
Teisman 1997: 114). The crucial question for assessing production performance is
whether goals are attained in an efficient way. These goals may be the initial goals as
well as goals that are defined during the process. They even may be goals that are
formulated during the performance assessment itself. In the latter case, processes of ex
post rationalization or social pressure may be at stake (Koppenjan and Klijn 2004: 126).
Some scholars argue that performance of policy networks should not be related to
the idea of goal attainment at all. Nelson (2001: 93) for instance argues that goal
attainment is not a performance measure in networks that formulate strategies, because
the network interactions are strategic, not operational. The interactions within the
network aim at developing goals (see also Klijn and Teisman 1997: 113). We touch a
middle ground by contending that goal attainment can be an important criterion in
assessing policy network performance, but it is certainly not the only one. Performance
also refers to, among others, changing systems of interactions, empowering
stakeholders, or developing new relationships (Mandell and Keast 2007). We regroup
780 Public Management Review

these performance arguments in the two additional categories; process performance and
regime performance.

Process performance: Ensuring democratic quality

A second important dimension of network performance is related to the democratic

quality of policy networks (Boyne 2002; De Rynck and Voets 2006). Policy networks
are multi-organizational, and therefore the institutional arrangements of representative
democracy are insufficient for a policy network to function democratically. The
democratic performance of the policy network is mostly ingrained in the processes of
policy making and less in the products the network delivers.
According to Skelcher et al. (2008) democratic performance of policy networks
consists of three components: legitimacy; accountability; and accordance. Each
component suggests a number of criteria to assess the democratic quality of a network.

Legitimacy
For a policy network to perform democratically, it needs formal and informal
legitimacy. Formal legitimacy refers to the formal authorization for the network to act on
certain policy matters (Skelcher et al. 2008). Because a policy network is considered
part of public policy and involves public resources, it should be formally mandated to
do so. An exemplary criterion is the existence of a formal declaration of support for the
network and its agenda. For the governmental actors in the network, a criterion is a
formal decision by political representatives to set up or to engage the organization in a
policy network. At the level of the community, a measure then might be the number of
stakeholders that have issued such a declaration. A similar measure can be defined at the
level of the network members, although their membership as such might be considered
a measure of formal legitimacy. While formal legitimacy refers to a formal mandate to
act, informal legitimacy is the extent to which actors also act supportive in relation to the
actual functioning of the network (Skelcher et al. 2008). While such legitimacy might
be more important, it is more difficult to assess than formal legitimacy (Pressman and
Wildavsky 1973). In this respect, Provan and Milward (2001: 416) define the criterion
‘member commitment to network goals’ at the network level. At the community level,
a similar criterion is ‘stakeholder behaviour in relation to the network’ (2001: 416).

Accountability
Accountability is a second component of process performance (Boyne 2002). Criteria
for this component should consider two aspects, namely giving account and being held
accountable. The policy network needs to give account to its members as well as to the
community. Account giving also refers to the feedback loops between representatives in
 Voets et al.: A framework for assessing the performance of policy networks 781

the network and their parent organizations. Communication can be considered the main
measure in relation to account giving. Indicators can be the number of meetings with
community members, the rate of participation in such meetings, and the like. Besides
giving account, the policy network and its members should also be held accountable for
the network activities. This criterion can be measured by looking at mechanisms to
appoint, to remove or to sanction network members. A measure might be the presence
of a public evaluation conference discussing the policy network and its members.

Accordance
Accordance is about asking and expressing consent (or denying it) for decisions or
actions by the policy network. Accordance covers at least three criteria: issues offered
for consent; mechanisms to give consent; and status of consent (Skelcher et al. 2008).
The first criterion is about issues offered for consent: what is up for discussion? Possible
measures are the importance and number of issues that are offered for consent to
network members or to stakeholders in the network community. From a democratic
perspective, the higher the number and importance of issues offered for consent to a
wide range of members and stakeholders, the higher the performance on this criterion.
The second criterion is related to mechanisms to give consent. In a representative
democracy, elections are the main mechanism for consent. Additional instruments
include referenda, user panels, and consumer assessments (in case of service delivery
networks). Policy network performance implies the presence and use of suitable
consent giving mechanisms. The final criterion refers to the status of consent. To what
extent does giving or denying consent have an impact on the policy network? Is there a
majority mechanism or does every network member or stakeholder have a veto, or is
the status of the consent only considered advisory? A cautionary note is that private
consent should be subordinated to consent of governmental actors with democratic
backing. The reconcilement of input/positions of non-governmental and non-elected
actors with the general interest is a balancing act.

Regime performance: Networks as capacity-building

A third dimension considers the policy network as an organizational and relational

infrastructure. This dimension reflects the robustness and resilience of the network as
part of government. Is government equipped to deal with situations of societal stress
thanks to the policy network? Is government more ‘future proof’?
The first criterion is membership (Boyne 2002). Policy network performance requires
access to a range of resources: information; finances; competencies; production
capacity; and legitimacy (Koppenjan and Klijn 2004). This means that the higher the
number of members, the higher the number of relevant resources present in the
network. A policy network that grows or at least keeps its number of members is
782 Public Management Review

considered to achieve regime performance; conversely, loss of members is considered a

loss of regime performance (Provan and Milward 2001: 418). Regime performance
therefore increases with membership, because the policy network joins more resources,
making it more robust and resilient as it has a joint policy capacity.
The expansion of membership can however also lead to a decrease of network
performance on other dimensions. If a certain threshold in terms of numbers is passed,
co-ordination costs for instance might exceed the advantage of joining (resources of)
actors in a policy network and lower production performance. Membership neither
implies a positive behaviour or contribution of an actor in the policy process – an actor
might even participate precisely to hamper a policy network to function properly.
Therefore, membership as a criterion should include the way they participate in the
policy network or are committed to the policy network (Provan and Milward 2001).
This element shows how regime performance and process performance can be linked.
The second criterion for regime performance is the endurance of the policy network
or network institutionalization (Provan and Milward 2001). Klijn and Teisman (1997:
117) argue that network structures themselves are an indication of policy network
performance, as they express the wish of participants to achieve results. Their
continued presence in the public arena therefore increases government’s robustness and
resilience, as they present coalitions of government with other actors to be more
responsive to policy challenges.
Provan and Milward (2001) extend this criterion with the degree of development of
a network administration. Regime performance then is not only related to the ability of
the policy network to acquire an institutional position, but also its capacity to develop
as a proper organization. They argue that the presence, size and scope of the latter
indicate the belief of actors that the network is valuable and they express their
engagement by investing in the network. The presence of a network administration can
contribute to regime performance because the network infrastructure arguably is more
robust. A more cynical view is that such a network organization is another example of
budget-maximization, which displaces the network goals from the participants’ to the
administration’s goals. While a well-developed network administration might make the
policy network robust and resilient, it might impede production performance.
The third criterion for regime performance is the quality of the relationships between
actors (Provan and Milward 2001). To what extent have relationships in the policy
network improved during the life cycle of the network? Quality can be understood in
terms of number/density and multiplexity of relations. The idea of density is that an
extensive set of linkages in the policy network are better than a small set of linkages.
The multiplexity looks at the extent to which these relations are singular (e.g. on one
issue) or plural (e.g. on different issues). Relations between actors that are only built on
single issues indicate a lower regime performance than relations between actors that
cover multiple issues.
The quality of relations may add substantially to the capacity of government to resist
crises or to adapt to future challenges. In times of societal stress, the traditional chain of
 Voets et al.: A framework for assessing the performance of policy networks 783

command and routines often break down. The conventional ways of doing things are no
longer adequate and the capacity of hierarchic and unilateral action of governmental
actors to demonstrate robustness and resilience has decreased substantially (Schneider
2005). Governmental actors that have relationships with other governmental
organizations and private actors are expected to be better able to make things happen
in spite of difficult circumstances, as they create groupware2 (Agranoff and McGuire
2001b).

Costs: There is no such thing as a free network

Policy network performance along the three dimensions should be related to costs.
While the idea of costs is often neglected in policy network literature because it is too
closely associated with product performance and the NPM-doctrine, cost remains a key
facet. All networks bring along costs, and they are often very costly (Huxham and
Vangen 2005). These costs can be related to the resources that an actor has to deploy in
relation to the network, but also the costs of the policy developed by the network.
Costs should not be interpreted narrowly in terms of money; costs also refer to, for
instance, time, production capacities, or legitimacy. A policy network and its activities
might cost little in terms of money, but can be very expensive in terms of time.
Therefore, the potential benefits of policy networks should be weighted against the
expected costs. According to Huxham and Vangen (2005: 13) ‘seeking collaborative
advantage is a seriously resource-consuming activity so is only to be considered when
the stakes are really worth pursuing. Our message to practitioners and policy makers
alike is don’t do it unless you have to.’
A cost–benefit analysis ex ante is however more difficult for policy networks
compared to service delivery or public–private partnerships because costs and benefits
are less tangible and typically reveal themselves in the long term. In order to be
successful in dealing with wicked issues, policy networks therefore also require a certain
level of risk-taking by actors (Koppenjan and Klijn 2004).
Production, process, and regime performance all come at a cost. While cost is
considered a typical component of production performance, it is neglected in discussing
process and regime performance. In our opinion, the cost factor should be a point of
attention for assessing all three network performance types.
In production performance, the evident measure then is cost per unit of outcome or
output (Boyne 2002). The process performance however also implies costs: what is the
time or money invested to ensure certain levels of legitimacy, accountability, and/or
accordance? Organizing meetings, preparing brochures, doing surveys to measure
stakeholder satisfaction, organizing referenda, etc. to improve process performance
obviously are not cost-free. The time required to ask for consent of stakeholders or to
organize communication as part of account giving requires resources that cannot be
used elsewhere. The costs of regime performance include the network administration
784 Public Management Review

costs, but might also be a lack of competition between actors at the level of the
community.
The cost factor is linked to production, process, and regime performance, but its
concrete shape can also differ for the three levels of assessment. What are costs at the
level of individual organizations, at the level of the policy network, and at the level of
the community? Are costs equally distributed among these levels, or does one level
externalize costs to another level? Each actor can make its own assessment in terms of
costs, but costs also need to be monitored at the network level and community level
(Provan and Milward 2001).
In policy networks, there is a risk of shifting costs to actors outside the network
(Kickert et al. 1997: 172–3). A policy network might achieve high production
performance at low costs for participants but high costs for the community. Iron
triangles for instance serve their members’ interests, while externalizing costs to the
larger community outside the policy network (Schattschneider 1960). Hence, costs for
the community also need to be included in an assessment of policy network
performance. Government should ensure that there is a proper calculation of costs at
the level of the policy network and the level of the community.
In Table 1, these main dimensions and criteria are summarized.

Table 1: An assessment framework for network performance

Dimensions Components/criteria Cost factor

Production performance Goal attainment and efficiency Costs linked to production for
goal attainment
Process performance Legitimacy Costs linked to ensure process
– Formal performance (legitimacy,
– Informal accountability, and accordance)
Accountability
– Account giving
– Being held accountable
Accordance
– Issues offered for consent
– Mechanisms for consent giving
– Status of consent
Regime performance Membership Costs linked to ensuring regime
Network institutionalization performance (membership,
– Network administrative organization instititutionalization, relationship
Quality of relationships quality)
– Density
– Multiplexity
 Voets et al.: A framework for assessing the performance of policy networks 785

LIMITATIONS AND RESEARCH IMPLICATIONS

The framework to assess policy network performance developed so far has important
limitations. Some critical comments need to be made in order to use the framework
properly.

Assessment is more than measurement

It is clear that the dimensions and criteria forwarded in this article are by no means easy
to quantify. It is imperative at this point to stress the distinction between performance
assessment (which can be both qualitative or quantitative) and performance
measurement (which is quantitative). This article is focused on performance assessment
of policy networks. We contend that an assessment framework needs to start from values
and principles of government. These values and principles should be developed into
dimensions, criteria, and measures where possible. We believe that a framework based
on aspects that can be measured clearly can never be sufficient to assess performance of
policy networks. The framework developed above can be used to develop performance
measures, but the latter goes beyond the scope of this article. The development of
indicators based on the performance dimensions is a main challenge for future research.

A balanced view

In network literature, there appears to be a tendency to neglect production performance

and to stress process criteria. de Bruijn and Ringeling (1997: 158) warn not to focus too
heavily on process criteria. They argue that assessing policy network performance based
on such criteria does not guarantee ‘good policy’. While process performance might be
high, the production performance might be close to zero. The quality of interactions is
important, but if the latter is overemphasized, a risk is real that substance is neglected;
policy networks might become mere chatterboxes and might even degenerate to
proceduralism. The challenge for policy networks is to join production and process
performance, without neglecting the relevance of regime performance. The issue of how
to balance these dimensions should lead to a lively debate among performance and policy
network scholars. An important consideration should be taken into account when
searching for this balance, namely the phase in the life cycle of the network.

Different performances for different phases in the life cycle of networks

Policy networks are dynamic relational structures. A policy network can for instance
focus on strategy development in one phase of its life cycle, but can move to service
786 Public Management Review

delivery later on. A policy network is not necessarily confined to one network type for
its whole lifetime. Hence, the balance of performance dimensions will shift accordingly.
This is linked to the issue of the frequency of assessment of network performance
(Nelson 2001). As policy networks (and the network environment) are dynamic, one
should define multiple moments to assess their performance. In our opinion, there
should always be an assessment of the situation at the time the policy network is
founded or created. Too often, the performance of policy networks is questioned
without referring to (or having a good view of) the situation before the start of a policy
network.
During the network life cycle, the frequency of assessment then is variable. While
actors might permanently evaluate network performance in relation to their
organization, we consider this insufficient. A rule of thumb should be that there are
periodical reviews of network performance during the network life cycle. How
elaborate and what the frequency of such a joint assessment should be, is again
dependent on the network type.

Who is the assessor?

An important issue is who the assessor is. Is there a single assessor or are there multiple
assessors? If the latter is the case, how are different assessments balanced? Are different
assessments aggregated because all judgements of all actors are considered equal, or are
some more relevant than others (Kickert et al. 1997: 172–3)?
We believe that governmental actors should play a key role. The assessment of
government takes precedence over that of private actors. The main argument is that a
policy network is engaged in public policy making, and therefore needs to respect the
special position of government, even if non-governmental actors are involved.
This however does not mean that other assessments are not relevant. The idea of
partnership that often underlies policy networks also requires a sufficient level of
acceptance of non-governmental participants’ views. The weight of government
assessment should also depend on its portion of resources in the network (does it
provide most resources or is it a minor contributor) and the impact of the networks’
policy (does it affect the community in important ways or not?).

Contingencies

It is important to take into account that network performance is also dependent on

contingent factors. As Boyne (2002: 24) stated in his study of local authority
performance, ‘performance indicators cannot tell the whole story about local authority
performance, because other relevant variables (such as local socio-economic
circumstances) need to be taken into account when interpreting the figures.’ This is
 Voets et al.: A framework for assessing the performance of policy networks 787

similar to what Healey (1997) and others argued: one needs to read and understand the
‘politics of space’ to assess (the nature of) performance (or the lack thereof) in any
given case.
While the contingency argument applies to all levels and types of performance, the
degree of complexity increases in multi-level arrangements as multiple actors, multiple
levels, and/or multiple issues make the impact of multiple contingencies more
plausible. This element should caution one to keep the expectations about policy
network performance realistic.

Special role of government

The special position of government in policy networks has been touched upon a few
times, but needs to be stressed. Governments – consciously or unconsciously – operate
as meta-governors in relation to policy networks (Sørensen and Torfing 2006), but they
are often also a participant in the network. While this might lead to conflicting roles,
government should guard the (balance of the) three dimensions of performance at all
times and ensure a proper cost calculation for each dimension at the network and
community level. In doing so, the emphasis can differ along network types or in
different stages of a network life cycle. An important aspect of the role of government
is to ensure that the bodies of representative democracy are properly involved. Even if
policy networks are developed precisely to overcome political conflict and to deal with
issues at a certain distance from the political realm, the link with the latter should
remain clear.
The role of government is also dependent on the dominant ideology. Based on their
analysis of contemporary governing modes, Pierre and Peters (2005) for instance
distinguish five future models or scenarios, each model encompassing different
principles and hence a different understanding of how performance is to be perceived.
These models go from an ‘étatiste’-model with government as principal actor on the
one end to a ‘governance without government’-model in which states are no more than
arenas filled with self-steering arrangements consisting of private actors at the other.
Network performance in the first model will focus dominantly on the instrumentality
of networks for governmental actors. The State is or wants to be the principal actor
(e.g. government as a producer, a service provider). In the latter model, government is
not the principal but a meta-governor that organizes arenas in which policy networks
operate. In such a model, the emphasis is on safeguarding a certain quality of the arena,
an openness for stakeholders to participate in the arena, and rules for the policy
processes taking place in that arena (e.g. government as a regulator) (Sørensen and
Torfing 2006).
We acknowledge however that the position of government in relation to
policy network performance is not that straightforward and therefore requires more
thought.
788 Public Management Review

CONCLUSION

This article forwarded a framework to assess policy network performance. The

framework joins three performance dimensions (production, process, and regime),
suggests a set of assessment criteria, and links them to costs as a key factor.
Production performance is important because policy networks have to deliver results,
even if the production logic in such a network is more complex than that of a single
organization. Because policy networks are multi-actor, process performance is also an
important dimension. The focus on process performance should ensure the democratic
nature of the network. Third, regime performance is the extent to which the network
structure is resilient and robust. Does the policy network strengthen the capacity of
government to deal with crises or future challenges? Finally, all three dimensions of network
performance should be related to costs, because there is no such thing as a free network.
The framework presented in this article is under construction. It does however
suggest some next steps. The scholarly agenda includes the debate on the balance of
dimensions, the elaboration for other types of networks, the issue of developing
measures and indicators, and the role of government. In dealing with these issues, the
framework also needs to be tested empirically.

NOTES
1 Iron triangles are networks (commonly referred to as sub-governments) of interdependent, autonomous
decision-makers promoting policies that serve their self-interest (Schattschneider 1960).
2 Groupware is ‘group development that reaches mutual understanding and transcends hierarchy-based
communication/interaction that allows multiple cultures, procedures, and divisions of labor to come
together’ (Agranoff 2007: 213).

REFERENCES
Agranoff, R. (2005) Managing Collaborative Performance: Changing the Boundaries of the State?, Public
Performance and Management Review. 29: 1 pp18–45.
—— (2007) Managing Within Networks: Adding Value to Public Organizations, Washington: Georgetown University
Press.
Agranoff, R. and McGuire, M. (2001a) ‘After the Network Is Formed: Process, Power and Performance’ in
M. Mandell (ed.) Getting Results through Collaboration: Networks and Network Structures for Public Policy and
Management, Westport, CT: Quorum Books.
—— (2001b) Big Questions in Public Network Management Research, Journal of Public Administration Research and
Theory. 11: 3 pp295–326.
—— (2003) Collaborative Public Management: New Strategies for Local Governments, Washington: Georgetown
University Press.
Boyne, G. A. (2002) Concepts and Indicators of Local Authority Performance: An Evaluation of the Statutory
Framework in England and Wales, Public Money and Management. 22: 2 pp17–24.
Castells, M. (1996) The Information Age, Vol I: The Rise of the Network Society, Oxford: Blackwell Publishers.
 Voets et al.: A framework for assessing the performance of policy networks 789

de Bruijn, J. A. and Ringeling, A. B. (1997) ‘Normative Notes: Perspectives on Networks’ in W. M. J. Kickert,

E.-H. Klijn and J. F. M. Koppenjan (eds) Managing Complex Networks: Strategies for the Public Sector,
London: Sage.
De Rynck, F. and Voets, J. (2006) Democracy in Area-Based Policy Networks: The Case of Ghent, American
Review of Public Administration. 36: 1 pp58–78.
Dubnick, M. (2005) Accountability and the Promise of Performance: In Search of Mechanisms, Public Performance
and Management Review. 28: 3 pp376–417.
Dunleavy, P. (1986) Explaining the Privatization Boom: Public Choice Versus Radical Approaches, Public
Administration. 64: 1 pp13–34.
Gore, A. (1993) From Red Tape to Results: Creating a Government That Works Better & Costs Less, Report of the
National Performance Review, Washington, DC: US Government Printing Office.
Graubard, S. (2003) ‘Democracy’ in Dictionary of the History of Ideas. The Electronic Text Centre at the University
of Virginia Library. Available at http://etext.lib.virginia.edu/cgi-local/DHI/dhi.cgi?id¼dv1-78 (ac-
cessed 20 July 2008).
Gray, B. (1989) Collaborating: Finding Common Ground for Multi-Party Problems, San Francisco, CA: Jossey-Bass.
Hatry, H. P. (1999) Performance Measurement: Getting Results, Washington: Urban Institute Press.
Healey, P. (1997) Collaborative Planning: Shaping Places in Fragmented Societies, Houndmills & London: MacMillan
Press.
Hedstrom, P. and Swedberg, R. (1998) Social Mechanisms: An Analytical Approach to Social Theory, Cambridge:
Cambridge University Press.
Hood, C. (1991) A Public Management for All Seasons, Public Administration. 69: 1 pp3–19.
Huxham, C. and Vangen, S. (2005) Managing to Collaborate: The Theory and Practice of Collaborative Advantage,
New York: Routledge.
Khator, R. and Brunson, N. (2001) ‘Creating Networks for Inter-Organizational Settings: A Two-Year Follow-
Up Study on Determinants’ in M. Mandell (ed.) Networks and Inter-Organizational Settings, Westport, CT:
Quorum Books.
Kickert, W. M. J., Klijn, E.-H. and Koppenjan, J. F. M. (1997) ‘Managing Networks in the Public Sector:
Findings and Reflections’ in W. M. J. Kickert, E.-H. Klijn and J. F. M. Koppenjan (eds) Managing
Complex Networks: Strategies for the Public Sector, London: Sage.
Klijn, E.-H. and Teisman, G. R. (1997) ‘Strategies and Games in Networks’ in W. M. J. Kickert, E.-H. Klijn
and J. F. M. Koppenjan (eds) Managing Complex Networks: Strategies for the Public Sector, London: Sage.
Koppenjan, J. F. M. and Klijn, E.-H. (2004) Managing Uncertainties in Networks: A Network Approach to Problem
Solving and Decision Making, London: Routledge.
Mandell, M. (1999) The Impact of Collaborative Efforts: Changing the Face of Public Policy through Networks
and Network Structures, Policy Studies Review. 16: 1 pp4–17.
Mandell, M. and Keast, R. L. (2007) Evaluating Network Arrangements: Toward Revised Performance
Measures, Public Performance and Management Review. 30: 4 pp574–97.
Moore, M. H. (1995) Creating Public Value, Cambridge: Harvard University Press.
Nelson, L. S. (2001) ‘Environmental Networks: Relying on Process or Outcome for Motivation’ in M. Mandell
(ed.) Getting Results through Collaboration: Networks and Network Structures for Public Policy and Management,
Westport, CT: Quorum Books.
O’Toole, L. J. (1997) Treating Networks Seriously: Practical and Research-Based Agendas in Public
Administration, Public Administration Review. 57: 1 pp45–53.
Pierre, J. and Peters, B. G. (2005) Governing Complex Societies: Trajectories and Scenarios, London: Palgrave
Macmillan.
Pollitt, C. and Bouckaert, G. (2004) Public Management Reform: A Comparative Analysis, Oxford: Oxford University
Press.
790 Public Management Review

Pressmann, J. and Wildavsky, A. (1973) Implementation, Berkeley, CA: University of California Press.
Provan, K. G. and Kenis, P. (2008) Modes of Network Governance: Structure, Management, and Effectiveness,
Journal of Public Administration Research and Theory. 18: 2 pp229–52.
Provan, K. G. and Milward, H. B. (2001) Do Networks Really Work? A Framework for Evaluating Public-Sector
Organizational Networks, Public Administration Review. 61: 4 pp414–23.
Provan, K. G., Fish, A. and Sydow, J. (2007) Interorganizational Networks at the Network Level: A Review of
the Empirical Literature on Whole Networks, Journal of Management. 33: 3 pp479–516.
Schattschneider, E. E. (1960) Semisovereign People: A Realist’s View of Democracy in America, New York: Holt,
Rinehart & Winston.
Schelling, T. (1978) Micro-Motives and Macro Behavior, New York: Norton.
Schneider, S. K. (2005) Administrative Breakdowns in the Governmental Response to Hurricane Katrina, Public
Administration Review. 65: 5 pp515–16.
Skelcher, C., de Rynck, F., Klijn, E.-H. and Voets, J. (2008) ‘Designing Democratic Institutions for
Collaborative Economic Development: Lessons from Belgian and Dutch Cases’ in M. Considine and
S. Giguere (eds) Government and Communities in Partnership: The Theory and Practice of Local Governance and
Economic Development, London: Palgrave.
Sørensen, E. and Torfing, J. (2006) ‘Theoretical Approaches to Metagovernance’ in E. Sørensen and J. Torfing
(eds) Theories of Democratic Network Governance, London: Palgrave MacMillan.
Stoker, G. (2006) Public Value Management: A New Narrative for Networked Governance?, American Review of
Public Administration. 36: 1 pp41–57.
Van Dooren, W. (2006) Performance Measurement in the Flemish Public Sector: A Supply and Demand Approach, Leuven:
Katholieke Universiteit Leuven.
—— (2007) ‘Quality and Performance Management: Toward a Better Integration?’ in P. Delancer Julnes, F.
Stokes Berry, M. P. Aristigueta and K. Yang (eds) International Handbook of Practice-Based Performance
Management, Thousand Oaks, CA: Sage.
—— (2008) Government and the GDP; the Politics of National Accounts Statistics, Journal of Comparative Policy
Analysis, (forthcoming).
Voets, J. (2008) Intergovernmental Relations in Multi-Level Arrangements: Collaborative Public Management in Flanders,
Leuven: Katholieke Universiteit Leuven.