European Journal of Operational Research 222 (2012) 350–360

Contents lists available at SciVerse ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Innovative Applications of O.R.

DPSIR = A Problem Structuring Method? An exploration from the

‘‘Imagine’’ approach
Simon Bell
Communication and Systems Dpt., Maths, Computing and Technology Faculty, Open University, Milton Keynes MK7 6AA, United Kingdom

a r t i c l e i n f o a b s t r a c t

Article history: The Drivers, Pressures, State, Impact and Response or DPSIR framework has been with us for over a dec-
Received 1 August 2011 ade now and it is widely used as a means to assess and measure and, eventually provide a guide to man-
Accepted 24 April 2012 aging the environment. With its repertoire of diagnostic and analytical components the DPSIR can be
Available online 11 May 2012
argued to be a Problem Structuring Method or PSM. Criticisms of the framework abound but it has a resil-
ience which is noteworthy. Some argue that DPSIR, by its nature, is a narrowly formulated, engineering
Keywords: device, incompatible with the multiple perspectives which human interaction in global ecology requires.
DPSIR
Is there a value in DPSIR being more flexible in expression and experience of users? In this article it is
PSM (Problem Structuring Method)
Imagine
shown how the DPSIR framework was applied within a multi-methodology approach called Imagine in
Coastal sustainability a number of coastal management projects around the Mediterranean and in other contexts. The article
argues that DPSIR, whilst admittedly limited in its scope and approach can, if applied in a participatory
and systemic multi-methodology, combine with other tools and help to create outcomes of value to local
populations.
Ó 2012 Elsevier B.V. All rights reserved.

1. Background: The DPSIR and PSMs ‘‘it is becoming more and more difficult for policy-makers to
grab the relevance and meaning of the existing environmental
To make informed decisions relevant information is needed and indicators, given the number and diversity of indicators pres-
multiple views and interpretations are helpful in order to gain bal- ently in use. And new sets of environmental indicators are still
ance and insight. Information is often held in the form of indica- to be expected. Therefore, some means of structuring and ana-
tors; so, it might be expected that decision making would lysing indicators and related environment/society inter-connec-
include the use of indicators? Building on this and looking for guid- tions is needed’’ (Smeets and Weterings, 1999, p. 4).
ance of good practice, the first place to look for evidence of the
inclusion of indicators in decision making might be in authoritative The challenge for DPSIR is to be both a precise PSM and of wider
books on use of Problem Structuring Methods (PSMs)? Looking in use to stakeholders. If it can be so managed DPSIR shows great flex-
Rosenhead and Mingers’ highly quoted book (Rosenhead and ibility and usefulness. The purpose of this paper is to show how
Mingers, 2001) for mention of indicators or more specifically, argu- this can be accomplished using Imagine. Emerging from this com-
ably, the most used Environmental Indicator framework: DPSIR ment, DPSIR can be seen as a ‘means’ for structuring and analysing;
(Drivers, Pressures, State, Impact and Response) resulted in a blank indeed a sense making device for a policy aiding tool – environ-
(although, it needs to be mentioned here that this paper owes a mental indicators. DPSIR is not without criticism. It has been ar-
debt of gratitude both in terms of domain content and inspiration gued to be very limited when it comes to sustainability research
for the as yet unpublished work: ‘‘A Problem Structuring Method (for example see the discussions in Carr et al. (2007) and Wiek
for Ecosystem-Based Management: The DPSIR Modelling Process’’ et al. (2011)).
by (Gregory, 2012). I was grateful to receive a pre-published In the Operational Research literature the issue of indicator use
version of this paper and it has been profoundly influential to my emerges on a regular basis (for an eclectic mix of various uses see:
subsequent thinking). Foxon et al., 1999; Andranik, 2007; Bellotti and Crook, 2009; Choi
Before progressing further it might be useful to be clear about and Bae, 2009; Medhurst et al., 2009; Ness et al., 2010; Ülengin
what DPSIR is? Smeets and Weterings (1999) suggested that the et al., 2010). However, the collective use of indicators as a Problem
DPSIR framework acts as a form of typology: Structuring Method within the DPSIR model devised by the Euro-
pean Environmental Agency (EEA, 1999) is, as with Rosenhead
and Mingers, much less well documented.
The near invisibility of DPSIR in the Operational Research liter-
E-mail address: [email protected] ature is not reflected more widely. The use of indicators in the

0377-2217/$ - see front matter Ó 2012 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.ejor.2012.04.029
 S. Bell / European Journal of Operational Research 222 (2012) 350–360 351

DPSIR framework is well noted and in diverse fields (for a diverse

selection from various journals see: Petit et al., 2001; Casazza et al.,
Drivers
2002; Elliott, 2002; Simon, 2004; Green et al., 2005; Stem et al.,
2005; Mace and Baillie, 2007; Benjaminsen and Svarstad, 2008;
Brenner et al., 2008; Svarstad et al., 2008; Maxim et al., 2009;
Atkins et al., 2011). But, maybe an assumption is incorrect in this Pressures Response
thesis. Can indicators, applied in a framework, be considered to
be a Problem Structuring Method?
Rosenhead and Mingers say that Problem Structuring Methods
(PSMs): ‘‘provide a repertoire of methods for making progress with
ill-structured problem situations’’ (Rosenhead and Mingers, 2001,
p. 9). However, they also distinguish ‘traditional’ operations re- State Impact
search approaches and PSMs of an alternative form such as Soft
Systems Methodology (Checkland, 1981; Checkland and Scholes,
1990) and SODA (Ackermann, 1996). Simplistically, the traditional Fig. 1. The DPSIR framework.
methods tend to quantification, whereas, as Rosenhead and Min-
gers say, the ‘alternative paradigm’: ‘‘use methods or models (often
in the plural, and with little or no quantification). . .’’ (Rosenhead Response that feeds back on the Driving forces, or on the state
and Mingers, 2001, p. 1). More tellingly, the two forms are com- or impacts directly, through adaptation or curative action’’
pared and contrasted. Table 1 provides an overview of this (Smeets and Weterings, 1999, p. 6).
comparison. Such a clear definition is represented in other practice literature
Rosenhead and Mingers’ types may be thought of as ideal types, (three interesting applications from diverse fields of application
representing extreme versions of two traditions but it could be ar- are: Kristensen, 2004; Borja et al., 2006; Jago-on et al., 2009).
gued that the traditional application of indicators in problem solv- But, even for a traditional view of problem solving, Smeets and
ing is top down, quantitative and therefore tending to the left hand Weterings’ is a simplified explanation which the authors recognise.
column. To explore the landscape of this more fully it is useful to However, their primary point is the value of DPSIR to policy mak-
ask, what is the purpose of indictor based problem structuring? ers. They argue:
Smeets and Weterings suggest that environmental indicators have
three purposes: ‘‘from the policy point of view, there is a need for clear and spe-
cific information on
‘‘1. to supply information on environmental problems, in order (i) Driving forces and
to enable policy-makers to value their seriousness; (ii) the resulting environmental Pressures, on
2. to support policy development and priority setting, by iden- (iii) the State of the Environment and
tifying key factors that cause pressure on the environment; (iv) Impacts resulting from changes in environmental quality
3. to monitor the effects of policy responses’’ and on
(Smeets and Weterings, 1999, p. 5). (v) the societal Response to these changes in the environment’’
This tends to confirm the view that the indicators are in the left (Smeets and Weterings, 1999, p. 6).
hand column of Rosendhead and Mingers’ table being intended for Thus, it can be interpreted that policy makers require informa-
single decision makers, etc. What is the DPSIR framework which tion in the form of simplified indicators which come pre-labelled to
Smeets and Weterings describe? On p. 6 of their book Smeets denote the characteristics which they are assessing – be this a
and Westerings provide a ‘systems analysis’ view of DPSIR and de- pressure on resources, a measurement of a response policy already
scribe it as follows (Fig. 1): in place, etc. The value of pre-categorising and labelling indicators
‘‘According to this systems analysis view, social and economic does not stop at DPSIR. Smeets and Weterings also divide DPSIR
developments exert Pressure on the environment and, as a con- indicators into types which denote descriptive, performance, effi-
sequence, the State of the environment changes, such as the ciency and welfare characteristics – a framework within a frame-
provision of adequate conditions for health, resources availabil- work. This view of a reductive tendency to reformulate the world
ity and biodiversity. Finally, this leads to Impacts on human in terms of measurable, pre-determined categories might be
health, ecosystems and materials that may elicit a societal thought to confirm the view that indicators are a ‘traditional’ ap-
proach, dealing with data and for the consumption of experts for
high level policy making. But, maybe this is a simplification? The
Table 1 DPSIR literature and application have evolved and extended in
Traditional and alternative paradigm PSMs compared.
the last 10 years.
Traditional approaches to problem Alternative paradigm PSMs Reading on into the DPSIR literature a sense of greater social
solving awareness and practicality at an operational level emerges. Elliott
Problem formulation in term of a Non-optimizing; seeks alternative in his application of DPSIR to the marine ecosystem suggest six te-
single object and optimization solutions which are acceptable nets for environmental management:
Overwhelming data demand Reduced data demand
Scientization and depoliticization, Simplicity and transparency, aimed at ‘‘Our actions have to be:
assumed consensus clarifying the terms of conflict environmentally sustainable (i.e. good for nature now and in
People are treated as passive objects Conceptualizes people as active
the future);
subjects
Assumption of a single decision Facilitates planning from the bottom technologically feasible (i.e. with appropriate methods and
taker up equipment);
Attempt to abolish future Accepts uncertainty economically viable (i.e. at a reasonable and tolerable cost);
uncertainty socially desirable (i.e. wanted by our societies);
Adapted from Rosenhead and Mingers, p. 11. legally permissible (i.e. within our defined laws at national and
international level);
352 S. Bell / European Journal of Operational Research 222 (2012) 350–360

administratively achievable (i.e. carried out by our system of Fig. 2 presents Imagine as possibly a multi-methodology. Imag-
departments, agencies and governments)’’ (Elliott, 2002, p. iii). ine itself has its intellectual origins in the Soft Systems Methodol-
ogy. The inclusion of indicators in the approach implies a much
The first three are well noted in the literature, the latter three are more conventional view of problem solving. The explanation for
added to ensure that the ‘solutions to environmental change sit this and a discussion of the basis for the claim is explored over this
within our developed systems’. This suggestion of greater embed- and the next section.
ding of the DPSIR approach within social, legal and administrative Between 1999 and 2012 the Imagine approach to indicator
systems might be argued to provide notice of the opportunity for application for communities of all kinds was developed and ex-
a slight oscillation of the DPSIR framework in practice from the left plored in a number of contexts.
hand column in Rosenhead and Mingers table to the right hand col- Primarily, it was applied in a series of coastal area management
umn. This, I argue in this paper, might prove to extend the useful- programmes in Malta, Lebanon, Algeria, Slovenia, Cyprus and
ness of DPSIR as a PSM – offering technical precision and Spain. Imagine arose from joint work of Bell and Morse and Plan
stakeholder engagement. I return to this in later stages of the paper. Bleu, a French environmental agency. Most of these applications
However, problems remain for DPSIR. More recently, Svarstad of Imagine are set out in the reports on the Plan Bleu website
et al. (2008) argue that DPSIR, as a framework containing pre- http://www.planbleu.org/publications/littoralUk.html and they
determined, measurable indicators must represent the world view are recorded in a number of published articles (Bell and Morse,
of the researcher. As they put it: 2001, 2003, 2004, 2005a,b, 2007a,b, 2009; Bell and Coudert,
‘‘The problem with the framework is the lack, so far, of efforts to 2005; Coudert, 2006; Coudert and Larid, 2011).
find a satisfactory way of dealing with the multiple attitudes Imagine was further refined as a participatory means for com-
and definitions of issues by stakeholders and the general pub- munity engagement and as an online Virtual Learning Environ-
lic’’ (Svarstad et al., 2008, p. 116). ment course by the Open University following development
investment from the UK Academy for Sustainable Communities
It is this key issue of the value of multiple perspective, the inclu- and the Homes and Communities Agency. The development of
sion of alternative stakeholder views and, therefore, the potential Imagine in these guises is documented at the Open University
for DPSIR to be more fully integrated across Rosenhead and Min- http://learn.open.ac.uk/course/view.php?id=5548, and in publica-
gers’ PSMs that the remainder of this paper is focused on. tion (Bell, 2008, 2011; Bell and Lane, 2009).
It would seem that DPSIR framework as presented is a measure- More recently, the method has been used by practitioner com-
ment device which can be nuanced for wider application and munities in internet development. Within the Open University eS-
appreciation but which is fundamentally an expert device – de- TEeM project: Infinite Bandwidth, Zero Latency (IBZL), Imagine has
vised by specialists, to be interpreted by specialists for the funda- been applied as the primary means to educe creative insight from
mental application of policy makers. But, does this have to be the workshop participants (Bell and Walker, 2010).
case? Surely almost all human societies make use of indicators in But what is Imagine? Fig. 2 provides an overview of the ap-
various ways – especially if we extend the interpretation of the proach as applied in some of the coastal area management pro-
word to include weather forecasts, prices of commodities and foot- grammes in the Mediterranean. Imagine as a PSM has had three
ball match results. Could DPSIR be ‘handed over’ to community as a separate expressions.
means to self – assess sustainability or other matters of public con- In 1999 Imagine was first realised as ‘Systemic Sustainability
cern? Is the framework potentially viable in a more stakeholder Analysis’ or SSA. In this guise it took a double loop, infinity symbol
facing, over-arching approach? figure and had nine stages:

2. DPSIR within ‘Imagine’ 1. Find out how things are – by use of devices like Rich Pictures
2. Understand context – refining the meaning of the pictures
In 1999 Bell and Morse argued that indicators – and specifically 3. Gather stakeholders – finding out who this effects
sustainability indicators (SIs) – needed to be more centred in the 4. Be clear on methods – gaining stakeholder ideas for and
needs of population and less in the academic/expert conceptualisa- acceptance of approach
tion of indicator makers. Pre-empting the sentiment already seen 5. Sub loop on the stakeholder sessions
in the work of Svarstad et al. the authors argued: Identify stakeholder coalitions
‘‘We are interested in moving towards a measurement of the Identify and agree indicators
immeasurable. It is our contention that the idea of sustainabil- Identify bands of equilibrium which mark what is consid-
ity being measured in absolute, traditional, reductionist terms ered to be ‘sustainable’
as with SIs is non-viable. It cannot be done because sustainabil- Develop AMOEBA diagrams (described later)
ity itself is not a single thing. Or better, it can be done but it will Extend the AMOEBA over time
be done badly, over simplifying complexity and reducing a vari- 6. Unpack the AMOEBA – what does the future hold?
ety of relevant and legitimate views and understandings to the 7. Respond to good AMOEBA – how to help good futures to
dominant mindset of the scientist. Sustainability is, we believe happen
we have shown, a highly complex term open to a wide variety 8. Respond to bad AMOEBA – how to avoid bad futures
of interpretations and conceptualisations. In short, it is a con- 9. Planning phase and working out what happens next
cept dependent upon the various perceptions of the stakehold-
ers in the problem context’’ (Bell and Morse, 1999, p. 100). Although SSA was to morph first into ‘Systemic Prospective Sus-
tainability Analysis’ (SPSA) with a higher degree of focus on future
The authors demonstrated a focus on the ‘alternative paradigm’ scenarios; and then into the simplified five step version which is
(for example use of terms like ‘problem context’ are redolent of the shown in Fig. 2 Imagine, the main themes of Imagine are clear in
Soft Systems Methodology) and yet also identified the value of the original SSA. Critical to SSA is the ‘sub-loop’ which effectively
indicators. This evidence of interest in the values of both brings stakeholders together with DPSIR. The essential ‘drive’ of
traditional and alternative approaches was then explored in great- the sub-loop is to encourage participants to express, in their own
er detail. The primary output of the work of Bell and Morse was the words, how they would know change was happening (for good
Imagine methodology shown in Fig. 2. or bad) in their context. The way in which they would understand
 S. Bell / European Journal of Operational Research 222 (2012) 350–360 353

Fig. 2. Imagine – a multi-methodology(?).

this would be via measurable indicators which would be defined the Imagine workshop. Each of these features we assume would
and explored in the participatory workshops. These indicators have a value within a range or band at which it would be consid-
could subsequently be defined in DPSIR terms, e.g.: ered by the participants to be in equilibrium or in balance. This
is not imposed on the participants but comes from their own
 Driving forces for change – for example an expanding tourist understanding and discourse.
industry Holling and Gunderson have considered the notion of equilib-
 Pressures resulting from change – e.g. demands on fragile rium and resilience. In their chapter of 2002 they noted two
local resources like drinking water models:
 State changes in the local context
‘‘The first definition, and the more traditional, concentrates on
 Impacts of this and
stability near an equilibrium steady state, where resistance to
 Measures of response
disturbance and speed of return to equilibrium are used to mea-
sure the property .. We term this engineering resilience . . .. The
In this sense the method does not put the DPSIR criteria first,
Second definition emphasizes the conditions far from any equi-
rather the stakeholders in the context are asked to express their
librium steady state, where instabilities can flip a system into
views regarding context and then these ‘views’ and resulting mea-
another regime of behaviour .. In this case resilience is mea-
sures can be framed in terms of the DPSIR.
sured by the magnitude of disturbance that can be absorbed
Imagine is described in greater detail elsewhere in publications
before the system changes its structure by changing the vari-
already noted in this paper. It is necessary here to understand the
ables and processes that control behaviour. This we term eco-
approach in outline and to note how indicators are developed and
system resilience’’ (Holling and Gunderson, 2002, pp. 27–28).
related to DPSIR.
Essentially Imagine has four stages which relate to a learning In the Imagine workshops the participants are encouraged to
cycle such as that noted by Kolb (1984). The early stages of Imagine consider the equilibrium of any feature they think important –
involve participatory visioning of the current situation facing the for example the equilibrium band in terms of the provision of hos-
community via workshop and Rich Picture exercises (for much pital beds or the equilibrium band of marine pollution. The band
more on this approach see: Bell and Morse, 2010). Participants notion is important. We do not assume that sustainability is a
are encouraged to consider how the features of the context as number (e.g. 42) or a point. It is a negotiated band which the group
drawn in the Rich Picture could be measured and ‘known’ for can agree from their multiple interests and perspectives. In the
change, i.e. they are encouraged to develop their own indicators. same way that the indicators come from the stakeholders, so the
At this stage no feasibility analysis takes place – the important band of agreed minimum and maximum sustainability or equilib-
point is for the participants to agree on a series of ways of knowing, rium arises from the discourse of the group. This equilibrium could
measuring and understanding changes in their context. In this way be equated with either of Holling and Gunderson’s definitions –
the participants are encouraged to own their own measurement. ultimately the decision rests with the stakeholders.
During this process those engaged in the workshop develop ‘bands Following the preliminary development of indicators and the
of equilibirum’ for each indicator which they agree. This is an establishment of a band of equilibrium for each indicator selected,
important point. Each indicator is assumed to measure some fea- stakeholders are encouraged to gain data for the model. By contrast,
ture of the context which is of importance to the participants of this can be a technical process where subject specialists explore ac-
354 S. Bell / European Journal of Operational Research 222 (2012) 350–360

tual and feasible indicator sets which have current credibility. In both cases there were some common features relating to the
There needs to be a balance of ‘real’ data gathered from conven- manner in which DPSIR was applied within the workshops. Ques-
tional sources (if such data or such sources exist) and simulation tions addressed included:
data taken from the views and thoughts of the group. The second,
simulation data is clearly subjective and prone to distortion and er-  How did people interpret the concepts?
ror. However, if the group is representative of the local context and  Did some people struggle with DPSIR and Imagine more
includes a range of professions and expertises as well as lay per- generally?
sons, it is sometimes surprising to see how accurate guesstimate  Did some stakeholders engage with these concepts more than
data can be when subsequently compared to actual figures. others?
When data has been gathered to populate the indicator model  Did people try to introduce other concepts that didn’t fit?
the participants are encouraged to produce AMOEBA diagrams rep-  Were there ‘aha’ reactions to DPSIR that helped people clarify
resenting the context. Such an AMOEBA is shown in Fig. 3. their thinking?
In the diagram each ‘arm’ of the AMOEBA is an indicator, the  Why did people reframe their outputs in terms other than
outer and inner circle/doughnut/represents the band of equilib- DPSIR if the framework was useful?
rium and the deviation of each AMOEBA arm from this band repre-
sents unsustainability either by deficit (if it falls short of the inner These questions will be returned to in the assessment of each of
circle of the band) or by excess (if it falls outside outer circle of the the cases.
band). In the example shown (from Malta) the participants have
developed three separate AMOEBA, for three snapshots. In each 3.1. Indicators and DPSIR in Malta
case the relative self-developed vision of comparative sustainabil-
ity can be seen in terms of the deviation from the band of equilib- The Coastal Area Management Programme in Malta was under-
rium. The relative ‘performance’ of each indicator can be rapidly taken from 1999 to 2002 and is reported by Ellul (Ellul 2002). El-
visualised and the group can begin to consider what action if any lul’s report sets out the form of the intervention (Imagine at this
is needed to improve the situation. Further, the participants can time known as Systemic Prospective Sustainability Analysis) and
explore possible trends over time, review pressures from different the embedding of various forms of DPSIR indicators within the pro-
indicators (for example, does the improvement of one indicator ject (Ellul, 2002, p. 10). Ellul sets out one of the primary objectives
over time come at the expense of a different indicator degenerat- of the project:
ing?) and assess the value of past and current activities in terms ‘‘to contribute to efforts towards a sustainable development of
of the reaction of indicators. Such use of the indicator set could the island, and in particular of its north-west area by preparing
be argued to be problem identification, analysis and solving. a set of sustainability indicators and a systemic sustainability
In the next section we look at the main features of this element analysis, to be made on the primary basis of a description and
of Imagine in more detail using data drawn from the case studies assessment of the level of sustainability by consideration of
undertaken in Malta and Slovenia. the main indicators and the process which generated them’’
(Ellul, 2002, p. 3).
3. Indicators and DPSIR in imagine workshops
The application of the SPSA/Imagine methodology is described
This paper is not attempting to present Imagine as a ‘silver bul- in the report. Six workshops were undertaken over two years
let’ for the participatory application of DPSIR but, there is evidence involving around 20 stakeholders at each. Stakeholders were se-
that the various workshops – most specifically those in Malta and lected by Maltese members of the project in order to represent a
Slovenia, provide evidence that participants in workshop contexts cross section of local interests. In so far as possible the same partic-
can: ipants were encouraged to attend all workshops. The main indica-
tors developed are shown in Table 2.
 Explore their context in terms of sustainability Six workshops over the three years involved over 40 stakehold-
 Express their problem formulation in terms of tasks and issues ers from diverse coastal groups (fisheries, tourism, farming cooper-
 Which can subsequently be expressed by these participants as atives, hotels, etc.). The groups produced a range of measurable
measurable indicators concepts regarding sustainability. In the table 27 indicators are
 And then considered in terms of the DPSIR framework shown. The indicators reflect concerns arising from the various
strands of the CAMP Malta project (Sustainable Coastal Manage-
This would provide some evidence that the framework can be ment, Marine Conservation Areas, Integrated Water Resource Man-
applied as part of a methodology in a bottom up and participatory agement, Erosion/desertification Control Management and
manner. If this claim can be substantiated then the argument can Tourism and Health). For each indicator a note provides guidance
be made that Imagine constitutes a form of multi-methodology, on how it will be measured, the upper and lower limits of the band
allowing the integration of qualitative and quantitative data, both of equilibrium (the band within which the stakeholders believe
bottom up and top down. For the purposes of this analysis two that the indicator shows a sustainable return) and the value of
cases are reviewed, CAMP Malta and CAMP Slovenia. In both cases the indicator in 2000.
the DPSIR was introduced as part of an introductory presentation The State, Pressure, Driving Force and Response concepts from
to the relevant section of the workshops. They were presented as the DPSIR framework were applied in the consideration of the bal-
a suggested means to frame the indicator work but were not in- ance of indicators selected at the time of the workshops but is not
sisted upon in subsequent reporting. The framework was also visible in this table. They informed the process of indicator selec-
handed over to the local facilitator and they were encouraged to tion but were not evident in reporting. In the table a retrospective
consider it and apply it in inter-workshop periods if this was allocation of DPSIR criteria to the indicators is set out. All indicators
thought to be helpful. In both case studies the DPSIR seemed to can be seen as being allocatable to DPSIR criteria.
help those engaged in the workshops to realise that sustainability Following discussion in the stakeholder group workshops it was
could be measured and assessed according to different criteria – decided that the framework for capturing the indicators and subse-
that it was a more complex and ill-defined concept than they quently showing them in AMOEBA form would be by domains:
might have at first considered. Society, Environment, Economy and Technology.
 S. Bell / European Journal of Operational Research 222 (2012) 350–360 355

Fig. 3. Indicators and AMOEBA diagram.

Table 2
North West Malta indicators.

No. Indicator Domain Polarity Note Maximum Minimum 2000

1 Scheduled/protected areas in NW 1 > % of the total coastal area of the NW 80 65 66
2 Abandoned agricultural land 1 > % of total agricultural land 25 7 15
3 FishTarms in the NW 1 < Number of farms 5 2 5
4 Cars (ravelling through Ihe NW 1 < Number of cars during peak 3000 1000 4500
5 Marine vessels in trie NW 1 < Number of marine craft during peak weekend 700 400 1000
6 Enforcement actions by PA 1 > Annual number of cases 60 25 68
7 Marine conservation/protected 1 > % of coastal length 20 10 0
areas
8 Diving in the NW 1 < No. of dives 40,000 15,000 55,000
9 Bathinq water quality 1 > % of samples meeting acceptable levels of faecal conforms 95 85 98.3
(<1000 milligram/liter)
10 Number of breaches in rubble walls 1 < No. of breaches 10 5 11
11 Pollution in ground water 1 < Level of nitrate (milligram/liter) 50 25 65.27
12 Unemployed as a % of working 2 < % of working population in NW 3 I 1.8
population
13 Full time farmers 2 > % of total farmers 50 40 44
14 Tourist accommodation occupancy 2 > Occupancy % during winter 55 35 26
– winter
15 Employment in tourism 2 > Fulltime employees in NW % of total 25 15 14
16 No. of claims for storm damaqe 2 < No. of annual claims 50 25 72
17 TSE recycled water 2 > % of water consumed 80 50 4.6
18 leaked water 2 < Cubic metres per hour 600 300 1200
19 level of bunkering operations 2 < % of total operations in Malta 20 5 19.3
20 Population growth in the NW 4 > Annual rate of growth 5 2 1.4
21 population density in NW 4 > Population persq km 500 300 328
22 Beach closure 4 < Number of days during summer 15 2 25
23 Tourist resident ratio – summer 4  Daily tourists as a % of residents 95 70 136
24 Gastroenteritis outbreaks in NW 4 < No. of total outbreaks in a year 3 1 5
25 Quality of drinking water (1) 4 < Level of chloride (milligram/liter) 800 200 517
26 Quality of drinking water (2) 4 < Level of nitrate (milligram/liter) 50 15 56
27 Quality of battling water 4 > No. of points obtained on faecal coliform readings 50 35 40

List of key Sis for the NW.

Note: due to the unavailability of data for 2000 some figures represent data for 1999.
Relevance to DPSIR: D – Indicators: 7, 10, 21 P: 4, 5, 8, 11, 12,18, 19, 20 S: 1, 2, 3, 9, 13, 25, 26, 27 I: 14, 15, 16, 23, 24, R: 6, 17, 22.

The resulting AMOEBA is shown in Fig. 4. to work) concepts of ‘driving force’, ‘impact’ and ‘state’ emerged
In reflection on the six questions I set out at the beginning of regularly from the workshop conversations. Certainly the techni-
this brief description, the following observations emerge from field cally minded and those engaged in pressure groups (e.g. Gaia
notes taken at the time. DPSIR was generally an enabling concept. Foundation) applied the terms and ideas more often and in one
Presented in a non-directive manner (not imposed as ‘the’ means case a stakeholder tried to apply the DPSIR as the primary frame-
356 S. Bell / European Journal of Operational Research 222 (2012) 350–360

Fig. 4. Maltese AMOEBA.

work for his group to think and work. The group did not take up Following the same pattern as Malta, the upper and lower limits
this rigid application but, from observation, nevertheless found it of the bands of equilibrium are shown for each indicator and the
a useful touchstone in the consideration of the indicator mix. DPSIR unit of measure is provided. Unlike Malta, the Slovenian stakehold-
ran in background more generally and did not seem to produce (as ers provided indicators for three time periods 1991, 1996 and
a single concept) ‘aha’ moments – these tended to emerge from 2003, thus showing evidence of trend. For both Tables 3 and 4
Rich Picture drawing. The local facilitators, responsible for report the indicators have been retrospectively allocated to the DPSIR
writing and bringing workshop output together into a common framework. The resulting AMOEBA diagrams for Cast and Coastal
format preferred to apply more generic and locally relevant terms areas for 2003 are shown in Figs. 5 and 6.
for collective gathering of the indicators. Again, echoing the approach in Malta, the Slovenian stakehold-
ers decided to use their own framework for organising the indica-
3.2. Indicators and DPSIR in Slovenia tors – in this case in terms of Economy, Social, Environment and
Tourism. In this manner the DPSIR which was applied in the work-
The work of the Slovenian CAMP project is presented in the re- shops provided a framework for group discussion but not the
port by Maher (2006). As with CAMP Malta, CAMP Slovenia was a means for indicator clustering. Some of the Slovenian stakeholders
programme containing a number of projects (Spatial Development were sophisticated users of DPSIR, having worked on previous
of South Primorska, Detailed Conception of Coastal Strip Spatial European Environmental Agency projects. There were attempts
Arrangements, Regional Strategy of Sustainable Tourism Develop- by sub-groups to impose DPSIR more explicitly but this was never
ment, Management of Protected Areas, Regional Programme of successful. The groups, as in Malta, tended to absorb the DPSIR
Environmental Protection and Water Resources and Sensitivity idea, include it in the formulation of the mix of indicators but defer
Maps of the Slovenian Coast). Also, as with the Maltese example, from explicit application.
the use of SPSA/Imagine was to be as a horizontal activity bringing
the various strands together and allowing local people to assess 3.3. Conclusions arising from the two cases
and plan for their own sustainability. Five workshops were under-
taken over eighteen months involving around 20 stakeholders at In both the Maltese and Slovenian cases a wide range of indica-
each. As in Malta, stakeholders were selected by Slovenian mem- tors were produced and, in both cases:
bers of the project in order to represent a cross section of local
interests. In so far as possible the same participants were encour-  The indicators represented a variety of coastal issues
aged to attend all workshops. Again, as in Malta, the report notes  All indicators were measured
the importance of DPSIR indicators (Maher, 2006, pp. 7–8) but,  Each indicator produced a upper and lower limit for the band
also, as in the Maltese case, this inclusion of the idea of DPSIR in of equilibrium
the underlying methodology, is not reflected evidentially in the  Each indicator was shown in relation to this band of
results. equilibrium
The Slovenian project produced indicators for the Cast and the  In both projects the DPSIR framework was applied as a means
Coastal areas. These are shown in Tables 3 and 4. to select and consider the indicators but
 S. Bell / European Journal of Operational Research 222 (2012) 350–360 357

Table 3
Cast area indicators from Slovenia.

No. Indicator Domain BoE Unit 1991 1996 2003

Minimum Maximum
1 Public waste removal Env 12 20 kg per population 21.07 52.18 25.31
2 % of connected households to public sewage system Env 80 90 % 18 19 24
3 Share of active working population Social 40 70 Share % 43 48 47
4 Daily migration/number of active working force Social 1500 2500 Rate 2100 3400 5000
5 Aging index Social 35 50 Rate 80 112.1 128.2
6 Educational structure of inhabitants: % of high education Social 20 30 % 11.5 16.7 10.31
7 Number of arrivals and nights of tourists per 100 inhabitants Tourism 250 350 Nights/100 241.37 210.33 211.88
8 Number of beds per 100 inhabitants Tourism 5 8 Beds/100 2.27 1.94 3.11
9 Gross income tax base per capita Economy 105 130 Index (Slovenia = 100) 103.8 107.2 104.2
10 Business – net profit/loss per employee Economy 300 600 In SIT ’000 329 289 286

Relevance to DPSIR: D – Indicators 1, 7 10 P: 3, 8 S: 2 I: 4, 5, 9 R: 6.

Table 4
Coastal area indicators from Slovenia.

N° Indicate Domain BoE Unit 1991 1996 2003

Minimum Maximum
1 Urbanization rate Social 60 70 % 63.1% 66.3% 71.8%
2 % of connected households to public sewage system Env 5 90 % 42% 55.2% 70.2%
3 Quality of drinking water, % of unsuitable samples Env 0 2 % 2% 2.1% 1.1%
4 Quality of sea water in public baths % of good microbiological samples Env/Tourism 90 100 % 72 74.4 86.7
5 Rate of coastline with regulative approach Tourism 30 50 % of land 28 35 45.2
6 Investment in management of nature protected areas on coast Economy 50 100 MIO SIT 18 23 50
7 Employment structure Economy 2 Number 1 1 1
8 Number of bed places per 100 inhabitants Tourism 30 35 Beds/100 25.8 25.8 27.7
9 Number of nights per 100 inhabitants Tourism 3000 4000 Nights/100 1865 568 2603
10 Educational structure of inhabitants Social 20 30 % 11.60 12.4 15.55

Relevance to DPSIR: D – Indicators: 1 P: 3, 7, 8, 9 S: 2 I: 4 R: 5, 6, 10.

Fig. 5. Slovenian cast AMOEBA 2003. Fig. 6. Slovenian coastal AMOEBA 2003.
358 S. Bell / European Journal of Operational Research 222 (2012) 350–360

 In both cases it was invisible in the final presentation of the 1998; Munro et al., 2000; Mingers and Rosenhead, 2004). Mingers
teams, rather they chose their own organising framework and Rosenhead note:
for their indicators
‘‘In fact it is common to combine together a number of PSMs, or
PSMs together with traditional methods, in a single interven-
The following quote from CAMP Slovenia is informative:
tion – a practice known as multi-methodology’’ (Mingers and
‘‘Viewed as an entirety, the workshop participants spent most Rosenhead, 2004, p. 533).
of their time seeking an agreement on the bands of equilibrium
Whilst accepting that the debate around multi-methodology is
for the individual indicators. Opinions on the issue of which val-
wide-ranging, contested (in terms of the specific meaning, evidence
ues of the individual indicators are valid for sustainable devel-
of application and interpretation of combination of methodologies
opment were different and it was imperative to seek
to name but three strands) and deeply argued (for just one small
compromises. Although many participants had already encoun-
example of the qualtiy of this see: Jackson, 2011), from both the
tered the data and indicators in practice, most of them had not
description of Imagine and the review of the application of the indi-
had the opportunity to work together in this field. In practice
cator element of Imagine in Malta and Slovenia a case can be made
some of the indicators or studies were conducted by a smaller
that the overall approach corresponds to a PSM multi-methodology
expert group where the opinions on the indicators were uni-
which would appear to straddle the two columns in Table 1. The
form, often determined already by the person who ordered
lack of formal recognition of DPSIR in the final outcomes of the pro-
the study. In the case of the Imagine process, the working groups
jects described in this paper is an issue. In the various reports there
were not burdened by a predetermined result and the partici-
is a lack of formal recognition that DPSIR is central to problem
pant structure was relatively diverse and often with a very var-
structuring. DPSIR was often implicit within the workshop context.
ied viewpoints on the same issue. That is why the hearing of
In this sense DPSIR criteria were presented and available for use as
different opinions and the seeking for compromises in deter-
sense making heuristic devices, enabling stakeholders to make eas-
mining the indicator values was an entirely new experience to
ier selection of key indicators. This case in turn requires consider-
many’’ (Maher, 2006, p. 26).
ation of DPSIR and other frameworks for indicator application for
Both Ellul and Maher go on to describe issues and problems formal inclusion in the PSM stable.
which arose in each case. Neither project was trouble free but in
each case the results of the Imagine methodology produced inter- 4. Discussion and conclusions
esting and diverse outcomes – some of which could be sustainable.
As Ellul put it: This paper has explored the application of the Imagine method-
ology in two case study contexts set in Malta and Slovenia. The
‘‘With regard to the SPSA a number of key indicators have been
objective of the study was to explore the value of DPSIR framework
identified and these SIs will be monitored over the next two
as a discreet or part of a Problem Structuring Method which has
years to evaluate whether actions being taken in the interim
flexibility, balancing the needs for precision and at the same time
period with regard to the specific sectors are leading towards
meeting needs of stakeholder populations.
a more sustainable level of activity in the NW (North West of
In both cases versions of the Imagine methodology were ap-
the Island). Negative signals should spark action in the respec-
plied to assist communities to review their own sustainability
tive areas to reverse the trend. The SPSA team should be in a
and to produce indicators to map this. The indicator development
position to maintain this monitoring exercise and produce a
was supported by reference to the DPSIR framework but in both
short report at the end of each year with the involvement of
cases the resulting indicators, although assessable as DPSIR types,
the stakeholders’’ (Ellul, 2002, p. 60).
were framed by local people in collective terms which they se-
This would indicate that both the indicator selection exercise lected for themselves. In both cases the indicators produced by
and the project process of developing a ‘team’ around the indica- the stakeholders can be seen to link evidentially to the DPSIR cri-
tors has had some limited success in terms of working from this teria but it should be re-emphasised that the explicit allocation of
basis beyond the life cycle of the CAMP project. Indeed, in the case indicators to DPSIR is a retrospective activity undertaken for this
of Malta, the formation of a National Commission for Sustainability paper, the specific allocation of stakeholders at the time is not
could be argued to be in part at least an outcome of the project. recorded.
However, if the projects has sought to embed the explicit use of A major element for discussion is the explicit and implicit use of
the DPSIR framework into the consciousness of the stakeholder method and framework. In both cases the authors of the reports in
communities in each country then the outcome is less clear. In Malta and Slovenia include the DPSIR as part of their own consid-
both cases DPSIR ideas were included in the framing of the indica- eration of the project workshops. The workshops themselves made
tors and in the selection of the balance of indicators, ensuring that explicit use of DPSIR to assist those attending in selecting a good
they were not all of one type. However, when the organisation of range of likely sustainability indicators but, the DPSIR framework
the indicators was finally concluded, local stakeholders chose their is invisible in the final work of the stakeholders. Two contrasting
own criteria for indicator organisation. conclusions might be drawn:

3.4. Imagine – a PSM multi-methodology? 1. that the DPSIR provided a spur and focus but was then dis-
carded as stakeholders found their own, more immediate
So far, the case has been made that the Imagine methodology framing for their indicators or,
provides the potential for traditional quantitative methods and 2. that the framework was not seen as being effective beyond a
alternative PSM approaches to be combined thus allowing local, means to introduce the indicator approach.
participatory quantification of issues and emergent problem struc-
turing. Such a claim would seem to imply that Imagine has the nec- In either case, what cannot be easily dismissed is the resulting
essary qualifications to be considered a Multi-methodology. flurry of indicators and the manner in which stakeholders applied
Multi methodology has been described in various publications this form of quantitative measurement to their own context. What
(Mingers and Brocklesby, 1996; Mingers and Gill, 1997; Zhu, is also clear is that the Imagine methodology was able to include
 S. Bell / European Journal of Operational Research 222 (2012) 350–360 359

and contain both ‘soft’ and qualitative as well as quantitative Borja, A., Galparsoro, I., Solaun, O., Muxika, I., Tello, E., Uriarte, A., Valencia, V., 2006.
The European water framework directive and the DPSIR, a methodological
methods.
approach to assess the risk of failing to achieve good ecological status. Esturine
The tentative conclusion of this research is to suggest that Coastal and Shelf Science 66, 84–96.
although the DPSIR framework is still in wide use it benefits from Brenner, J., Jiménez, J.A., Sardá, R., 2008. Environmental indicators GIS of the Catalan
inclusion in multi-methodology for greater social and stakeholder Coast. Journal of Coastal Conservation 11 (4), 185–191.
Carr, E.R., Wingard, P., Yorty, S., Thompson, M., Jensen, N., Roberson, J., 2007.
use and impact. In this article it has been argued that this wider Applying DPSIR to sustainable development. International Journal of
application with DPSIR as both technical device and means to al- Sustainable Development & World Ecology 14, 543–555.
low wider stakeholders access to and engagement with the policy Casazza, G., Silvestri, C., Spada, E., 2002. The use of bio-indicators for quality
assessments of the marine environment: examples from the Mediterranean Sea.
issues at the heart of sustainable development. This, of course, Journal of Coastal Conservation 8 (2), 147–156.
needs further testing with DPSIR in rounded, multi-methodology Checkland, P.B., 1981. Systems Thinking. Systems Practice. Wiley, Chichester.
PSM mode. As Svarstad, et al. puts it: Checkland, P.B., Scholes, J., 1990. Soft Systems Methodology in Action. Wiley,
Chichester.
‘‘We do not argue that the DPSIR framework necessarily should Choi, S.H., Bae, S.M., 2009. Strategic information systems selection with incomplete
preferences: a case of a Korean Electronics Company. The Journal of the
be rejected, however. Rather, we call for further research to Operational Research Society 60 (2), 180–190.
explore the potentials for expanding the DPSIR framework, Coudert, E., 2006. Seminaire imagine: presentation et application au Plan Bleu.
and its application, so as to incorporate the mentioned social Sophia Antipolis, Plan Bleu, 18 (dissemination restricted).
Coudert, E., Larid, M., 2011. IMAGINE: A Set of Tools and Methods to Assist
and economic concerns. This implies bringing adequate atten-
Integrated Coastal Zone Managment in the Mediterranean. Sophia Antipolis,
tion not only to the state of the environment, but also to the Blue Plan UNEP/MAP Regional Activity Centre.
state of social matters. The understanding of socio-economic EEA, 1999. Environmental Indicators: Typology and Overview. Copenhagen,
European Environment Agency. Technical Report No. 25.
and cultural drivers should be broadened, and it is pivotal to
Elliott, M., 2002. The role of the DPSIR approach and conceptual models in marine
properly analyse social, economic and cultural conflicts that environmental managment: an example of offshore wind power. Marine
surround the issue in focus. Thus, there is a particular need Pollution Bulletin 44, iii–vii.
for elaboration of methodology to address attitudes and defini- Ellul, T., 2002. Final Report on the Systemic and Prospective Sustainabilty Analysis
Project within CAMP Malta. Sophia Antipolis, Blue Plan the the Ministry for
tions of the problem held by stakeholders and the general pub- Economic Services, Malta.
lic’’ (Svarstad et al., 2008, p. 123). Foxon, T., Leach, M., Butler, D., Dawes, J., Hutchinson, D., Pearson, P., Rose, D., 1999.
Useful indicators of urban sustainability: some methodological issues. Local
Environment 4 (2), 137–168.
Green, R.E., Balmford, A., Crane, P.R., Mace, G.M., Reynolds, J.D., Turner, R.K., 2005. A
framework for improved monitoring of biodiversity: responses to the world
References summit on sustainable development. Conservation Biology 19 (1), 56–65.
Gregory, A J., Atkins, J P., Burdon, D. and Elliott, M. 2012. A Problem Structuring
Ackermann, F., 1996. Participant’s perceptions on the role of facilitators using group Method for Ecosystem-Based Management: The DPSIR Modelling Process.
decision support systems. Group Decision and Negotiation 5 (1), 93–112. Unpublished manuscript.
Andranik, T., 2007. Analysis of the third European survey on working conditions Holling, C., Gunderson, L., 2002. Resilience and Adaptive Cycles. Panarchy:
with composite indicators. European Journal of Operational Research 181 (1), Understanding Transformations in Human and Natural Systems. L. Gunderson
468–499. and C. Holling. New York, Island Books.
Atkins, J., Elliot, M., Burdon, D., Gregory, A., 2011. Managment of the marine Jackson, M.C., 2011. The multi-methodology debate: a response to Harwood. Journal
environment: integrating ecosystem services and societal benefits with the of the Operational Research Society 62, 811–813.
DPSIR framework in a systems approach. Marine Pollution Bulletin 62, 215–226. Jago-on, K., Kaneko, S., Fujikura, R., Fujiwara, A., Imai, T., Matsumoto, T., Zhang, J.,
Bell, S., 2008. Creating Sustainable Communities: Student Workbook. Academy for Tanikawa, H., Tanaka, K., Lee, K.B., Taniguchi, M., 2009. Urbanization and
Sustainable Communities. subsurface environmental issues: an attempt at DPSIR model application in
Bell, S., 2011. From sustainable community to big society: ten years learning with Asian cities. Science of the Total Environment 407, 3089–3104.
the imagine approach. International Research in Geographical and Kolb, D., 1984. Experiential Learning: Experience as the Source of Learning and
Environmental Education 20 (3), 247–267. Development. Prentice-Hall, London.
Bell, S., Coudert, E., 2005. A Practioner’s Guide to ‘‘IMAGINE’’: The Systemic and Kristensen, P., 2004. The DPSIR Framework. Comprehensive Assessment of the
Prospective Sustainability Analysis – Guide d’Utilisation pour ‘‘IMAGINE’’: Vulnerability of Water Resources to Environmental Change in Africa using River
l’Analyse de Durabilité Systémique et Prospective. Sophia Antipolis, Blue Plan. Basin Approach. European Environment Agency, Nairobi, Kenya.
Bell, S., Lane, A., 2009. Creating sustainable communities: enhancing social Mace, G.M., Baillie, J.E.M., 2007. The 2010 biodiversity indicators: challenges for
mobility? Local Economy 24 (8), 646–657. science and policy. Conservation Biology 21 (6), 1406–1413.
Bell, S., Morse, S., 1999. Sustainability Indicators: Measuring the Immeasurable. Maher, I., 2006. The Systemic and Prospective Sustainability Analysis ‘Imagine’
London, Earthscan. within CAMP Slovenia. Sophia Anitpolis. Regional Development Agency, South
Bell, S., Morse, S., 2001. Breaking through the glass ceiling: who really cares about Primorska, Koper.
sustainability indicators? Local Environment 6 (3), 291–309. Maxim, L., Spangenberg, J., O’Connor, M., 2009. An analysis of risks for biodiversity
Bell, S., Morse, S., 2003. Measuring Sustainability: Learning from Doing. London, under the DPSIR framework. Ecological Economics 69, 12–23.
Earthscan. Medhurst, J., Stanton, I.M., Bird, H., Berry, A., 2009. The value of information to
Bell, S., Morse, S., 2004. Experiences with sustainability indicators and stakeholder decision makers: an experimental approach using card-based decision gaming.
participation: a case study relating to a ‘Blue Plan’ project in Malta. Sustainable The Journal of the Operational Research Society 60 (6), 747–757.
Development 12, 1–14. Mingers, J., Brocklesby, J., 1996. Mutlimethodology: towards a framework for
Bell, S., Morse, S., 2005a. Delivering sustainability therapy in sustainable critical pluralism. Systemist 18 (3), 101–131.
development projects. Journal of Environmental Management 75 (1), 37–51. Mingers, J., Gill, A. (Eds.), 1997. Multimethodology: The Theory and Practice of
Bell, S., Morse, S., 2005b. Holism and understanding sustainability. Systemic Combining Managment Science Methodologies. Wiley, Chichester.
Practice and Action Research 18 (4), 409–426. Mingers, J., Rosenhead, J., 2004. Problem structuring methods in action. European
Bell, S., Morse, S., 2007a. Problem structuring methods: theorising the benefits of Journal of Operational Research 152 (3), 530–554.
deconstructing sustainable development projects. Journal of the Operational Munro, I., Mingers, J., 2000. The Use of Multimethodology in Practice – Results of a
Research Society 58, 576–587. Survey of Practitioners. University of Warwick, Warwick.
Bell, S., Morse, S., 2007b. Story telling in sustainable development projects. Ness, B., Anderberg, S., Olsson, L., 2010. Structuring problems in sustainability
Sustainable Development 15, 97–110. science: the multi-level DPSIR framework. Geoforum 41 (3), 479–488.
Bell, S., Morse, S., 2009. Participatory Visioning of Indicator Use. International Petit, S., Firbank, L., Wyatt, B., Howard, D., 2001. MIRABEL: models for integrated review
Sustainable Development Research Conference, Utrecht. and assessment of biodiversity in European landscapes. Ambio 30 (2), 81–88.
Bell, S., Morse, S., 2010. Rich Pictures: A Means to Explore the ‘Sustainable Mind’? Rosenhead, J., Mingers, J. (Eds.), 2001. Rational Analysis for a Problematic World –
Sustainable Development Wiley Online. Revisited. Wiley and Sons, Chichester.
Bell, S., Walker, S., 2010. Futurescaping infinite bandwidth, zero latency. Futures 43, Simon, O., 2004. Evaluation of mountain case studies by means of sustainability
525–539. variables: a DPSIR model as an evaluation tool in the context of the north–south
Bellotti, T., Crook, J., 2009. Credit scoring with macroeconomic variables using discussion. Mountain Research and Development 24 (4), 336–341.
survival analysis. The Journal of the Operational Research Society 60 (12), Smeets, E., Weterings, R., 1999. Environmental Indicators: Typology and overview.
1699–1707. European Environment Agency, Copenhagen.
Benjaminsen, T.A., Svarstad, H., 2008. Understanding traditionalist opposition to Stem, C., Margoluis, R., Salafsky, N., Brown, M., 2005. Monitoring and evaluation in
modernization: narrative production in a Norwegian Mountain conflict. conservation: a review of trends and approaches. Conservation Biology 19 (2),
Geografiska Annaler. Series B, Human Geography 90 (1), 49–62. 295–309.
360 S. Bell / European Journal of Operational Research 222 (2012) 350–360

Svarstad, H., Petersen, L., Rothman, D., Siepel, H., Watzold, F., 2008. Discursive biases Wiek, A., Withycombe, L., Redman, C., 2011. Key competencies in sustainability – a
of the environmental research framework DPSIR. Land Use Policy 25, 116–125. reference framework for academic program development. Sustainability
Ülengin, F., Kabak, Ö., Önsel, Sß., Ülengin, B., Aktasß, E., 2010. A problem-structuring Science 6, 203–218.
model for analyzing transportation–environment relationships. European Zhu, Z., 1998. Conscious mind, forgetting mind: two approaches in
Journal of Operational Research 200 (3), 844–859. multimethodology. Systemic Practice and Action Research 11 (6), 669–690.