Environmental Science & Policy 54 (2015) 142–151

Contents lists available at ScienceDirect

Environmental Science & Policy

journal homepage: www.elsevier.com/locate/envsci

Measuring the sustainability of urban water services

Rui Cunha Marques a,*, Nuno Ferreira da Cruz b, João Pires c
a
CESUR – Center for Urban and Regional Systems, CEris, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisbon, Portugal
b
LSE Cities, London School of Economics and Political Science, Houghton Street, London WC2A 2AE, United Kingdom
c
Catholic University of Lisbon, Travessa Palma, 1649-023 Lisbon, Portugal

A R T I C L E I N F O A B S T R A C T

Article history: This paper discusses the concept of ‘sustainable water services’ and suggests a multicriteria method to
Received 15 April 2015 assess it. Although conceptual discussions around this notion are often confined to the triple bottom line
Received in revised form 2 July 2015 (TBL) classification, it seems that the TBL approach does not provide the suitable framework to measure
Accepted 3 July 2015
water services sustainability. It is argued that assets (or technical) and governance aspects are also
Available online 18 July 2015
indispensable dimensions. After revisiting the concept in broader terms, several criteria and metrics are
suggested to operationalize and quantify the sustainability level of urban water services. To aggregate
Keywords:
the numerous aspects that are relevant in this scope a multicriteria decision analysis approach is
Drinking water
proposed. Furthermore, to illustrate the real-world application of the method, a multicriteria model
Indicators
Multi-criteria decision analysis applicable to the case of Portugal was developed and calibrated with the input of a decision-maker with
Sustainability extensive experience in the sector. With the suggested framework it is possible to assess the global
Wastewater sustainability level of the water services (e.g. of each utility) and also the performance in each particular
dimension (‘social’, ‘environmental’, ‘economic’, ‘governance’ and ‘assets’).
ß 2015 Elsevier Ltd. All rights reserved.

1. Introduction Taking into account the theoretical and practical difficulties,

this paper proposes a framework and a method to assess the
Although some recent efforts may be found in the literature (e.g. sustainability of UWS. To accomplish this, the research strategy
van der Steen, 2011; van Leeuwen et al., 2011), there is no widely encompasses three major steps: (1) the dimensions and objectives
accepted or established method to assess the sustainability level of of UWS sustainability are defined along with the respective
urban water services (UWS). In fact, sustainability assessments face assessment criteria; (2) the quantitative and qualitative perfor-
many obstacles. At the outset, the very definition of sustainability mance metrics are identified for each criterion; (3) a multicriteria
poses significant conceptual challenges. How can one operationalize decision analysis (MCDA) model is structured taking into account
such a wide-ranging notion? However, although there are many the Portuguese context (for illustrative purposes). Most of the
challenges towards achieving the sustainable urban water cycle conceptual work reported in this study (mainly corresponding to
(Brown et al., 2009), this pursuit is a major societal objective which the abovementioned steps 1 and 2) was developed within the
therefore makes measurement important. TRansitions to the Urban Water Services of Tomorrow (TRUST)
Sustainability is usually associated with the triple bottom line research project. The current operationalization of the multi-
(TBL) framework, composed of social, environmental and econom- criteria model (step 3) was carried out through a participatory
ic dimensions or principles (Thornton et al., 2007). These process involving an expert who acted as the decision-maker in the
dimensions find correspondence in the ‘people, planet and profit’ modelling process (a former Director of EPAL,1 member of the
phrase of Shell or to the ‘folk, place and work’ of the planner Patrick board of ERSAR2 and current Executive Director of PPA3).
Geddes in his definition of sustainability. The question is whether
the TBL framework is suitable to deal with UWS sustainability. We,
as some other authors, believe that the TBL approach is not 1
EPAL (Empresa Portuguesa das Águas Livres) is a major water utility serving the
sufficient in this regard (ASCE and UNESCO, 1998 or Ashley et al., region of Lisbon (the capital of Portugal).
2003). 2
ERSAR (The Water and Waste Regulation Authority) is the Portuguese sector-
specific regulator.
3
PPA (Portuguese Water Partnership) is an association that ‘aims to develop
synergies and maximize potential for the development of the water sector in the
world, promoting the construction and consolidation of alliances and partnerships
* Corresponding author. between national institutions and all nations engaged in sustainable water use and
E-mail address: [email protected] (R.C. Marques). enhancement of water resources.’

http://dx.doi.org/10.1016/j.envsci.2015.07.003
1462-9011/ß 2015 Elsevier Ltd. All rights reserved.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 143

After this brief introduction the second section reviews some of assessment of the indicators and criteria were identified by these
the previous efforts to assess the sustainability level of urban water authors, particularly the clear definition of objectives and
services. In the third section we discuss the concept of operational objectives, the concerted and multidisciplinary mea-
sustainability and argue for the addition of the ‘assets’ and surements and the quality of the metrology.
‘governance’ dimensions to the TBL framework. The fourth section Mitchell (2006) studied the concept of ‘integrated urban water
demonstrates the suitability of MCDA modelling regarding the management’ (IUWM) as a central pillar of sustainability. A full
objective of assessing water services sustainability whereas the process of urban water services, whose components of an
fifth section presents a real application of the procedure. Finally, integrated ‘physical system’ are the drinking water supply and
the sixth section concludes the paper. the wastewater collection systems, also comprises the organiza-
tional framework and the surrounding environment. It was shown
2. Sustainability assessments of urban water services that it was possible to successfully put into effect the IUWM
approach, while being technically reasonable and acceptable to
Sustainability is a trendy concept that is used in different stakeholders. Some of the available IUWM tools that must be
contexts and with several aims. Sustainable practices are currently combined to result in the integrated total system solutions
an objective that most organizations pursue; however, the required by urban communities are water recycling, water
question is often to know about what is not sustainable rather efficiency programmes, and water sensitive stormwater manage-
than what is sustainable (AWWARF and CSIRO, 2007). Considering ment. According to this author, the IUWM approach considers that:
the various stakeholders, the uncertainties, the numerous possible (a) all components of the water cycle make part of an integrated
metrics and the trade-offs between them, it is difficult to know system; (b) all dimensions of sustainability are well-adjusted; (c)
how to become sustainable and to measure to what extent a all stakeholders are involved, including all water users; (d) all
particular sector or activity is sustainable. water uses are addressed and (e) all idiosyncrasies of the local
Nowadays, the success of a water (and wastewater) utility context are taken into account.
depends on the provision of drinking water supply or the collection A typology of five organizational development phases in
of wastewater with the adequate quality levels but it also needs to sustainable urban water management (including the project,
take into account the impact of its actions and decisions on people, outsider, growth, insider, and integrated development phases) was
places and related resources, both in the short and long-term. proposed by Brown (2008) as a heuristic model and/or a capacity
Therefore, along with the economic performance, water utilities benchmarking tool for stakeholders. The results proved that the
are now implementing new approaches to evaluate themselves. political institutionalization of environmental matters along with
Some of these approaches are directly related to sustainability the commitment to local leadership and organizational learning
assessment through the TBL lenses, centred on financial, environ- ‘are key corporate attributes for enabling sustainable manage-
mental and social performances (although other sustainability ment’.
scorecards embrace further dimensions). In the UK, the industry umbrella in the water sector built a set of
The Swedish Foundation for Strategic Environment Research is 25 indicators to measure the water utilities progress concerning
responsible for one of the first studies referred to in the literature. environmental sustainability (Water UK, 2000). Since then this
The project adopted the metabolism model and ‘health and organization has been computing a set of sustainable water
hygiene’, ‘social and cultural’, ‘environmental’, ‘economic’, ‘func- indicators yearly (see Water UK, 2010). Afterwards, a research
tional’, and ‘technical’ categories where the criteria adopted for project known as SWARD produced a sustainable water services
assessing UWS sustainability were arranged according to their procedural guide (Ashley et al., 2003, 2004). The aim of this tool is
influence on the water and wastewater systems (Hellström et al., to help water utilities to make sustainable decisions. It makes a
2000). At that time the authors concluded that the tools to evaluate distinction between principles, criteria and indicators of sustain-
the socio-cultural and functional aspects were not suitable. The ability and sorts the criteria (or dimensions) into economic,
criteria used in the water sector were also analysed by Balkema environmental, social and technical and each one of these into
et al. (2002) in 15 publications to compare technologies relating to performance indicators.
sustainability and concluded that adequate attention was not In the U.S., Monsma et al. (2009) put emphasis on the role of
given to the socio and cultural aspects. integrated water resources management to improve the sustain-
A set of sustainability indicators was developed by Lundin ability in water systems. The sustainable path for the U.S. water
(2003) to assess progress of water and wastewater systems as far infrastructure found support in a set of 20 performance metrics
as sustainability is concerned. Those indicators comprise environ- which includes the following components: affordability, advanced
mental and technical factors and their choice took into account a procurement and project delivery methods, asset management,
number of principles. The International Hydrological Programme climate change mitigations and adaptation, conservation and
alongside the American Society of Civil Engineers (ASCE and water efficiency, costs of development, energy management,
UNESCO, 1998) defined criteria and set guidelines for the environmental impacts, full cost pricing, good governance,
sustainability of water resource systems. By including economic, modernized plant operations, network optimization, public
environmental, ecological, social and physical objectives the outreach and stakeholder investment, regulatory optimization,
authors acknowledged the multidisciplinary, multi-objective and security and emergency preparedness, stewardship, transparency,
multi-participatory characteristics of water resource sustainability. watershed and regional optimization, workforce management and
In view of the need for new actions to achieve sustainable research, and technological and managerial innovation.
management of the urban water systems, Bertrand-Krajewski et al. Recently, an indicator approach called City Blueprint was
(2000) came across methodological issues related to definition of developed by the KWR Watercycle Research Institute and Deltares
objectives, modelling, decision-making tools, metrology and to assess the sustainability of the urban water cycle. The City
multidisciplinarity. According to the authors these are essential Blueprint includes elements of water footprints, urban metabo-
conditions to develop the knowledge on the indicators and criteria lism, ecosystem services and indicators (van Leeuwen et al., 2011,
used in the methodologies of sustainability assessment. Two 2012; van Leeuwen and Chandy, 2013). The research uses
integrated approaches were followed: one related to time and 24 indicators from eight broad categories, namely (1) water
space scales and the other in the scope of multidisciplinarity. Three security following the water footprint approach as developed by
groups of methodological problems that get in the way of a suitable Hoekstra (2003), (2) water quality, which includes surface and
144 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151

groundwater, (3) drinking water, (4) sanitation, (5) infrastructure, Other relevant case-studies are found in the U.S. (e.g. Seattle
(6) climate robustness, (7) biodiversity and attractiveness and (8) and San Francisco Public Utilities), the UK (e.g. the above-
governance. This approach has been applied in different Dutch mentioned Water UK efforts), Canada (e.g. the cities of Hamilton
cities by means of an interactive multi-stakeholder approach and Toronto), Singapore (Public Utilities Board), Philippines (e.g.
which includes relevant stakeholders. The results are disclosed in Manila Water), Israel (Tel-Aviv-Yafo urban water system) and in
‘spider web’ graphs (equivalent to a scorecard). Meanwhile van some European countries (e.g. EPAL in Portugal). All these case-
Leeuwen and his TRUST colleagues applied this methodology in studies were considered while developing the conceptual model
other cities (van Leeuwen and Chandy, 2013). Because the presented in the following sections (dimensions, objectives,
indicators are not aggregated, this tool does not allow for criteria and metrics). However, it comes as no surprise that
estimating the overall sustainability level of UWS. sustainability is not yet a priority in most developing countries.
The ISO 24.500 also expresses a concern about the sustainability
of water and wastewater services. The ISO 24.500 series makes 3. The concept of sustainability
reference to the ‘sustainability of water and wastewater utilities’
and to a ‘sustainable development of the community’, and provides In the late 20th century the prevailing paradigm of develop-
the definition of performance indicators for service improvement. ment was questioned. People became aware of the relationship
Therefore, the utilities must ‘address sustainable development, i.e. between the development of the economy and environmental
the ability for the community to grow and prosper within the preservation and understood that only if environment protection
environmental, infrastructural and economic resources available, and social inclusion were addressed could economic progress be
without limiting the use of those resources by future generations.’ sustained. These ideas became associated with the concept of
Furthermore, the ISO 24.500 argues that the utilities should (a) sustainable development. There is a wide range of definitions and
contribute to and implement sustainable water resources man- meanings of sustainability for different people. The definition of
agement policies and practices, (b) contribute to development ‘Our Common Future’, also known as the Brundtland Report, from
planning and resource allocation through consultation, provision the United Nations World Commission on Environment and
of information and analysis in conjunction with appropriate Development (WCED, 1987, p. 43) is one of the most used. The
institutions, (c) contribute to public health and safety, and (d) report defines sustainability as the ‘development that meets the
implement information and education of the community on these needs of current generations without compromising the ability of
topics, particularly on the efficient use of water and pollution future generations to meet their needs and aspirations’. In a
prevention. somewhat different way, the International Union for the Conser-
Finally, a comprehensive sustainability reporting framework vation of Nature defines sustainability as the ‘development that
produced by the Global Reporting Initiative (GRI) became improves the quality of human life while living within the carrying
important for the water utilities worldwide since it allows utilities capacity of supporting ecosystems’ (IUCN, 1991, p. 10). Although
to manage their impact on sustainability development, showing the latter definition deals explicitly with the environment, both of
them how to exert positive change on the state of the economy, them focus on intergenerational equity.
and on environmental and social conditions. This reporting In the scope of UWS, sustainability was firstly associated with
framework was also well accepted due to its capability to measure, ‘those water resource systems designed and managed to fully
track, and improve utilities’ performance on specific issues; and, contribute to the objectives of society, now and in the future, while
therefore, to diminish potential business risks (GRI, 2011). maintaining their ecological, environmental and hydrological
Furthermore, GRI helps the utilities and other organizations to integrity’ (ASCE and UNESCO, 1998). Although the ‘objectives of
manage and promote transparency and accountability because all society’ may also include the economic and social aspects, this
the information is freely available. definition clearly emphasizes the environmental dimension of
Although sustainability scorecards were rapidly adopted UWS. Indeed, UWS have made significant progress over time. A few
among water utilities and UWS in some countries, examples of decades ago, water quantity, drinking quality and adequate
this type of assessment are still scarce. The Australian utilities are pressure were sufficient requirements for a suitable drinking
notable exceptions with a pioneering position in this scope. For water service. Although necessary, today these aspects are far from
example, the Sydney Water Company uses the ‘sustainability being sufficient requirements. UWS now must be efficient,
scorecard’ which includes social, economic and environmental effective and customer-responsive. In addition, water utilities
concerns among its objectives. The yearly published sustainability recognize their role of organizations with a corporate social
scorecard comprises the company’s sustainability performance responsibility that therefore must invest on the community.
measurement. It measures performance through sustainability The impact that the use of water resources for drinking water
indicators and defines objectives for the metrics determined. The might have on the water footprint can be small (Berrittella et al.,
sustainability scorecard encompasses summary statements and 2007, 2008a). Nevertheless, serious consequences for the daily life
progress ratings based on management evaluation of Sydney of residential and industry customers might come up from other
Water’s performance against its sustainability indicators and water uses (e.g. agriculture or industry), water pollution or even
includes a set of performance indicators aimed at serving the lack of drinking water supply. Thus, the environment issues
customers (three indicators), maintaining a water efficient city related to the UWS are not the only ones that matter and they are
(4), providing clean and safe drinking water (2), contributing to also very relevant in other dimensions. They must be considered in
clean beaches, oceans, rivers and harbours (4), optimizing resource an integrated way looking for the IUWM and the life-cycle costs of
use (5), developing a safe, capable and committed workforce (5), the systems (Kluge, 2007). The many actors and stakeholders
and being an economically efficient business (4). Furthermore, in belonging to several areas with various objectives and interests
New South Wales (NSW) the NSW Office of Water and IPART make governance issues (e.g. participation and transparency) very
(water regulator) use a TBL approach for each water utility so as to important in this scope as well. Moreover, the increasing cost of
provide a balanced view of the long-term sustainability. In the TBL water (and wastewater) services, the level of investment that is
report there is a summary of the utilities’ compliance with the necessary and the need to reflect the costs into the polluter and
requirements of the ‘best-practice guidelines and its performance user pay principles, along with the economic and social dimensions
for over 50 key performance indicators together with the state- are becoming more and more fundamental issues. UWS are central
wide medians.’ for the social and economic cohesion of society. Not only does the
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 145

Table 1
Dimensions related to sustainability identified in the literature.

Authors Dimensions/Criteria

Spangenberg and Bonniot (1998) Economic; environmental; social; human rights; society
Hellström et al. (2000) Health and hygiene; social and cultural; environmental; economic; functional; technical
Hiessl et al. (2000) Social; economic; ecologic
UNCSD (2001) Environment; social; economic; institutional
Balkema et al. (2002) Economic; environmental; technical; social; cultural
Foxon et al. (2002) Economic; ecological; social; technical
GRI (2006) Economic; environment; social; human rights; society
Van Leeuwen et al. (2011) Water security; water quality; drinking water; sanitation; infrastructure; climate robustness; biodiversity
and attractiveness; and governance
Adapted from Marques (2012).

population wish to have sound and transparent drinking water quality of the accountability and adjustment mechanisms (da Cruz
services at affordable prices but the customers also need to and Marques, 2013b).
feel the value for money spent. The awareness that the Fig. 1 highlights the concept of UWS sustainability and its
sustainability of communities and cities (the so-called ‘city of dimensions (Marques, 2012; Brattebø et al., 2013). If one assumes
the future’) calls for the sustainability of urban infrastructure that the TBL is the skeleton of UWS, a common area will correspond
and particularly of UWS is also increasing. Water services are a to the ‘assets’ and ‘governance’ dimensions (without which the
crucial element of the livability and sustainability of cities objectives of the TBL dimensions will not be reached). Hence,
(Binney et al., 2010). sustainable development in UWS needs these dimensions to be
As mentioned above, the sustainability concept is often considered as instrumental to achieve sustainability.
confined to the TBL dimensions which can be taken as aggregated Taking into account the literature review and the expertise of a
objectives concerning a particular sector that should be developed. panel of experts (from the TRUST research project), the dimen-
Some authors call ‘criteria’ to these dimensions which find sions, objectives and criteria of UWS sustainability were defined as
correspondence in the ‘set of factors that may be used to assess shown in Table 2 (adapted from Brattebø et al., 2013). The metrics
which of a range of options offers the greatest contribution to that will allow to measure performance in each criterion were
achieving sustainability objectives’ (Ashley et al., 2004). But more deliberately not defined in detail by the TRUST team (Marques,
than discussing terminology issues the main question is whether 2012). The suitable qualitative or quantitative descriptors should
the TBL approach is the most suitable to deal with UWS be sought with the aid of the relevant decision-makers in each city,
sustainability. Many authors, such as Ashley et al. (2004), disagree region or country (depending on the scope of the assessment)
with this notion and propose the technical, economic, environ- through structured participatory methods. However, some sugges-
mental and social dimensions. Murray et al. (2009) suggest the tions or examples of how the criteria can be operationalized were
economic, ecological, social, technical and human health dimen- also developed (Marques and van Leeuwen, 2012). In sum, in the
sions and Sahely et al. (2005) sort them out into environment, current study UWS sustainability encompasses five dimensions
economic, engineering and social dimensions. Also the study of that may be further divided in 14 objectives. These objectives are
ASCE and UNESCO (1998) suggests the socio-economic, environ- achieved if certain targets are met for several criteria.
mental, public health and management dimensions as the major
ones. Table 1 illustrates the UWS sustainability dimensions 4. Multicriteria modelling
suggested by other authors in the literature (adapted from
Marques, 2012). 4.1. Methodological approach
The ‘social’ dimension of UWS sustainability should take into
account the access to the services, the satisfaction of the users’ MCDA modelling is especially suitable to assess the sustain-
needs and expectations, the public acceptance and the relevant ability of UWS. Sound multicriteria models are able to consider the
role in the community of these services (Fleming, 2008). The many aspects of sustainability and, perhaps more importantly, the
‘environmental’ dimension is related to the impact of UWS on distinctiveness of each UWS with its own group of stakeholders,
living and non-living natural systems and includes the optimiza- their preferences and objectives and its particular local (regional or
tion of the use of water, energy and materials and the minimization
of the downstream negative impacts. The ‘economic’ dimension
encompasses all the objectives related to economic and financial
matters (such as the full cost recovery, da Cruz and Marques,
2013a).
However, some other crucial aspects of UWS sustainability are
still missing in this framework. The aspects related to the ‘assets’
sustainability are clearly out of the TBL dimensions and the ones
concerning ‘governance’, despite being included in the three
dimensions of the TBL, are also not specifically detailed. The ‘asset’
dimension is associated with the system of physical infrastructure
and might hold aspects related to the performance of the system,
its durability, reliability, flexibility and adaptability (Ashley et al.,
2002, 2003). Governance is related to transparency, the rules of the
game, the respect for those rules by the various stakeholders, their
participation in the decision-making process (particularly the
customers), the existence and alignment of city planning with the Fig. 1. Dimensions of sustainability.
UWS, the effectiveness and efficiency of the decisions and the Source: Brattebø et al. (2013).
146 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151

Table 2
Dimensions, objectives and criteria of the UWS sustainability.

Dimension Objectives Criteria

Social (a) Access to urban water services (a1) Physical service accessibility
(a2) Economic service accessibility
(b) Effectively satisfy the current users’ needs and expectations (b1) Quality of service
(b2) Drinking water quality
(c) Acceptance and awareness of UWCS (c1) Willingness to pay
(c2) Complaining
(c3) Acceptance of new sources of water
(d) Relevant role in community (d1) Social responsibility
(d2) Work conditions
Environment (e) Optimize the use of water, energy and materials (e1) Efficient use of water
(e2) Energy use
(e3) Material use
(e4) Final uses of efficiency
(f) Minimize downstream negative impacts (f1) Pollution prevention
(f2) Pollution control
Economic (g) Ensure economic sustainability of the UWCS (g1) Investment
(g2) Efficiency
(g3) Leverage
(g4) Liquidity
Governance (h) Public participation (h1) Participation initiatives
(i) Transparency (i1) Availability of information and documents
(i2) Accessible information and written documents
(i3) Public disclosure
(j) Accountability (j1) Individual mechanisms of accountability
(j2) Collective mechanisms of accountability
(k) Clearness, steadiness and measurability of the UWCS policies (k1) Clearness of policies defined ex-ante
(k2) Change of policies
(k3) Implementation of policies
(l) Existence and alignment of city planning (l1) Corporate planning
(l2) City planning
(l3) Water resources planning
Infrastructure (m) Performance (m1) Failures
(n) Robustness (n1) Flexibility
(n2) Adaptability
(n3) Reliability
Adapted from Brattebø et al. (2013).

national) conditions. Indeed, MCDA models have been commonly be used to calculate the sustainability score of each UWS (see
used in the water sector since the pioneering research on UWS Eq. (1)).
sustainability (e.g. Ashley et al., 1999; Icke et al., 1999; Srinivasa-
X
n X
Raju et al., 2000). Moreover, they have been used in other jSðui Þ ¼ C j  S j ðu j Þ with C j > 0 and Cj
j
applications in this sector (see, e.g. Hajkowicz and Collins, 2007; j¼1
Huang et al., 2011) as well as in others related to the environmental 
S j ð‘good’ j Þ ¼ 100
issues (Berrittella et al., 2008b). The current study proposes a ¼ 1 and (1)
S j ð‘neutral’ j Þ ¼ 0
MCDA approach to evaluate the sustainability of UWS that involves
the consideration of sustainability attributes (and the respective Where, S(ui) is the global sustainability score of UWS ui, cj is the
metrics) for the abovementioned five dimensions of UWS weighting coefficient of criterion j, Sj(ui) is the local score of UWS ui
sustainability. MCDA allows for aggregating the performances in considering criterion j, and ‘good’j and ‘neutral’j are the reference
all of those attributes by applying weighting techniques that reflect levels of performance on criterion j (corresponding to what is
the legitimate stakeholders’ preferences. considered by the decision-maker as being a good practice and
Using a multicriteria model to assess UWS sustainability has satisfactory or acceptable in terms of sustainability, respectively).
several advantages, for instance (da Cruz and Marques, 2013b): (1) For each modelling problem, the input of the suitable decision-
all types of attributes (criteria) and respective metrics (qualitative maker will be crucial to determine and calibrate each one of these
or quantitative) can be included; (2) the several objectives, criteria, parameters.
scores and weighting coefficients are unambiguous and transpar-
ent, allowing for open and informed discussion; (3) the modelling 4.2. Criteria set
process is participatory and traceable, which facilitates communi-
cating the results and auditing and reviewing the model; (4) it is Assessing UWS sustainability is a problem that involves the
possible to calculate partial (e.g. in a single dimension of consideration of several features (e.g. dimension, objectives,
sustainability) and global (taking into account the all criteria attributes, consequences, constraints. . .). To use an additive
from all dimensions) scores, which increases the informative value evaluation model such as the one represented by Eq. (1), the
of the model and can be useful for policy-making (Larson et al., criteria that represent the fundamental points of view (or
2009); (5) the procedures to ascribe scores and weights are attributes) must respect certain properties (da Cruz and Marques,
theoretically robust. 2013b). Above all, the criteria set must be exhaustive. That is,
After the designation of the legitimate decision-maker (or considering that the assessment model should be as simple and
decision-making group) who will provide his/her judgments concise as possible (the goal is to transform a complex problem in a
regarding the relative contribution of scoring in each criterion simpler problem), it must also be complete, in the sense that all key
for overall sustainability, a simple additive aggregation model can aspects concerning UWS sustainability ought to be included.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 147

Furthermore, the criteria must be preferentially independent (i.e. opinion of the decision maker (see Section 5). The final criteria are
the performance in one criterion must not influence or be usually represented in graphs known as ‘value trees’ to organize
contingent upon the performance in any other criterion). Including the various attributes according to a logical framework (Fig. 2
different criteria that ultimately will ‘measure’ the same phenom- shows the value tree of our illustrative model).
ena should also be avoided since redundancy in the model may
lead to the overvaluation of some aspects. 4.3. Performance descriptors
As we have seen, a panel of experts from the TRUST research
project proposed the criteria set shown in Table 2 to operationalize Each criterion will have one performance descriptor that
and measure UWS sustainability. In a real-world development of a operationalizes it. A performance descriptor is an ordered set of
MCDA model these criteria may be adapted taking into account the plausible impact levels for a criterion (i.e. an indicator, performance

Fig. 2. Dimensions, objectives and criteria of UWCS sustainability (M-MACBETH software).

148 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151

categories, etc.) that allows us to measure the degree to which the performance descriptor suggested by the conceptual work
objectives are being attained (da Cruz and Marques, 2013a). developed within the TRUST R&D project was thoroughly analysed
Descriptors can be quantitative or qualitative. Either way they by the decision-maker. Several adaptations were necessary so that
must respect some properties to correctly describe the performance the MCDA model would adhere to the preferences of the decision-
of the UWS in each criterion, namely they must describe the maker and therefore some criteria were deleted and some new
performance of the UWS in each criterion as objectively as possible items were also created. The final criteria are shown in Fig. 2. The
and preserve the independence of the criteria. respective performance descriptors are available in the Appendix
to this paper.
4.4. Scoring functions After pinpointing the fundamental criteria (the basis of any
additive evaluation model) we asked the decision-maker to define
In MCDA models there are two evaluation stages. For instance, the upper (‘good’) and lower (‘neutral’) reference levels in terms of
in our case: (a) the local (or partial) assessment of UWS performance for each criterion (see Table 3). This operationalizes the
sustainability in each criterion and (b) the global assessment of idea of a ‘good performance’ (the level above which are the
UWCS sustainability taking into account all criteria (i.e. the sustainability best practices) and a ‘neutral performance’ (the level
aggregation of the partial scores). During the first stage a scoring below which performance is perceived to be insufficient or
function will have to be determined for each criterion. Indeed, with substandard). As a simplification (and to expedite the decision
the descriptors an analyst is able to measure the performance or meetings), this model assumes a linear relationship between
impact of the UWS in each criterion (e.g. in Euros, number of performance and score (i.e. linear scoring functions). This simplifi-
customers, qualitative level or any other unit, depending on the cation was discussed with the decision-maker who concluded to be
selected descriptor). But he/she will need a scoring function to perfectly reasonable to assume liner scoring functions considering
transform the impacts into scores (cardinal values in predefined the performance descriptors used in the model.
scales). These scores represent the attractiveness of the impacts
according to the preferences of the decision-maker. The shape of 5.2. Computing the weights
the scoring functions is important because for some criteria the
marginal increase in the score may vary with the impact ranges As we have seen, weights must be computed taking into
(i.e. a linear relationship between performance and score may not account the criteria impact scales. The weighting coefficient of a
always be admissible). There are several methods available for criterion reflects the increase in the global score associated with
estimating non-linear scoring functions, for example, ‘direct-rating’ a swing in that criterion between the ‘neutral’ level and the
or the ‘bisection method’ (von Winterfeldt and Edwards, 1986). ‘good’ level (da Cruz and Marques, 2013a,b). To compute the
weights through a transparent and participatory procedure the
4.5. Weighting coefficients MACBETH approach was adopted (Bana e Costa et al., 2003). This
approach uses pairwise comparisons of performance swings in
Weighting coefficients are ‘scaling constants’ that transform the criteria (i.e. qualitative judgments made by the decision-
partial scores (in each criterion) in global scores. Assuming that maker when comparing two criteria at a time) to determine the
weights reflect the intrinsic ‘importance’ of the criteria for the weights. To model the differences in preference the decision-
overall score is ‘the most common critical mistake’ of MCDA maker may use the seven MACBETH categories for the
modelling (Keeney, 1992). In fact, weighting coefficients represent judgments, namely: ‘no’, ‘very weak’, ‘weak’, ‘moderate’, ‘strong’,
the importance that the decision-maker gives to the swings in ‘very strong’ or ‘extreme’ difference. The following protocol was
performance (e.g. from ‘neutral’ to ‘good’) in each criterion. In other used in each decision meetings (i.e. for each UWS sustainability
words, the ‘importance’ is contingent upon the performance range dimension):
in each criterion. Consider a number of ‘hypothetical’ UWS (one UWS for each
In compensatory models (such as the one shown in Eq. (1)), criterion) each one of them ‘neutral’ in all criteria except in one for
weights operationalize the notion of ‘trade-off’ (i.e. what perfor- which the performance is ‘good’ (all alternatives would have a
mance the decision-maker considers to be necessary to attain in ‘good’ performance in a different criterion).
one criterion to compensate a poor performance in another Rank these ‘hypothetical’ UWS according in order of preference
criterion). Once again, there are several methods available for (from the more sustainable to the least sustainable).
weight elicitation such as ‘swing weighting’ or the ‘trade-off’ Compare these UWS in terms of preference by providing
procedure (Greco et al., 2010). There are also non-numerical qualitative judgments using the seven MACBETH categories.
techniques such as the MACBETH approach (Bana e Costa et al., Fig. 3 displays the results of these protocols for the ‘assets’
2003) with which it is possible to determine the weights by asking dimension (the procedure was repeated for every dimension). As
the decision-maker to carry out pairwise comparisons through the matrix below shows, there is no need to make all possible
qualitative judgments of the differences in preference of certain pairwise comparisons. During the decision meetings only conse-
reference profiles (see Section 5). cutive criteria was compared. The M-MACBETH assists the
modelling process and automatically provides the weights that
5. Real application: a working model for Portugal are coherent with the judgments of the decision-maker (through
linear programming algorithm, see Bana e Costa et al., 2003).
5.1. Taking into account the preferences of the decision-maker The matrix of judgments displayed in Fig. 3 allows for
computing the intra-weights of the ‘asset dimension’. To
To illustrate the application of the MCDA model, we discussed compute the global weights (so one can determine de global
the criteria and the descriptors of the sustainability dimensions sustainability score of a given UWS) it was necessary to
with a decision-maker with extensive experience in UWS normalize all the intra-weights. This was accomplished by
management in Portugal and abroad (the third author of this selecting one criterion from each of the five dimensions (the
study). These structured discussions were held in five decisions criteria with the highest intra-weights were arbitrarily chosen)
meetings (one meeting per sustainability dimension where in the and repeating the protocol described above (see Fig. 4). With this
last meeting the procedure that allows for the normalization of all final matrix of judgments we were able to normalize all weights
the weights was also carried out). Each criterion and respective through linear transformations.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 149

Table 3
‘Neutral’ and ‘Good’ reference levels according to the decision-maker’s preferences.

Code Criteria ‘Good’ reference level ‘Neutral’ reference level

A1 Physical service accessibility Level II Level IIII

A2 Economic service accessibility 1.5% 3.0%
A3 Quality of service Level II Level IIII
A4 Drinking water quality 99.0% 97.5%
A5 Willingness to pay (WTP) Level II Level IIII
A6 Complaining 1 3
A7 Acceptance of new sources of water 90.0% 80.0%
A8 Social responsibility 0.50% 0.25%
A9 Work conditions Level II Level IIII
B1 Efficient use of water Level II Level IIII
B2 Energy use Level II Level IIII
B3 Material use 70% 50%
B4 Final uses of efficiency Level II Level IIII
B5 Pollution prevention and control Level II Level IIII
C1 Investment Level II Level IIII
C2 Efficiency Level II Level IIII
C3 Leverage 2.5 4.0
C4 Liquidity 1.0 0.5
D1 Participation initiatives Level III Level IV
D2 Existence of information and its documentation Level II Level IV
D3 Access to information Level II Level IV
D4 Individual mechanisms of accountability Level II Level IV
D5 Collective mechanisms of accountability Level II Level IV
D6 Clearness and stability of policies Level II Level IV
D7 Strategy and alignment with city planning Level II Level IIII
E1 Failures Level II Level IIII
E2 Flexibility Level II Level IIII
E3 Adaptability Level II Level IIII
E4 Reliability Level II Level IIII

Fig. 3. Matrix of judgments for the ‘assets’ dimension (M-MACBETH software).

Fig. 4. Matrix of judgments for the hierarchical model (M-MACBETH software).

5.3. Results and discussion determine the overall sustainability level any UWS (e.g. a utility)
and also how it is faring in each dimension (provided that the data
Fig. 5 shows the results (i.e. the weights) for all the criteria for the selected metrics are available).
according to preferences of the decision-maker. With the criteria The definition used for the establishment of the ‘good’ and
set (see Fig. 2 and Table 3), the performance descriptors (see ‘neutral’ levels of reference in each criterion associates an
Appendix) and having all the weights (Fig. 5), it is possible to empirical meaning to the global score attained by any UWS.
150 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151

Fig. 5. Weights of the UWCS sustainability criteria.

Indeed, in face of a sustainability score attained through the model assessments. And third, with a practical example of how a model
described above, one is immediately able to categorize the UWS as was structured considering the Portuguese reality (although to
follows: increase the legitimacy of the model the decision meetings
should have involved more stakeholders with responsibilities in
 Highly sustainable UWS – if it has a score above 100 (i.e. more the water sector and even customer representatives and other
sustainable than a UWS that has a performance equal to the actors), the current research provides a useful roadmap for
‘good’ level in all criteria). MCDA modelling through participatory procedures. Possible
 Sustainable UWS – if it has a score between 0 and 100. follow-up research might involve the calibration of this model
 Unsustainable UWS – if it has a score below 0 (i.e. less sustainable and its application to a group of Portuguese water and
than a UWS that has a performance equal to the ‘neutral’ level in wastewater utilities.
all criteria).
Acknowledgements
Finally, it should be noted that to carry out a global evaluation
with an additive aggregation model (Eq. (1)) the legitimate The first part of this work was carried out within the EU
decision-maker has to accept the ‘compensatory’ assumption. If, Research Project TRUST (Transitions to the Urban Water Services of
for some reason, a low performance in a given criterion should Tomorrow). We would like to thank the TRUST team for their
automatically mean that the global sustainability score cannot be useful discussions. The European Commission is acknowledged for
above a certain threshold (irrespective of the actual performance in funding TRUST in the 7th Framework Programme under Grant
all the other criteria), then the aggregation of partial scores Agreement No. 265122.
through a typical compensatory model is not admissible. The ‘veto
power’ of some criteria would require the use of non-compensa-
tory models to perform a global evaluation. However, these models
Appendix A. Supplementary data
entail some methodological drawbacks and reduce the acceptabil-
ity and informative value of the results (due to the ‘black box’
Supplementary data associated with this article can be found, in
effect).
the online version, at http://dx.doi.org/10.1016/j.envsci.2015.07.
003.
6. Conclusion
References
This study proposes a UWS sustainability scorecard based on a
ASCE and UNESCO, 1998. Sustainability Criteria for Water Resource Systems.
MCDA model. It reviews the concept of UWS sustainability as well
ASCE Task Committee on Sustainability Criteria and the Working Group of
as previous efforts regarding the operationalization and measure- UNESCO/IHP IV Project M-4.3, Reston, Virginia, USA.
ment of this concept. The scorecard is based on five dimensions Ashley, D., Butler, P., Pearson, P., Jowit, P., Moir, J., 2003. Sustainability decision
(the TBL plus the ‘assets’ and ‘governance’ aspects), 14 objectives of making for the UK water industry. Proc. Inst. Civ. Eng. Eng. Sustain. 156 (12),
41–49.
UWS sustainability, their criteria and performance metrics. Since it Ashley, D., Souter, D., Butler, P., Davies, J., Dunkerley, J., Hendry, S., 1999.
involves a participatory and intuitive procedure, the MACBETH Assessment of the sustainability of alternatives for the disposal of domestic
approach was used in the MCDA modelling illustration to compute sanitary waste. Water Sci. Technol. 39 (5), 251–258.
Ashley, D., Blackwood, D., Butler, P., Davies, J., Jowit, P., Smith, H., 2002.
the weights in order to aggregate the several criteria. While Sustainability criteria for decision support in the UK water industry. J.
applying this weighting procedure it is possible to take into Environ. Plan. Manag. 45 (2), 285–301.
account the local conditions and reflect the preferences of the Ashley, R., Blackwood, D., Butler, D., Jowitt, P., 2004. Sustainable Water Services:
A Procedural Guide. IWA Publishing, London.
stakeholders in the model. AWWARF AND CSIRO, 2007. Triple Bottom Line Reporting of Sustainable Water
The paper contributes to the literature in three different Utility Performance. AWWA Research Foundation and CSIRO Land and
ways. First, the conceptual argument around the need to include Water, Australia Capital Territory.
Balkema, A., Preisig, H., Otterpohl, R., Lambert, F., 2002. Indicators for the
the ‘assets’ and ‘governance’ dimensions along with the classical
sustainability assessment of wastewater treatment systems. Urban Water 4
‘environmental’, ‘economic’ and ‘social’ concerns when addres- (2), 153–161.
sing UWS sustainability is a relevant innovation. Indeed, as Bana e Costa, C., De Corte, J., Vansnick, J., 2003. MACBETH. Working Paper
LSEOR, no. 56. London School of Economics and Political Science,
pointed out in the specialized literature, these two overarching
London.
aspects are instrumental to achieve sustainability in the water Berrittella, M., Katrin Rehdanz, K., Roson, R., Tolb, R., 2008a. The economic
sector and not adequately enclosed in the TBL framework. impact of water taxes: a computable general equilibrium analysis with an
Second, the suggested criteria and the performance descriptors international data set. Water Policy 10 (3), 259–271.
Berrittella, M., Certa, A., Enea, M., Pietro, Z., 2008b. Transport policy and climate
provided in the Appendix may be valuable sources for academics change: how to decide when experts disagree. Environ. Sci. Policy 11 (4),
and practitioners willing to undertake their own sustainability 307–314.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 151

Berrittella, M., Hoekstra, A., Rehdanz, K., Roson, R., Richard, T., 2007. The Economics of Global Environmental Change. International Cooperation for
economic impact of restricted water supply: a computable general Sustainability. New Horizons in Environmental Economics. Edward Elgar,
equilibrium analysis. Water Res. 41 (8), 1799–1813. Cheltenham (UK), Northampton (MA), pp. 134–154.
Bertrand-Krajewski, J., Barraud, S., Chocat, B., 2000. Need for improved Larson, K.L., Whiteb, D.D., Gobera, P., Harlanc, S., Wutichc, A., 2009. Divergent
methodologies and measurements for sustainable management of urban perspectives on water resource sustainability in a public-policy-science
water systems. Environ. Impact Assess. 20 (3), 323–331. context. Environ. Sci. Policy 12 (7), 1012–1023.
Binney, P., Donald, A., Elmer, V., Ewert, J., Philips, O., Skinner, R., Young, R., 2010. Lundin, M., 2003. Indicators for Measuring the Sustainability of Urban Water
IWA cities of the future program. In: Spatial Planning and Institutional Systems: A Life Cycle Approach. (Ph.D. thesis)Chalmers University of
Reform Conclusions from the World Water Congress, September. Technology.
Brattebø, H., Alegre, H., Cabrera, E., Marques, R., Hein, A., Cruz, C., 2013. A Marques, R., 2012. MCDA approach for the UWCS sustainability scorecards.
Master Framework for UWCS Sustainability. TRUST Project. TRUST Internal Report.
Brown, R., 2008. Local institutional development and organizational change for Marques, R., van Leeuwen, K., 2012. Current State of Sustainability of Urban
advancing sustainable urban water futures. Environ. Manag. 41 (2), 221–233. Water Cycle Services. TRUST Project.
Brown, R., Keath, N., Wong, T., 2009. Urban water management in cities: Mitchell, V., 2006. Applying integrated urban water management concepts: a
historical, current and future regimes. Water Sci. Technol. 59 (5), 47–855. review of Australian experience. Environ. Manag. 37 (5), 589–605.
da Cruz, N.F., Marques, R., 2013a. A multi-criteria model to determine the Monsma, D., Nelson, R., Bolger, R., 2009. Sustainable Water Systems: Step One –
sustainability level of water services. Water Asset Manag. Int. 9 (3), 16–20. Redefining the Nation’s Infrastructure Challenge. A Report of the Aspen
da Cruz, N.F., Marques, R., 2013b. New development: the challenges of designing Institute’s Dialogue on Sustainable Water Infrastructure in the US,
municipal governance indicators. Public Money Manag. 33 (3), 209–212. Washington, DC.
Fleming, N., 2008. Understanding ‘what’s really going on’ as a basis for Murray, A., Ray, I., Nelson, K., 2009. An innovative sustainability assessment for
transforming thinking, action and our cities. In: Enviro 08 Australasia’s urban wastewater infrastructure and its application in Chengdu, China. J.
Environmental & Sustainability Conference & Exhibition, Melbourne, Environ. Manag. 90 (11), 3553–3560.
Australia. Sahely, H., Kennedy, C., Adams, B., 2005. Developing sustainability criteria for
Foxon, T., Mcilkenny, D., Gilmour, C., Oltean-Dumbrava, N., Souter, R., Ashley, D., urban infrastructure systems. Can. J. Civil Eng. 32 (1), 72–85.
Butler, P., Pearson, P., Jowit, P., Moir, J., 2002. Sustainability criteria for Spangenberg, J., Bonniot, O., 1998. Developing sustainability criteria for urban
decision support in the UK water industry. J. Environ. Plan. Manag. 45 (2), infrastructure systemsIn: Sustainability Indicators: A Compass on the Road
285–301. Towards Sustainability. Wuppertal Paper no. 81, Wuppertal. .
Greco, S., Ehrgott, M., Figueira, J.R., 2010. Trends in Multiple Criteria Decision Srinivasa-Raju, K., Duckstein, L., Arondel, C., 2000. Multicriterion analysis of
Analysis. Springer, Berlin. sustainable water resources planning: a case study in Spain. Water Resour.
GRI, 2006. Sustainable Reporting Guidelines. Global Reporting Initiative, Manag. 14 (6), 435–456.
Amsterdam. Thornton, G., Franz, M., Edwards, D., Pahlend, G., Nathanaile, P., 2007. The
GRI, 2011. Sustainability Reporting Guidelines. Global Reporting Initiative, challenge of sustainability: incentives for brownfield regeneration in Europe.
Amsterdam. Environ. Sci. Policy 10 (2), 116–134.
Hajkowicz, S., Collins, K., 2007. A review of multiple criteria analysis for water UNCSD, 2001. Indicator of Sustainable Development. Guidelines and
resource planning and management. Water Resour. Manag. 21 (9), 1553–1566. Methodologies. United Nations Commission for Sustainable Development,
Hellström, D., Jeppsson, U., Kärrman, E., 2000. A framework for systems analysis New York.
of sustainable urban water management. Environ. Impact Assess. 20 (3), van der Steen, P., 2011. Application of Sustainability Indicators within the
311–321. Framework of Strategic Planning for Integrated Urban Water Management. A
Hiessl, H., Walz, R., Toussaint, D., 2000. Design and Sustainability Assessment of Training Manual for Process Facilitators of Urban Strategic Planning
Scenarios of Urban Water Infrastructure Systems. Published Report. Processes. SWITCH, UNESCO-IHE Institute for Water Education.
Fraunhofer Institute Systems and Innovation Research, Breslauer Str., van Leeuwen, K., Chandy, P., 2013. The cities blueprint: experiences with the
Karlsruhe. implementation of 24 indicators to assess the sustainability of the urban
Hoekstra, A., 2003. Virtual Water Trade: Proceedings of the International Expert water cycle. Water Sci. Technol. 13 (3), 769–781.
Meeting on Virtual Water Trade. Value of Water Research Report Series No. van Leeuwen, K., Frijns, J., van Wezel, A., 2011. Indicators for the Sustainability
12. UNESCO-IHE. of the Urban Water Cycle. Watercycle Research Institute (KWR), The
Huang, I.B., Keisler, J., Linkov, I., 2011. Multi-criteria decision analysis in Netherlands.
environmental sciences: ten years of applications and trends. Sci. Total van Leeuwen, K., Frijns, J., van Wezel, A., van de Ven, F.H.M., 2012. Cities
Environ. 409 (19), 3578–3594. blueprints: 24 indicators to assess the sustainability of the urban water
Icke, J., Boomen, I., Aalderink, R., 1999. A cost-sustainability analysis of urban cycle. Water Resour. Manag. 26, 2177–2197.
water management. Water Sci. Technol. 39 (5), 211–218. von Winterfeldt, D., Edwards, R., 1986. Decision Analysis and Behavioral
IUCN, 1991. Caring for the Earth: A Strategy for Sustainable Living. Earthscan, Research. University Press, Cambridge.
London. Water UK, 2000. Towards Environmental Sustainability: Indicators for the UK
Keeney, R., 1992. Value-Focused Thinking: A Path to Creative Decision Making. Water Industry. Water UK, London.
Harvard University Press, Cambridge, MA. Water UK, 2010. Sustainable Indicators 2009–10. Water UK, London.
Kluge, T., 2007. Integrated water resources management (IWRM) as a key for WCED, 1987. Our Common Future (Brundtland Report). World Commission on
sustainable development. In: Cogoy, M., Steininger, K.W. (Eds.), The Environment and Development. Oxford University Press, New York.