Journal of Environmental Planning and Management

ISSN: 0964-0568 (Print) 1360-0559 (Online) Journal homepage: https://www.tandfonline.com/loi/cjep20

Success and failure assessment methodology for

wastewater and faecal sludge treatment projects
in low-income countries

Magalie Bassan, Doulaye Koné, Mbaye Mbéguéré, Christof Holliger & Linda
Strande

To cite this article: Magalie Bassan, Doulaye Koné, Mbaye Mbéguéré, Christof Holliger &
Linda Strande (2015) Success and failure assessment methodology for wastewater and faecal
sludge treatment projects in low-income countries, Journal of Environmental Planning and
Management, 58:10, 1690-1710, DOI: 10.1080/09640568.2014.943343

To link to this article: https://doi.org/10.1080/09640568.2014.943343

Published online: 11 Sep 2014.

Submit your article to this journal

Article views: 646

View related articles

View Crossmark data

Citing articles: 6 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=cjep20
Journal of Environmental Planning and Management, 2015
Vol. 58, No. 10, 1690
1710, http://dx.doi.org/10.1080/09640568.2014.943343

Success and failure assessment methodology for wastewater and faecal

sludge treatment projects in low-income countries
Magalie Bassana*, Doulaye Kon
eb, Mbaye Mb
egu
er
ec, Christof Holligerd and
Linda Strandea
a
Eawag: Swiss Federal Institute of Aquatic Science and Technology, Sandec: Department of Water
and Sanitation in Developing Countries, 8600 Duebendorf, Switzerland; bBill and Melinda Gates
Foundation, East Seattle, WA, USA; cOffice National de l’Assainissement du S

en
egal, Dakar,
S
egal; dSchool of Architecture, Civil and Environmental Engineering, Laboratory for
en

Environmental Biotechnology, Ecole Polytechnique F
ed
erale de Lausanne (EPFL),
Lausanne, Switzerland

(Received 10 October 2013; final version received 3 July 2014)

Many factors influence success and failure of large-scale faecal sludge and wastewater
treatment projects in low-income countries. Benchmarking indicators and multicriteria
analysis were adapted to define key institutional, technical and financial factors, to
analyse their interrelations, and understand priorities to consider when planning and
managing treatment plants. For the first time, these methods have been combined in a
quantitative manner to assess planned and on-going treatment plant projects. This new
methodology will aid sanitation utilities, private consultants, and funding institutions to
prioritise activities and organise the operation of treatment plants.
Keywords: assessment methodology; faecal sludge; low- and middle-income countries;
priority definition; wastewater

1. Introduction
A change of mind-set is required for the implementation of sustainable sanitation projects,
including new methods to select appropriate alternatives that ensure long-term operation
(Mara and Alabaster 2008). Indeed, infrastructures and capital investments alone are not
sufficient to provide functioning sanitation systems in low-income countries. Currently, the
measurement of the success of projects is frequently limited to whether the treatment plant
is built, especially from the perspective of funding agencies. However, assessment of
success or failure should rather be based on the effective provision of the expected services
of the project (e.g. wastewater and faecal sludge treatment). Therefore, organisational and
managerial aspects are important (Strande, Ronteltap, and Brdjanovic 2014).
Rates of success can be increased by a thorough understanding of reasons for failures,
by identifying areas that require increased capacity and risk mitigation strategies (IWMI
2008). Although many wastewater treatment plants (WWTPs) and faecal sludge
treatment plants (FSTPs) have failed, the exact reasons are typically not fully understood,
as each employee or local stakeholder understands only a part of the complete situation.
Several authors and institutions have shown interest in understanding the reasons for
success or failures over the last 10 years. Inappropriate technology selection, lack of
operation and maintenance (O&M) and financial capacity are often cited (Fernandes,

*Corresponding author. Email: [email protected]

Ó 2014 University of Newcastle upon Tyne

 Journal of Environmental Planning and Management 1691

Kirshen, and Vogel 2005; Lennartsson et al. 2009; L€uthi et al. 2011; Nikiema et al. 2013).
However, a rigorous and general methodology to analyse causes of failures does not exist,
resulting in limited means to prevent them and to improve management systems. Previous
evaluations mainly focused on the adequacy of the technology selection and technical
aspects related to the operation, and managerial aspects were not thoroughly assessed,
although acknowledged as important (Murray and Drechsel 2011; Oliveira and von
Sperling 2011).
Benchmarking indicators designed to measure performance in service coverage,
quality and operational costs are frequently used to compare the organisation among
utilities in different cities and to determine best practices (Cabrera 2008). The
‘International Benchmarking Network for the Water and Wastewater Utilities’ (www.ib-
net.org, as of 14 May 2013) works towards this goal. However, a set of benchmarking
indicators specially developed to optimise operational management of WWTPs and
FSTPs does not yet exist. Benchmarking could be expanded to include success and
failure factors if extensive socio-economic and environmental data were collected, but
this is expensive and time consuming (van den Berg and Danilenko 2008).
Multicriteria Analysis (MCA) is a decision-making method that includes socio-
economic, historical, technological, environmental and business aspects through
involvement of local stakeholders (Ben Mena 2000; Zopounidis and Pardalos 2010).
However, MCA methodology as it was developed cannot be applied in a quantitative way
to evaluate existing facilities such as WWTPs or FSTPs. Modifications can be made to
allow prioritisation of actions for successful management, considering aspects related to
institutional and human resources.
A method to evaluate how aspects influencing the success are integrated during the
project is required to strengthen the planning process. During this study, a comprehensive
method was developed and adjusted through case studies in Senegal. This integrated
multiple domains (i.e. Institutional Management, Technical Design, and Financial and
Energy Resources), a need highlighted by Balkema et al. (2002) and Kvarnstr€om et al.
(2004). The method assesses specific reasons for the success or failure of centralised
WWTPs and FSTPs from the project inception, through on-going O&M. It can be
implemented to identify priority actions for improvements of the management of existing
treatment plants.
This paper presents the assessment method in its first development stage. The list of
criteria and indicators can also be used as a checklist of important aspects to take into
account when planning, designing and operating WWTPs and FSTPs. Recommendations
for further use of the list and analysis methodology are given in the Results. Further
outcomes from these case studies in Senegal are presented in Bassan (2009).

2. Methodology
The evaluation methodology was based on case studies in three cities in Senegal,
including two FSTPs and three WWTPs. The effluent of the FSTPs in Camb
erene and
Rufisque is co-treated with the wastewater in the neighbouring WWTPs. In Thies, a
simple WWTP was assessed. Senegal was selected for the study location as the
Senegalese National Sanitation Utility (ONAS) has built, operates and manages several
FSTPs and WWTPs, as described in Dodane et al. (2012).
This study focused on management and O&M of treatment plants, and did not include
other aspects of the sanitation infrastructure (e.g. sewer, onsite sanitation systems).
Decision makers, engineers, treatment plant employees and private consultants were
1692 M. Bassan et al.

consulted during the study. All stakeholders had experience with design, O&M,
monitoring and/or financial management of WWTPs and FSTPs in Senegal. A
participatory approach was employed to ensure a representative evaluation of the
understanding of local sanitation experts, and the consideration of local conditions.
The developed criteria and indicators list was first applied to assess the five treatment
plants. MCA analysis was then adapted based on the case study to ensure a quantitative and
reliable analysis of the level of importance of the criteria and indicators, and to evaluate
improvement priorities. The results of the interviews based on the criteria and indicators list
were assessed with six adapted analysis steps. Subjective importance given by the
stakeholders to indicators could thus be compared with more objectively set priorities. How
the criteria and indicators list and the analysis steps were developed is presented below.

2.1. Development of criteria and indicators list

A preliminary list of criteria and indicators was developed based on a review of
international scientific literature, ONAS internal documents and site visits to the five
treatment plants. According to Maystre, Pictet, and Simos (1994), it was ensured that all
stakeholders accepted the methods to develop the list, that the criteria and indicators were
defined prior to interviews and that they allowed assessment of all possible situations.
Regional circumstances affecting the design and management were also considered (e.g.
electricity availability) (Massoud, Tarhini, and Nasr 2009).
As acknowledged by Weissenbach et al. (2013), many technical and managerial
aspects influence success of treatment plant projects. Therefore, three capacity domains
were considered: Institutional Management (e.g. relations between stakeholders),
Technical Design (e.g. treatment efficiency, O&M), and Financial and Energy Resources
(e.g. financial sustainability, energy usage). Financial and Energy Resources were
purposely separated from the Technical Design to allow sufficient consideration of the
influence of energy requirement and distinction with O&M activities, as discussed by
Murray and Drechsel (2011) and Weissenbacher et al. (2013).
Each of the three study domains was broken down into criteria, sub-criteria and
indicators, each representing more detailed scales of information for the evaluation. The
goal was to provide a comprehensive and representative overview of indicators, criteria
and sub-criteria that influence the long-term operation of treatment plants. The criteria
represent large-scale management concerns (e.g. human resources), sub-criteria further
define each of the criteria (e.g. management of employees, education management), and
the indicators are independent metrics to evaluate the relative success or failure of the
sub-criteria (e.g. ability to replace employees, frequency of continuous education).
The indicators developed by Koottatep et al. (2005), Dodane, Makboon, and Torrens
(2006) and Zhou et al. (2009) for WWTPs and FSTPs at the technical and economic level
were included. Indicators were also incorporated from NETSSAF (2006), Sujaritpong
and Nitivattananon (2009) and Gaulke et al. (2009) for the evaluation of sanitation
options, including also the institutional framework as an enabling condition for these
aspects (Strauss, Kon
e, and Montangero 2003). The final list of criteria and indicators is
presented in the results.

2.2. Analysis methodology

A methodology to distinguish the strengths and types of influence between a large number
of criteria and indicators was developed by adapting the MCA analysis steps outlined in
 Journal of Environmental Planning and Management 1693

Figure 1. Steps of the analysis methodology (boxes on the left) along with their outputs (arrows on
the right).

Sh€arlig (1985) and Guene, Tour
e, and Maystre (1999). The six evaluation steps enabling a
thorough understanding of the management system and the definition of priority actions
are presented below and summarised in Figure 1, together with the outputs of each step.

2.2.1. Validation of criteria and indicators

The first analysis step was a participatory workshop with the main stakeholders (e.g.
ONAS directors and engineers) to validate the preliminary list of criteria and indicators.
This ensured the efficiency of the recommendations resulting from following analyses.
Each criterion, sub-criterion and indicator was presented and evaluated, and aspects
could be incorporated based on the knowledge of local stakeholders.
A theoretical validation of the completed list was carried out after the participatory
validation to ensure that the indicators accurately represented the analysed systems.
Therefore, some indicators, sub-criteria and criteria were reformulated or modified to
answer the three conditions outlined by Maystre, Pictet, and Simos (1994): (1) an
exhaustive list of represented issues; (2) coherence between information scales; and
(3) non-redundancy. The output of this step was a validated list of criteria and indicators,
which local stakeholders found exhaustive, and for which the definition limited the
redundancy of indicators as much as possible. This list was then used to assess
the understanding of local stakeholders through step two, which allows weighting the
importance of indicators.
1694 M. Bassan et al.

Table 1. Example of weights given by three stakeholders (S1, S2, S3), and calculations for the
mean weights of indicators and sub-criteria. The highest rank is given to the sub-criterion with the
highest weight.

Indicators Sub-criteria

S1 S2 S3 Mean Mean Rank

a1 5 4 3 4 a 4.2 1
a2 4 5 4 4.3
b1 2 3 2 2.3 b 2.3 4
c1 4 2 1 2.3 c 2.8 3
c2 3 4 3 3.3
d1 3 3 4 3.3 d 3.3 2
d2 4 5 2 3.7
d3 3 3 3 3.0

2.2.2. Weighting of the indicators and sub-criteria

To develop an accurate model of the system and facilitate the synthesis and representation of
the results, the data were analysed quantitatively. A weight was thus given on a scale from
1 (no influence) to 5 (strong influence) to the indicators during semi-guided interviews with
each stakeholder. A balance of administrative and operating staff, from both the private and
public sectors, were selected for the interviews to ensure representative results.
The goal of the weighting was to reflect the stakeholders’ opinions about the
importance of each indicator. This was completed independently from the evaluation of the
situation through the indicators (e.g. satisfactory or poor performances). The final weight of
an indicator was calculated as the mean of the weights given by all stakeholders. As shown
by example in Table 1, the weights of the indicators were then aggregated in sub-criteria.

2.2.3. Refining the criteria and indicators list

In order to avoid redundancy, the list was then reviewed. Indicators that repeated the same
point of view were consolidated through aggregation. Some repetitions related to the local
context were revealed during the interviews. Each group of indicators that concerned the
same issue was treated separately to determine a new consolidated indicator. The aggregated
mean for the consolidated indicator is equal to the mean of the former indicators:

WeightðaÞ þ WeightðbÞ
¼ Weightða 0 Þ (1)
2

For example, if indicators ‘a’ (Quality of field study and designer understanding of
context; weight 4.7) and ‘b’ (Design parameter adequacy to the context; weight 4.9) were
assessing the same sub-criterion (Consideration of local context in preliminary studies) of
the same criterion (Design studies and technical choice), they were aggregated to a new
indicator ‘a1’ (Field study quality and design parameter adequacy; weight 4.8).

2.2.4. Determination of influences

To allow quantitative comparison between the priority defined by the stakeholders through
the weight and the objective level of influence of the indicators, sub-criteria and criteria, a
 Journal of Environmental Planning and Management 1695

Figure 2. Example of a correlation diagram. Boxes represent indicators and arrows represent
influences between them.

correlation diagram was built. This diagram, which was made independently from the
weighting, provided a global picture of interdependent relationships. The correlation diagram
was adapted from MCA steps, and the concept of influence was used: an indicator or sub-
criteria has the potential to determine another one, independently from the local situation.
Indicators that are not influenced by any other can be considered as ‘stronger’. Due to their
influence on other indicators, ‘strong’ indicators pinpoint where changes should be made if
the current situation needs to be modified. This provides a way to analyse indicators and sub-
criteria in a logical way, and to obtain a generic picture of possible correlations. Therefore,
this requires expertise in the sanitation domain.
As shown in the example of Figure 2, each indicator or sub-criterion was represented
as a block, and arrows represented the influences among them. Each arrow has only one
direction, but influences can be in either direction (e.g. ‘d’ could also be influencing ‘c’,
instead of being influenced by it, if these were different indicators).
A preparatory step classified the indicators based on a temporal scale, referring to the
phase in the project at which the indicators must be considered, and a hierarchy scale,
referring to the level of the concerned stakeholders. Scores were given for indicators
concerning the concept (score D 3), the design (2), the O&M (1) on the temporal scale.
Scores on hierarchical scale assigned for indicators concerning the national level (3), for
utility management (2), and for treatment plant employees (1). Indicators having the
maximum temporal and hierarchical influence (i.e. sum of the two scores) were
designated as having the greatest impact.
A first correlation diagram was made with indicators to ensure the integration of all
influences at the most detailed information scale. This was built incrementally around the
indicators having the lowest impact (i.e. low rank on the temporal and hierarchical
scales). For example, in Figure 2 indicator ‘d’ has the lowest temporal and hierarchical
influence. The potential influence or correlation of all indicators on ‘d’ is assessed: ‘b’
and ‘c’ are linked to indicator ‘d’, as they influence it. The influence of other indicators
on ‘c’ and then ‘b’ is assessed, as for all other indicators, and other arrows are drawn to
represent potential and realistic influences.
The correlation diagram revealed relations among indicators, to account for influences
that were not discussed during interviews. Indicators that were not linked to another were
eliminated or reformulated, as they do not influence success.
The correlation diagram at the sub-criteria scale then summarised the information
from the indicator correlation diagram and provided a better overview. Therefore, all
influences drawn in the diagram at the indicator level were represented at the sub-criteria
level. This facilitated the identification of most influential sub-criteria.
1696 M. Bassan et al.

Table 2. Influence and ranks of sub-criteria determined for direct, inverse, flux balance and final
rankings, based on example of Figure 2.

Direct ranking Inverse ranking Flux balance ranking Final ranking

Sub-criterion Influence Rank Influence Rank Influence Rank Influencea Rank

a 0 1 3 1 2 1 1 1
b 1 2 2 2 1 2 2 2
c 1 2 1 3 0 3 2.75 3
d 2 3 0 4 ¡2 4 3.75 4

Note: aThe influence of the final ranking was calculated with the ranks of direct, inverse and flux balance (e.g.
criteria a: ((1C1)/2C1)/2=1). The final rank is attributed with the higher rank for the lower influence value.

2.2.5. Ranking of sub-criteria

To rank the sub-criteria based on their level of influence, and understand their relative
influence, calculations were made based on the observation of the correlation diagram.
These calculations were inspired by MCA steps, and allowed the process to be
quantitative, rather than subjective. Ranking was made at the level of sub-criteria to
obtain a general understanding of the influences.
The sub-criteria were ranked depending on the level of influence they showed, as
defined by the number of arrows that were connected to each block in the correlation
diagram. Arrows were weighted for more precision (not shown in Figure 2). Thus,
distinction was made between strong influences, which were evident (weight D 2) and
weak influences, where an indicator could, but did not necessarily influence the other
(weight D 1).
Three ranking modes were adapted from Sh€arlig (1985): direct, inverse and flux balance.
A table was generated with the result of the three different calculations, which aimed to
assess the relative importance of the sub-criteria. Table 2 gives an example for the theoretical
correlation diagram of Figure 2, with unweighted influences. As weighted influences were
applied, the ranking was calculated as following: influence D ranking  weight.
The direct ranking arranges the sub-criteria based on the number of entering
influences that determine a sub-criterion. As commonly employed for MCA, this is
calculated by minimising the length of the shortest influence entering to a sub-criterion.
For example, in Figure 2, as there is no influence arrow directed toward sub-criterion ‘a’,
but there is an influence arrow from sub-criterion ‘a’ to sub-criterion ‘b’, then ‘a’ receives
a rank of 0, and ‘b’ a rank of 1 (see Table 2). The sub-criteria with the lowest sum were
ranked first as being less influenced by others. The direct ranking highlighted the
arrangement between sub-criteria determining the system at the highest hierarchy level,
which correspond to the starting points of the correlation diagram.
The inverse ranking was based on the level of influence of each criterion on the rest of
the system (i.e. outgoing influences). It corresponds to the maximisation of the length of
the outgoing influences from one sub-criterion to end-points of the correlation diagram
(i.e. indicator having no influence on other). In Figure 2, ‘d’ is an end-point and is
indirectly influenced by ‘a’ through ‘b’ (‘a’ directly influences ‘b’, and ‘b’ directly
influences ‘c’). Calculations based on the inverse ranking give a result of two for
indicator ‘a’ (first rank) because the path to ‘d’ contains two arrows. Indicator ‘b’ obtains
the second rank after ‘a’, as the path to ‘d’ contains one arrow. Indicator ‘d’ obtains 0 and
is ranked in the third rank, as it is the end-point. The sub-criteria in the last ranks are the
 Journal of Environmental Planning and Management 1697

Table 3. Comparison of the results from the weight analysis in Table 1 and the sub-criteria ranking
in Table 2.

Sub-criterion Weight rank Influence rank

a 1 1
b 4 2
c 3 3
d 2 4

most impacted by others and not influencing any. They are representative of the
performance of the system and of operational problems.
As indicated by Sh€arlig (1985), when the results of direct and inverse ranking are very
different, they should be considered as extremes, and flux balance ranking is required to
determine the final, most representative ranking. The flux balance ranking minimises the
difference between the direct entering and the outgoing influences. For example, in Figure 2,
sub-criterion ‘a’ has two outgoing and 0 entering influences, it therefore has a difference of two
(2-0 D 2); and ‘b’ has two outgoing and one entering influences, it thus receives a difference of
one (2-1 D 1). Because ‘a’ has more outgoing than entering influences, ‘a’ is ranked before ‘b’.
The sub-criteria with the highest value of flux balance ranking are placed in the first
ranks. The arrangement based on the flux balance proved to be more precise than the
direct and inverse rankings for the sub-criteria in the middle ranks. Based on the analysis
of the flux balance ranking, sub-criteria that can have an impact on the entire system
could be highlighted, providing springboarding opportunities for improving national
sanitation strategies.

2.2.6. Definition of priority action

The direct, inverse and flux balance ranking were combined in the following equation,
which yields the final ranking in order to provide a general understanding of the sub-
criteria influences:

.
Direct þ Inverse
þ FluxBalance 2 (2)
2

This equation was derived to be representative of the real influences of the sub-criteria
and to rank priorities. It takes into account the fact that the direct and inverse ranking
arranged indicators in the first and last ranks with precision, respectively, but showed
incoherencies for the other ranks. The final ranking represented twice as much weighting
for the flux balance, as it was impacted both by entering and outgoing influences, is more
complete and therefore better arranges sub-criteria in the middle ranks.
The resulting numerical ranking provided a representative hierarchy of all sub-
criteria objectively affecting the treatment plant management. In comparison, the
weights given by stakeholders did not take into consideration the complete system,
as they were mostly not aware of difficulties influencing activities that were not
under their responsibility. The differences between the objective final ranking and
the subjective weight given by the stakeholders therefore revealed the weaknesses
of the existing management system and mind-set. In the example of Table 3,
1698 M. Bassan et al.

sub-criteria ‘a’ and ‘c’ are given the same importance or influence by the
stakeholders and by the quantitative analysis of the final ranking. However, sub-
criterion ‘d’ is given too much importance by the stakeholders in comparison to sub-
criterion ‘b’. This highlights the need in optimisation for criterion ‘b’, which needs
to be better integrated in the management system or in the design procedure.

3. Results and discussion

3.1. Selected criteria and indicators
In total, the preliminary list contained 51 indicators, from eight Institutional Management
criteria, three Technical Design criteria and two Financial and Energy Resources criteria.
These were selected based on the literature review, visits and preliminary discussions
with local stakeholders and experts, to ensure relevance to the local context. Discussions
were also conducted with international experts to ensure that indicators and criteria can
be applied in other low- and middle-income countries.
This preliminary list was further validated, refined and used for analysis, to finally
obtain 42 indicators (i.e. following steps 1, 2, and 3 of the analysis methodology). The
final criteria and indicator list was designed to assess the key criteria, sub-criteria and
indicators leading to success and sustainability, and their interrelations. Treatment
performance (e.g. efficiency of removal of pollutants) was not included, as treatment
objectives and standards vary for different situations, and the objective of this
methodology is improving the management system. It can therefore be implemented in
each context, with scores adapted to local treatment standards.
The resulting list is more complete than previous assessment lists available in the
literature. It addresses the need for comprehensive assessment methods to improve the
sustainability of treatment plant management systems, and for standardisation of
benchmarking processes, as highlighted by Pybus and Schoeman (2001). The assessment
process, expert interviews, literature review and visits confirmed the importance of
including institutional, technical, financial and socio-economic aspects. The result is a
comprehensive list of indicators, which is necessary to adequately assess existing and
complex sanitation systems, and/or to plan new ones. Therefore, expert and local
stakeholder interviews must be comprehensive enough to ensure adequate coverage.
After validation and refining of the criteria and indicators list, 27 indicators for
Institutional Management, 10 for Technical Design, and 5 for the Financial and Energy
Resources were retained. Quantitative analysis of institutional aspects is complex, and
hence requires many indicators for Institutional Management, which also needs to be
preliminarily agreed upon with local stakeholders. However, assessing Institutional
Management with similar metrics of technical and financial aspects contributes to a
global understanding of the situation.
Brown and Holcombe (2004) highlighted the need for a strong institutional and legal
framework, and of competent institutions with independent funding. This is important
and contrasts with the funding procedure of many development projects that focus mainly
on the construction of infrastructure, and less on management aspects (Kon
e 2010;
Strande, Ronteltap, and Brdjanovic 2014). Current planning and management systems for
WWTPs and FSTPs in low- and middle-income countries, which often neglect one or
several management aspects in the local context, are conducive to failure. Therefore, it
should be acknowledged that planning or optimising treatment plants at large scale
requires expertise, good willingness of local authorities and stakeholders to set up an
 Journal of Environmental Planning and Management 1699

efficient strategy, and financial means to ensure efficient institutional organisation and
management.

3.2. Analysis methodology

An innovative outcome of this study was that the methodology can be implemented for
the optimisation of the management of existing treatment plants. This is contrary to the
classical MCA procedure, which is used as decision aid tool and therefore dedicated
only for planning. The six evaluation steps allowed the analysis of management
systems, and could readily be used by utilities, consultants or funding organisations in
order to identify areas for improvements. The advantages of each of the steps are
further detailed below and recommendations are given for the implementation of the
methodology.
In general, the optimisation of the management of existing treatment plants requires:
(1) a meeting to validate the indicators and criteria; (2) interviews with local stakeholders
to assess the situation and prioritisation; (3) control and adjustment of the ranking of sub-
criteria based on the local situation (only if indicators were added); and (4) definition of
priority actions. The theoretical validation, refining of the list and determining influences
through a new correlation diagram should only be performed if necessary to adapt to the
local situation and in that case, new indicators are also strongly required.

3.2.1. Validation of criteria and indicators

The participatory validation of the preliminary criteria and indicators list with utilities
directors and engineers ensured the proper consideration of two important aspects
highlighted by Schutte (2001): (1) the integration of the main stakeholders in the process
that ensures the acceptance of the study goals; and (2) the consideration of key success
factors that directly concern the operator. Indicators were added to the first criteria and
indicator list, based on the input of workshop participants. Following this, approximately
10 managers and employees of ONAS found the list to be complete and representative of
their different points of view. Another advantage of this validation meeting was that it
raised awareness of all employees and stakeholders on important aspects and challenges
that others were dealing with. When implementing this methodology in other contexts, a
validation meeting is important to adapt the criteria and indicator list, and to ensure that
the person in charge of the assessment process fully understands the local situation.
Scoring of indicators to assess the local situation and weighting to evaluate priorities
should also be discussed at this time.
The theoretical validation ensured a non-redundant and coherent list. Similar
indicators concerning the same sub-criterion were aggregated, and the list was modified
to answer the three requirements stated in Section 2.2.1. For future implementations of
this methodology if no indicators are added, this step is not required. New indicators
should only be added if it can be ensured that redundancy can is avoided. In this case, the
theoretical validation is not required.
After participative and theoretical validation, a list containing 13 criteria was
developed. These are briefly presented below, by the domain they fall under.
Recommendations for the different indicators defining each criterion are given to allow
the user of the methodology to understand how to improve the situation with a focus on
relevance to low- and middle-income countries. However, the scoring scale presented in
Bassan (2009) can be adapted for any context.
1700 M. Bassan et al.

Institutional Management

(1) Institutional autonomy: The level of autonomy of the sanitation utility to other
state institutions is assessed to ensure that political changes do not have too
strong of an influence on the entire sanitation system. It is better if the sanitation
department is independent from other state institutions and if projects and
contracts can be defined without intensive bureaucracy at the state level.
(2) Education in country: The availability of curricula and training for wastewater
and faecal sludge treatment is assessed. Such training should be available in the
country and accessible for engineers and technicians in charge of the planning,
design and construction of sanitation infrastructure.
(3) Decision-making process: The internal hierarchy and communication efficiency
is assessed. Communication must flow frequently horizontally and vertically
among departments. It is recommended that design, construction and operational
experiences are shared frequently (e.g. not only in annual reports) and that
technicians get rapid feedback on demands for work execution and material,
ensuring an uninterrupted operation. Therefore, procedures must be well
coordinated and information rapidly distributed.
(4) Human resource management: Hiring conditions and training opportunities are
evaluated. The operator must gain loyalty of the employees by both financial
incentives and professional development. To improve the internal know-how,
programmes should exist to facilitate access to higher education and regular
training for all employees. Hiring of new employees based on their competencies
is recommended.
(5) Direction expertise: The upper management is assessed by their level of technical
and managerial knowledge. It is a more ideal situation if the directorate has
confirmed experience in sanitation and appropriate knowledge for their positions.
Engineers should not manage more than one large-scale project at a time and the
contract awarding process should be carried out by a committee including
technical and financial experts to avoid corruption.
(6) O&M department expertise: O&M in terms of human resources and procedures
are assessed. O&M competencies are considered good if treatment plant
managers have a complete understanding of treatment processes and if
preventive maintenance is well planned.
(7) Private consultant services: The local expertise of private consultants is assessed
through the qualification of the design engineering, and the construction
company. The qualifications should include experiences in the successful design
and construction of at least two similar treatment plants. Guarantees should also
be provided to the utility for the design, equipment and construction.
(8) Social integration: Community outreach is assessed. Social acceptance of the
population living in the direct surroundings of the treatment plant should be
addressed through studies. Economic benefit of the community is also important
through the construction and operation of the treatment plant (e.g. labour
contracts, resource recovery).

Technical Design

(9) Quality of preliminary study: The technical options and the quality of field studies
are assessed. Complete field visits and surveys are recommended during
 Journal of Environmental Planning and Management 1701

preliminary studies, and technical options need to be appropriate to local

conditions.
(10) O&M constraints management: Acquisition and management of spare parts,
tools, consumables and maintenance procedures are assessed. All spare parts
that are frequently changed should be readily available and accessible in the
country, and a complete stock of tools and supplies should be in place at each
treatment site or easily accessible. Dependency to external services for O&M
should be well controlled and on-time delivery of urgent services ensured.
(11) Monitoring and optimisation: Skills available for the monitoring and the
optimisation of processes are assessed. A well-equipped laboratory should be
accessible for regular monitoring of each treatment facility. Employees should
be trained on treatment principles and laboratory methods, and O&M needs to
be optimised based on monitoring results.

Financial and Energy Resources

(12) Financial balance: The functionality of the budget is considered good if the
budget is based on real O&M constraints, if it is possible to mobilise funds for
the extension of treatment plants to adapt to the loads, and there is resource
recovery of treatment end-products.
(13) Energy balance: The energy balance is considered for the technological choice,
as electricity shortage can affect O&M. The total energy cost should be
minimised, and the quantity of energy produced onsite or from renewable
sources maximised.

3.2.2. Weighting of the indicators and sub-criteria

Interviews were conducted one at a time in order to capture important issues in the
specific field of each stakeholder. To ensure an even distribution at the decision-making
level, each indicator was weighted by at least three stakeholders. Weights were
representative of the importance of the indicators, whether their assessment was
considered to be positive or negative (i.e. high or low scores). The willingness of
stakeholders to participate in interviews and the weighting process was facilitated by
their interest in the success of their department, and their anonymity.
The analysis of the weight revealed that 88% of the indicators were considered as
important (weight  four), proving a good representativeness of the criteria and
indicators list and confirming the applicability of this method in the West African
context. At the end, a good overview of the issues that are considered to be important by
local stakeholders could be obtained.
The weighting process is an important step that prevents imbalance between technical or
administrative aspects in the managerial system. It should therefore always be implemented
in future use of this methodology. The objectives need to be clearly explained to
stakeholders. Interviews of approximately one to two hours should be organised with the
main administrators who are responsible for each of the utilities, the engineers responsible
for design and operation, and the employees operating each treatment plant.

3.2.3. Refining the criteria and indicators list

The indicators aggregation completed the theoretical validation and resulted in a more
even representativeness. From the preliminary list, all the criteria presented above were
1702 M. Bassan et al.

retained, but a few indicators that were representative of similar issues were aggregated.
After this, all indicators were maintained for the next analysis steps.
In future implementations of this methodology, this step can be avoided. New
indicators are only added if sufficiently different from those already presented in this paper.

3.2.4. Determination of influences

The indicators correlation diagram allowed a very precise understanding of the
influences. The sub-criteria correlation diagram then provided a good synthesis. Figure 3
illustrates the interrelatedness of the 23 sub-criteria and the complexity of the system. As
the influence network is very dense, numbers are provided at the top of each box with the
criterion number of the entering influence for each criterion. These numbers are related to
the order in which indicators are presented in Section 3.2.1. For example criterion 11.1 is
influenced by 9.1, 12.1, and 12.2 (see bottom left of Figure 3). The analysis of this
correlation diagram shows that every sub-criterion has an influence on the global system
and thus must be considered by every stakeholder.
Three types of positions in the diagram that reveal trends can be easily observed:
(1) starting points are very influential sub-criteria (i.e. boxes with black-bold outline);
(2) convergence nodes constitute the interface between all stakeholders and have the
greatest number of entering and outgoing influences (i.e., boxes with dashed outline); and
(3) end-points are influenced by all previous sub-criteria (i.e. boxes with bold grey
outline). The other boxes represent sub-criteria concerning the procedures and technical
means to run a treatment plant.
Figure 3 demonstrates the importance of an approach that considers all three domains
simultaneously, and not separately. It also highlights the importance of conducting an
analysis of how criteria are interrelated to explain the numerous examples of failure or
abandonment of treatment plants.
The drawing of the correlation diagrams is a complex, but crucial step of this
methodology. In future implementations, if indicators were added, this requires an expert
with a good understanding of the sanitation domain, the local context and the result of the
validation meetings and interviews. Requiring this level of expertise is a limitation of
MCA methods, as they are designed for the understanding of complex situations with
several influencing parameters. Therefore, it is again not recommended to add indicators
unless critical aspects for the local context are not included in the criteria and indicators
list, and if the requisite expertise is available.

3.2.5. Ranking of sub-criteria

The sub-criteria ranking (i.e. direct, inverse, flux balance) proved to be efficient to
distinguish sub-criteria based on their influence, and to indicate at which phase in the
project they should be considered. The outcome of three rankings is presented in Table 4,
where the results of calculations based on the correlation diagram of Figure 3 and final
ranks are also shown.
The direct ranking identified influences on the basis of the system (i.e. high hierarchy
level, conception phase) and resulted in five ranks, whereas the inverse ranking identified
sub-criteria that are influenced by the entire system (i.e. O&M of treatment plant).
The inverse and flux balance ranking both contained 13 ranks. Sub-criteria related to the
institutional domain at the national level mainly constituted the highest ranks of the three
rankings. The lowest ranks contained more technically or O&M related sub-criteria.
Three sub-criteria differ from this trend: ‘Valorisation’, ‘Optimisation skill’ and
 Journal of Environmental Planning and Management
1703

Figure 3. Correlation diagram between the sub-criteria. White, grey and black boxes refer to Institutional Management, Technical Design and Financial and
Energy Resources domain. Boxes with a black bold outline are starting points, with grey bold outlines are end-points, and with dashed outlines are convergence
nodes. The number at the top of each box shows the entering influences.
1704 M. Bassan et al.

Table 4. Direct, inverse, flux balance and final ranking of sub-criteria based on their influence on
success or failure of WWTPs and FSTPs. The calculations for the final ranking are based on the
ranks of the direct, inverse and flux balance ranks. Final ranking is then distributed from the lowest
value to the highest. A rank is attributed to each different value. Ranks of lower value are attributed
to sub-criteria with higher influence.

RANKING

Final
Direct Inverse Flux balance
Sub-criterion rank rank rank Result Rank

Institutional autonomy 1 1 2 1.5 1

Access to education 1 5 1 2 2
Capitalisation 1 6 3 3.25 3
Budget planning 3 3 4 3.5 4
Management ability 3 4 4 3.75 5
Management of employees 4 5 4 4.25 6
Education management 4 5 4 4.25 6
Internal communication 1 8 4 4.25 6
Valorisation 3 2 6 4.25 6
Appropriation 1 8 5 4.75 7
Optimisation of energy usage 3 2 8 5.25 8
Planning ability 3 7 6 5.5 9
Optimisation skills 2 1 10 5.75 10
Funding ability 3 10 6 6.25 11
Mastery of technologies 2 8 8 6.5 12
Contractual responsibility 4 10 7 7 13
Methodological approach 3 9 9 7.5 14
O&M skills 3 11 10 8.5 15
Economic integration 4 13 9 8.75 16
Dependency on external energy 5 13 10 9.5 17
Monitoring quality 4 13 11 9.75 18
Handling of technician request 3 13 12 10 19
O&M needs 3 12 13 10.25 20

‘Optimisation of energy usage’. These are managed at the utility level, but can impact the
national sanitation strategy or decisions at a higher hierarchical level. They are potential
springboards and constitute key sub-criteria for success. The valorisation of treated end-
products locally in particular can contribute to a better financial viability of the sanitation
system (Diener et al. 2014).
In future implementations of this methodology this step only needs to be completed if
the list of criteria and indicators has been modified and new correlation diagrams were
made. Otherwise, a rapid verification that the correlations presented in Figure 3
correspond to the local context is sufficient.

3.2.6. Definition of priority actions

The final ranking presented in Table 4 gives a precise representation of the hierarchy
between sub-criteria, confirming the trends discussed above. These are general
 Journal of Environmental Planning and Management 1705

calculations corresponding to the five treatment plants assessed in Senegal. As

correlations were analysed directly from the diagram in Figure 3, these can be extended
to other situations.
The arrangement of ranks reveals the decision level (e.g. national, utility, treatment
plant) and the project phases (e.g. concept, design, O&M) at which the sub-criteria
should be considered. The result is a better understanding of temporal and hierarchical
influences. This shows that the classical assessment criteria organised by capacity domain
is not optimal. This method thus contributes to the understanding on how to
conceptualise, implement and operate treatment plants, considering the project phases
and decision levels.
The difference between the weights given by the stakeholders and the sub-criteria
ranking also highlighted important sub-criteria that are typically not considered but
strongly contribute to success of treatment plant projects. Based on this analysis, these
should be considered as priorities.
The final ranking revealed the importance of ‘Institutional autonomy’ at the national
level, and that ‘Budget planning’ is more influential than ‘Funding ability’. ‘Valorisation’
and ‘Optimisation of energy usage’ have great potential to improve operation and
sustainability, although frequently they are not given enough emphasis. The institutional
strategy and technological choice are critical to the success of treatment plants, as they
are prerequisites to enable a good technical design, and thus efficient O&M. The
comparison of the final ranking with the ranks based on weights is the main output of this
methodology. It allows for the identification of priority aspects for optimisation, based on
their great potential to improve the long-term operation and management of treatment
plants. If no indicators are added, the result of the weight ranking can directly be
compared with the final ranking of Table 4. Outputs of this comparison should be
discussed with local decision makers.

3.3. Final criteria and indicators list

Table 5 presents the final list of criteria, organised into categories that take into account
the decision level and the project implementation phase. A scale is described for each
indicator in Bassan (2009). This arrangement provides a good basis to make
recommendations to the stakeholders at each level. However, the efficiency of such
recommendations remains to be tested in the field.
The six categories in Table 5 are presented below, with the project phase at which
they should be considered:

(1) Category 1 must be considered at the very beginning, when setting up or

optimising a treatment plant management system. The five indicators define the
overall sanitation strategy of the country.
(2) Category 2 presents four springboard indicators. They are managed at the utility
level but can change sanitation strategy at the national level. The knowledge and
budget available for resource recovery and performance optimisation can
increase interest and investment from politicians and funding organisations.
(3) Category 3 presents the elements that must be considered when determining the
internal organisation of the sanitation utility. They influence the relations and
communication between stakeholders.
Table 5. Criteria, sub-criteria and corresponding indicators of six categories organised by decision levels and project implementation phases. The numbers refer to
1706

the presentation order in Section 3.2.1, and correspond to the numbers in Figure 3. As some criteria are influenced by various stakeholders at different phases in a
project, they are repeated in different categories.

Criteria Sub-criteria Indicators

Category 1: Organisation structure at national level

1 Institutional status 1.1 Institutional autonomy 1.1.a Importance of sanitation utility in the state’s organisation
1.1.b Sanitation utility autonomy
2 Education in country 2.1 Access to education 2.1.a Access to education on sanitation technologies fitting to the context
12 Financial balance 12.1 Budget planning 12.1.a O&M budget type
12.2 Funding ability 12.2.b Ability to mobilise funds for new infrastructures
Category 2: Springboard between national and operator level
11 Monitoring, evaluation and optimisation 11.2 Skills for analysis and optimisation 11.2.a Competency to modify decisions based on existing conditions
11.2.b Collaboration with research centres
12 Financial balance 12.3 Valorisation 12.3.a Quantity of end-product sold/quantity produced
13 Energetic balance 13.2 Optimisation of energy usage 13.2.a Renewable energy generated/total energetic need
Category 3: Organisation structure at the operator level
M. Bassan et al.

3 Decision making process 3.1 Internal communication 3.1.a Hierarchy weight

3.1.b Internal communication frequency
3.1.c Interface management between department working on same project
3.2 Capitalisation 3.2.a Quality and frequency of technical experiences capitalisation
3.2.b Existence of a department to harmonise procedures
7 Private consultant services 7.2 Contractual responsibility 7.2.a Guarantee offered by consultants
7.2.b Guarantee offered by construction company
8 Social integration 8.1 Appropriation 8.1.a Participation level to decisional process

(continued)
Table 5. (Continued )

Criteria Sub-criteria Indicators

Category 4: Skills and resources consecrated to the treatment plant management

4 Human resource management 4.1 Management of employees 4.1.a Ability to gain loyalty of HR
4.1.b Ability to replace employees
4.2 Education management 4.2.a Access to higher education
4.2.b Frequency of continuous education
5 Direction expertise 5.1 Management ability 5.1.a Management competencies
5.1.b Employee’s profile adequacy to post
5.1.c Director’s knowledge of sanitation installations
5.2 Planning ability 5.2.a Number of projects per engineer
5.2.b Engineer’s experience in managing projects
5.2.c Transparency of process to award contracts
5.2.d Quality of construction monitoring
6 O&M department expertise 6.1 O&M skills 6.1.a Understanding level of treatment process
6.1.b Quality of maintenance planning
Category 5: Planning and design studies’ quality
7 Private consultants services 7.1 Mastery of technologies 7.1.a Experiences in the sanitation domain
8 Social integration 8.2 Economic integration 8.2.a Economic compensation
9 Quality of preliminary study 9.1 Quality of methodological approach 9.1.a Field study quality and adequacy of design parameters
9.1.b Adequacy of technical options to local constraints and opportunities
Category 6: Practical management of plant operation
3 Decision-making process 3.3 Handling of technician request 3.3.a Answer time to a technician request
Journal of Environmental Planning and Management

10 O&M constraints management 10.1 Answer to O&M needs 10.1.a Availability of spare parts
10.1.b Stock of tools and supply
10.1.c Reparation work frequency
10.1.d Dependency on external services
11 Monitoring, and optimisation 11.1 Monitoring quality 11.1.a Frequency of laboratory analysis
11.1.b Existing well-equipped lab
1707

13 Energetic balance 13.1 Dependency on external energy 13.1.a Energy cost/Total O&M costs
1708 M. Bassan et al.

(4) Category 4 includes issues which are the last means of action to improve the
institutional procedures for the O&M, and should be considered at the same level
as category 3.
(5) Category 5 constitutes a critical step between private and public stakeholders, and
the planning and operation phases. It is determined by the previous categories.
These indicators must be considered if other aspects are not to be improved, as
they define difficulties with O&M.
(6) Category 6 is the results of all the criteria, sub-criteria and indicators represented
in previous categories. At this organisational level, no efficient optimisation can
be done. Only emergency repair action can be undertaken. Thus, the practical
management indicators serve to assess performance of the management system in
the three domains.

The list presented in Table 5 can readily be applied for a rapid assessment through steps 1
(criteria and indicators validation), 2 (weight attribution through interviews) and
5 (priority definition). For future implementations with new projects, the list of criteria and
indicators can be used alone as a checklist of important aspects to consider for planning.

4. Conclusions
Previous methods of evaluation to assess the success or failure of treatment plants have
focused on separate domains. The methodology developed here is unique in its approach
that defines the importance of each criterion, sub-criterion and indicator, and considers
their interrelatedness. It encompasses the advantages of benchmarking and MCA and
provides a manageable way to handle a large amount of data.
The case study conducted in Senegal and the assessment with the criteria and
indicators list highlights the importance of a comprehensive approach to plan and operate
large-scale wastewater and faecal sludge treatment plants. The administrative and
decision-making process is also crucial in the success of treatment plants.
The most critical criteria identified at the technical level concerns the design studies
and concepts during the early stages of project implementations. Three important
springboarding criteria implemented at the utility level that can greatly improve the O&M
of treatment plants and national sanitation strategies are: (1) Monitoring, evaluation and
optimisation skills; (2) Valorisation of treatment end-products; (3) Optimisation of energy
usage. Other key results include:

 The final list with key criteria and indicators to consider when conceptualising,
designing, implementing, operating and monitoring WWTPs and FSTPs.
 An easy to implement arrangement of the criteria, sub-criteria and indicators that fit
to the project phases and decision level.
 A methodology for the evaluation of new and on-going WWTPs and FSTPs,
providing a means to understand relative importance of key criteria, and to define
priority actions in various contexts, useful to funding organisms, ministries,
utilities, practitioners and researchers.

When carrying out the methodology, it is important to recruit an expert to conduct the
complete analysis, which also involves a strong willingness to improve the situation. This
is a general condition for the success of any sanitation system. The method will need to
be improved in the future with field tests in other locations to verify that the criteria
 Journal of Environmental Planning and Management 1709

and indicator lists applies, and possible improvements if utilities actually implement
identified priority actions. This will also provide complete verification of the
methodology.
Considering the potential impact on the successful long-term operation of treatment
infrastructures, and in comparison to the significant capital investments they require, this
methodology is efficient both in terms of time and financial requirements. The
methodology can also be readily adapted to incorporate collection and transport of
wastewater and faecal sludge in order to include the entire sanitation service chain.

Acknowledgments
This study was part of the research collaborative programme between EAWAG (Swiss Federal
Institute of Aquatic Science and Technology) and ONAS (the National Sanitation Utility of
Senegal). This project was funded by the VELUX foundation, the Swiss Agency for Development
and Cooperation (SDC), the Swiss National Centre of Competence in Research (NCCR) EAWAG.
At the time of this work, D. Kon
e was employed at EAWAG. The authors would like to thank the
ONAS board and employees for their participation in this assessment study.

References
Balkema, A. J., H. A. Preisig, R. Otterpohl, and F. J. D. Lambert. 2002. “Indicators for the
Sustainability Assessment of Wastewater Treatment Systems.” Urban Water 4: 153
161.
Bassan, M. 2009. “M
ethodologie d’
evaluation des facteurs de succes et d’
echec des stations de
traitement des eaux us
ees et des boues de vidange a grande
echelle.” Masters Thesis, Ecole
Polytechnique F
ed
erale de Lausanne, Switzerland http://infoscience.epfl.ch/record/140429
Ben Mena, S. 2000. “Introduction aux m
ethodes multicriteres d’aide a la d
ecision.” Biotechnology
Agronomy, Society and Environment 4 (2): 83
93.
Brown, C., and A. Holcombe. 2004. “In Pursuit of the Millennium Development Goals in Water and
Sanitation.” Water Policy 6: 263
266.
Cabrera, E. 2008. “Benchmarking in the Water Industry: A Mature Practice?” Water Utility
Management International 3 (2): 5
7.
Diener, S., S. Semiyaga, C. Niwagaba, A. Murray, J. B. Gning, M. Mb
egu
er
e, J. E. Ennin,
C. Zurbr€ugg, and L. Strande. 2014. “A Value Proposition: Resource Recovery From Faecal
Sludge
Can it be the Driver for Improved Sanitation?” Resources Conservation & Recycling
88: 32
38.
Dodane, P. H., J. Makboon, and A. Torrens. 2006. Assistance a la mise en place de l’exploitation du
lagunage de la ville de Ouagadougou. Lyon, France: CEMAGREF.
Dodane, P.-H., M. Mb
egu
er
e, O. Sow, and L. Strande. 2012. “Capital and operating costs of full-
scale faecal sludge management and wastewater treatment systems in Dakar, Senegal.”
Environmental Science and Technology 46 (7): 3705
3711.
Fernandes, A. M., P. Kirshen, and R. Vogel. 2005. World Water and Environmental Resources
Congress 2005: Faecal Sludge Management, St Elizabeth, Jamaica. Anchorage, 15
17 May.
Gaulke, L. S., X. Weiyang, A. Scanlon, A. Henck, and T. Hinckley. 2009. “Evaluation Criteria for
Implementation of a Sustainable Sanitation and Wastewater Treatment System at Jiuzhaigou
National Park, Sichuan Province, China.” Environmental Management 45 (1): 93
104.
Guene, O., C. S. Tour
e, and L. Y. Maystre. 1999. Promotion de l’hygi ene du milieu: une m
ethode
participative. Lausanne, Switzerland: Presses Polytechniques et Universitaires Romandes.
IWMI (International Water Management Institute). 2008. Agenda for Research, Capacity Building &
Action on the Safe Use of Wastewater and Excreta in Agriculture. IWMI, Accra, Ghana. http://
www.iwmi.cgiar.org/research/research-themes/resource-recovery-water-quality-and-health/accra-
consensus/
Kon
e, D. 2010. “Making Urban Excreta and Wastewater Management Contribute to Cities’
Economic Development
A Paradigm Shift.” Water Policy 12 (4): 602
610.
Koottatep, T., N. Surinkul, C. Polprasert, A. S. M. Kamal, D. Kon
e, A. Montangero, U. Heinss, and
M. Strauss. 2005. “Treatment of Septage in Constructed Wetlands in Tropical Climate

1710 M. Bassan et al.

Lessons Learnt From Seven Years of Operation.” Water Science & Technology 51 (9):
119
126.
Kvarnstr€om, E., P. Bracken, A. Ysunza, E. K€arrman, A. Finnson, and D. Saywell. 2004.
“Sustainability Criteria in Sanitation Planning.” Paper presented at the 10th WEDC
International Conference. Ventiane, Lao, October. http://wedc.lboro.ac.uk/resources/
conference/30/Kvarnstrom.pdf
Lennartsson, M., E. Kvarnstr€om, T. Lundberg, J. Buenfil, and R. Sawyer. 2009. Comparing
Sanitation Systems Using Sustainability Criteria. Stockholm: Stockholm Environment Institute.
L€uthi, C., A. Panesar, T. Sch€utze, A. Norstr€om, J. McConville, J. Parkinson, D. Saywell, and
I. Rahul. 2011. Sustainable Sanitation in Cities: A Framework for Action. Rijswijk, The
Netherlands: Sustainable Sanitation Alliance.
Mara, D., and G. Alabaster. 2008. “A New Paradigm for Low-Cost Urban Water Supplies and
Sanitation in Developing Countries.” Water Policy 10: 119
129.
Massoud, M. A., A. Tarhini, and J. A. Nasr. 2009. “Decentralized Approaches to Wastewater
Treatment and Management: Applicability in Developing Countries.” Journal of
Environmental Management 90: 652
659.
Maystre, L. Y., J. Pictet and J. Simos. 1994. M
ethodes multicrit
eres ELECTRE. Description,
conseils pratiques et cas d’application a la gestion environnementale. Lausanne, Switzerland:
Presses Polytechniques et Universitaires Romandes.
Murray, A., and P. Drechsel. 2011. “Why Do Some Wastewater Treatment Facilities Work When
the Majority Fail? Case Study From the Sanitation Sector in Ghana.” Waterlines 30 (2):
135
149.
NETSSAF. 2006. Criteria for the evaluation and classification of conventional and innovative low
cost sanitation technologies. http://www.susana.org/docs_ccbk/susana_download/2-592-netssaf-
2006-criteria-evaluation-sanitation-en.pdf
Nikiema, J., A. Figoli, N. Weissenbacher, G. Langergraber, B. Marrot, and P. Moulin. 2013.
“Wastewater Treatment Practices in Africa
Experiences From Seven Countries.” Sustainable
Sanitation Practices 14: 26
34.
Oliveira, S. C., and M. von Sperling. 2011. “Performance Evaluation of Different Wastewater
Treatment Technologies Operating in a Developing Country.” Journal of Water, Sanitation and
Hygiene for Development 1 (1): 37
56.
Pybus, P., and G. Schoeman. 2001. “Performance Indicators in Water and Sanitation for Developing
Areas.” Water Science and Technology 44 (6): 127
134.
Schutte, C. F. 2001. “Managing Water Supply and Sanitation Services to Developing Communities:
Key Success Factors.” Water Science and Technology 44 (6): 155
162.
Sh€arlig, A. 1985. D
ecider sur plusieurs criteres: Panorama de l’aide a la d

ecision multicrit
ere.
Lausanne, Switzerland: Presses Polytechniques et Universitaires Romandes.
Strande, L., M. Ronteltap, and D. Brdjanovic. eds. 2014. Faecal Sludge Management: Systems
Approach for Implementation and Operation. London: IWA Publishing.
Strauss, M., D. Kon
e, and A. Montangero. 2003. Recherche appliqu
ee dans le domaine de la gestion
des boues de vidange dans les pays en d
eveloppement - Probl

ematique, questions et aperçu du
projet. D€ubendorf, Switzerland: EAWAG.
Sujaritpong, S., and V. Nitivattananon. 2009. “Factors Influencing Wastewater Management
Performance: Case Study of Housing Estates in Suburban Bangkok, Thailand.” Journal of
Environmental Management 90 (1): 455
465.
van den Berg, C., and A. Danilenko. 2008. “IBNET
A Global Database of the Water Sector’s
Performance.” Water Utility Management International 3 (2): 8
10.
Weissenbacher, N., J. Nikiema, M. Garfi, and A. Figoli. 2013. “What Do We Require From Water
Biotechnologies in Africa?” Sustainable Sanitation Practices 14: 35
40.
Zhou, J. B., M. M. Jiang, B. Chen, and G. Q. Chen. 2009. “Energy Evaluations for Constructed
Wetland and Conventional Wastewater Treatments.” Communication in Nonlinear Science and
Numerical Simulation 14 (5): 1781
1789.
Zopounidis, C., and P. M. Pardalos. 2010. Handbook of Multicriteria Analysis (Applied Optimization).
Berlin: Springer.