Value For Money Assessment Oxfam in South Africa: Australia Africa Community Engagement Scheme (AACES) Capacity Building Component

Oxfam in South Africa
Australia Africa Community Engagement Scheme (AACES)

Capacity building component
www.oxfam.org.au
Nyanga Village, Barotse Flood Plains,
Zambia- The community gather to meet
with Oxfam and the visiting partners and
discuss the WASH program in the village.
Photo Alexia Webster

Title: Value for Money Assessment Oxfam South Africa Australia Africa Community
Engagement Scheme (AACES) Capacity Building component
Published: April 2016
ISBN: 978-0-620-70548-6
Author: Michelle Besley, Independent Consultant
Design: LUMO design & illustration (www.lumo.co.za)
Copyright: Oxfam gives permission for excerpts from this paper to be photocopied or reproduced provided that
the source is clearly and properly acknowledged. Suggested citation Besley, M. (2016) Value for
Money Assessment Oxfam South Africa Australia Africa Community Engagement Scheme (AACES)
Capacity Building component, Oxfam.
Disclaimer: The views in this publication are those of the respective author and do not necessarily represent
those of Oxfam, The Australian Department of Foreign Aid and Trade or any funding agency.
Contact Details: Oxfam OXFAM AUSTRALIA

Suite 1B, Strathway Building 132 Leicester Street
Strathmore Office Park Carlton 3053
305 Musgrave Road Victoria, Australia
Durban, South Africa +61 3 9289 4444
+27 (0) 31 201 0865 [email protected]
[email protected] www.oxfam.org.au
Supported by:
This research was funded under the Australia Africa Community Engagement Scheme (AACES).
AACES is a partnership of DFAT (Department of Foreign Aid and Trade), ten Australian
Non-Governmental Organisations (NGOs) and their Africa-based partners. It contributes to
the DFAT strategy for Africa through community-based interventions across the sectors of
food security, maternal and child health and water, as well as sanitation and hygiene. The
program focuses on marginalised communities, with particular attention to women, children,
people with disability and people vulnerable to disaster.
Commissioned
and published by:
1
Abbreviations 3
Executive Summary 4
Oxfam Australias definition of Value For Money 5
Development of a Value For Money assessment framework 6
Program Assessment Summary 8

Program background 9
Value For Money assessment process 9
Summary of key findings 10
Key learnings 12
Key recommendations 13
Full Value For Money Assessment 14

Program information 15
Assessment summary 16
Component 1: Economy (Costs and inputs: What went in?) 18
Component 2: Efficiency (Inputs to outputs: What happened?) 22
Component 3: Effectiveness (Inputs to outputs: What happended?) 28
Component 4: Equity (Equal inclusion of vulnerable groups) 34
Methodological Rationale and Theoretical Underpinnings

of the Oxfam Australia Value For Money Assessment Tool 40
Purpose 41
The approach 41
Defining the value of VFM 41
Placing theory of change at the heart of VFM 42
Comparison as a means of assessment 42
Leveraging as an indicator of VFM 43
Assessing scale and depth of change in VFM 43
Participation of partners and community in assessment of VFM 44
Evidence-based VFM assessment 44
2
AACES Australia Africa Community Engagement Scheme
ATSIPP Aboriginal and Torres Strait Islander Peoples Program
ACRP Australian Community Rehabilitation Program
BER Basic efficiency resource
ECD Early childhood development
ToC Theory of change
DFAT Department of Foreign Affairs and Trade
KAP Knowledge, attitude and practices
MDIC Maputa Land Development and Information Centre
MEL Monitoring, evaluation and learning
MTR Mid-term review
NGO Non-government organisation
OGB Oxfam Great Britain
RAPCAN Resources Aimed at the Prevention of Child Abuse and Neglect
SROI Social return on investment
TU Tholulwazi Uzivikele
VfM Value for Money
WASH Water, sanitation and hygiene
3
Oxfam Australias definition of value for money
Oxfam Australia defines Value for Money (VfM) as the best use of resources to contribute to positive significant
change in the most vulnerable peoples lives. Significant change includes:
consideration of scale (the number of people benefitting);

depth (the intensity and sustainability of change); and
inclusion (the change benefits people who are vulnerable and marginalised).
VfM has four core dimensions referred to as the four Es:
Economy Costs and inputs what went in?

Reasonable cost to acquire good quality inputs that will enable realisation
of the desired changes
Efficiency Inputs to outputs what happened?
A measure of productivity; how much you get out in relation to what is put in
(increasing output for a given input, or minimising input for a given output,
with a regard for maintaining quality)
Effectiveness Outputs to outcomes what resulted?
Is the program generating positive and sustainable outcomes?
Equity Equal inclusion of vulnerable groups

Is the program appropriately engaging and producing equal benefits for
different groups?
Within the concept of VfM the four Es are interdependent, such that VfM cannot be determined in the absence
of one or more of the four Es. Oxfams approach is weighted towards effectiveness (delivery of desired outcomes
and sustained impacts) and equity (benefits are distributed fairly).
5
Development of a value for money assessment framework
Oxfam Australia sought to progress its understanding of VfM and how it can be practically applied within its
programs. In 2015 it engaged Michelle Besley, an independent consultant, to work with three program teams to
facilitate reflection on the VfM of their programs, and conduct three independent VfM assessments:
Oxfam in South Africa Australia Africa Community Engagement Scheme (AACES)

capacity development support to partners component.
Oxfam Sri Lanka Australian Community Rehabilitation Program (ACRP3)

institutional strengthening support to Community Based Organisation partners in Eastern Sri Lanka component.
Oxfam Australias Aboriginal and Torres Strait Islander Peoples Program (ATSIPP) Straight Talk
whole of program.
The VfM tool was developed iteratively as the assessments were undertaken by testing its application in different
ways and in different contexts. Assessment was based on evidence obtained through traditional evaluation
methods such as desktop reviews, stakeholder interviews and questionnaires. The process explored the use
of different methods and ways to incorporate different stakeholder perspectives on the VfM of these programs.
Some assessments used highly participatory methods, such as partner workshops whereby partners themselves
rated Oxfams performance, and some assessments were conducted more remotely and informed by desktop
review and staff focus group discussions.
The three VfM assessment processes were tailored according to the different contexts and needs of program
teams. For the Oxfam in South Africa AACES and Oxfam Sri Lanka ARCP3 VfM assessments, formal assessments
with ratings were undertaken. This was because these programs were nearing completion and there were
internal and donor imperatives to formally assess VfM. The ATSIPP Straight Talk assessment was completed
without ratings. This was because the process had a focus on learning and was carried out as the team wanted
to get a sense of how the program was tracking on VfM and explore how VfM could be better incorporated into the
programs upcoming re-design.
Assessment is made against two sub-criteria within each of the four Es (eight sub-criteria in total). These criteria
were developed through a consideration of how VfM can best be assessed in relation to Oxfams particular
development approach and programming context. Each of these sub-criteria are then aggregated to produce
an overall VfM proposition. The assessment framework recognises that an assessment can only be made on the
basis of evidence; it does not make an unsubstantiated judgement due to lack of evidence or penalise programs
by rating VfM as poor due to a lack of data. Rather, the assessment notes the extent to which the judgement
is well supported by evidence, or if a tentative judgement is made noting gaps in data. When insufficient
information is available to enable an informed judgement to be made, this is stated. As with many evaluation
methodologies, the judgement is ultimately the subjective interpretation of the evaluator.
6
The approach recognises that Oxfam delivers programs in complex settings, and that context specific
factors determine VfM options and considerations. For example, contextual factors may have implications
for the different components such as costs of operating in specific countries and presence and capacity
of other in-country actors that a program engages with. These factors may make it costlier or difficult to
implement activities or achieve outcomes, or conversely make it more affordable and easier to implement
programs and achieve outcomes. A program should therefore only be judged in relation to what it can
feasibly and realistically achieve in light of these factors. The approach does not attempt to compare
across contexts or programs; assessment is based on how well the program has been designed and
delivered to bring about changes within its scope of control in light of the level of investment, the operating
context and what alternative approaches may have been possible. Additional information on the theoretical
underpinnings of the tool is provided on page 41.
VfM can be approached from a compliance and/or quality perspective. The process adopted sought to
apply a VfM lens to see if this could enhance program quality and improvement. As such, the tool seeks to
draw out learnings and recommendations related to the VfM of a program and how it can be improved. It
also aims to strengthen the assessment of costs and investment, which is often lacking in program review,
and help teams to better consider cost and investment options and make evidence-based VfM decisions.
The process recognises that assessment of VfM can only be made if appropriate data across the four Es is
available. The approach therefore uses VfM assessment as an opportunity to help teams strengthen their
Monitoring, Evaluation and Learning (MEL) frameworks to collect particular information to inform on VfM in
the future, and better manage for VfM.
7
Program background
Australia Africa Community Engagement Scheme (AACES) is the largest Australian Department of Foreign Affairs
and Trade (DFAT) program of funding for Australian Non-Government Organisations (NGOs) in Africa. The program
is being implemented over five years (20112016) and has a budget of up to $90 million. The program focuses
on DFAT priority sectors in Africa: food security, water sanitation, and maternal and child health. It targets
marginalised communities in 11 countries with particular attention to women, children, people with disability,
and people vulnerable to disaster. AACES is implemented by ten Australian NGOs through 39 in-Africa partners.
The Oxfam AACES program is managed by the Oxfam in South Africa country office.
The Oxfam AACES program focuses on water, sanitation and hygiene (WASH) in South Africa and Zambia. The
overall goal of Oxfams AACES program is to improve the health and quality of life of poor and vulnerable people
in targeted areas of Zambia and South Africa. Oxfams program in South Africa is delivered through existing
NGO partners that did not previously work in the WASH sector. The program approach has been to support
partners to mainstream WASH into their own organisations and programs. It has used an integrated approach,
supporting partners to develop technical expertise in WASH, but also develop non-WASH related capacity central
to strengthening the organisational architecture that supports and enables partners WASH work (such as
community engagement, advocacy, and monitoring, evaluation and learning). Oxfams role in the partnership has
been bringing partners together and providing opportunities for them to learn collectively to integrate WASH
providing technical expertise in WASH and other forms of institutional capacity development support to partners,
and guiding partners to deliver aspects of the program including WASH and donor requirements around inclusion.
Oxfams AACES program in South Africa is delivered by NGO partners who have existing programs in different
thematic areas including food security, HIV and AIDS and child protection. The program initially had six partners
that integrated WASH into their programs: Maputa Land Development and Information Centre (MDIC); Save the
Children KZN; Fancy Stitch; Tholulwazi Uzivikele (TU); OneVoice South Africa; and Woza Moya. Two partners (MDIC
and Fancy Stitch) subsequently exited the program. Two other project partners provide technical support to the
other AACES partners responsible for integrating WASH: CREATE, who provides support in disability inclusion, and
Resources Aimed at the Prevention of Child Abuse and Neglect (RAPCAN), who provides support in child protection.
Value for money assessment process
Oxfams AACES program in South Africa was selected for this process as the program team sought to build
on previous work undertaken around VfM as part of the AACES Mid-Term Review (MTR). It also saw this as an
opportunity to undertake in-depth analysis of its capacity building work to AACES NGO partners and identify
learnings and recommendations to take forward during the final phase of the program and use in the end of
program report.
Due to the large scale and complexity of the program, clear parameters were put in place to make the VfM
assessment manageable. This VfM assessment examines the capacity development support provided by Oxfam in
South Africa to implementing partners through the AACES program. It does not aim to provide a VfM assessment
across the whole AACES program in South Africa. The assessment was undertaken towards the end of the
program period, prior to the final program report. It was carried out by an external consultant and provides an
independent assessment. The assessment used common evaluative methodologies including: a desktop review
of program documentation; a Focus Group Discussion with Oxfam South Africa program staff; and an NGO partner
questionnaire completed by five AACES NGO partners. The questionnaire sought partner feedback regarding the
quality, appropriateness and impact of Oxfams capacity building support.
9
Summary of key findings
The overall Value for Money of the AACES capacity building program component is excellent. Oxfam in South Africa
has invested and appropriately used resources to ensure it has successfully built the capacity of its NGO partners
to integrate WASH into their programs. It has selected the right partners who were able to reach and provide
WASH services to a range of vulnerable groups and supported them to link and learn collectively to integrate
WASH. The programming approach of supporting the wider organisational capacity development of partners in
rights-based community programming and advocacy, rather than solely focusing on building partners technical
expertise in WASH was found to be appropriate. As a result, the investment in non-WASH related organisational
capacity building support was central to strengthening the organisational architecture that supports partners
to effectively deliver WASH. Overall, the investment has been well directed towards the achievement of
program objectives.
The strong and respectful partnership established between Oxfam and NGO partners, and between NGO partners,
has supported the efficient delivery of the program. The level of trust among partners and willingness to
share challenges and solutions in inclusive WASH programs has supported a focus on program delivery and
collaboration for practical program improvement. Internal leveraging of partner knowledge, skills and resources
within the program has been extremely high, with several partners going on to form collaborations outside of
the program. Partners have shared their internal systems and processes with each other to support collective
improvement. Failures in WASH systems have been shared, allowing partners to avoid mistakes. In the same way,
successful technologies and innovations, such as tippy taps, that have emerged through the program have
also been shared. In some instances, the integration of WASH into partners programs and the capacity
building inputs provided by Oxfam has enabled partners to leverage additional external support to further
benefit their communities.
The program approach adopted has been highly effective. There is clear evidence to indicate that partners
understand WASH and have contextualised it to their programming contexts. This is significant given that initially
partners reported having little knowledge of WASH and how to integrate it, and encountered resistance to it
within their own organisations. The capacity building inputs have clearly cascaded to benefit communities, with
partners bringing about a range of WASH outcomes for communities, including changes in WASH knowledge and
practices and increased access to WASH services. The program design supports a high level of sustainability
by using an integrated approach which aims to ensure NGO partners can continue to deliver WASH after the
partnership with Oxfam has come to an end. The indications that WASH will continue to be integral to NGO partner
organisations and their programs are high. However, some constraints to sustainability were identified including
the ability of some partners to ensure duty bearers provide and maintain WASH infrastructure. The assessment
highlights opportunities to continue to address sustainability issues in the final phase of this program.
Reduced inequalities in WASH is a program objective that has been well facilitated by the program, with
comprehensive data provided on the many benefits experienced by women, children and people with disability
across the program. Overall, the program has performed well in supporting partners to ensure WASH programs
equitably target and reach vulnerable groups. Significant progress has been made in relation to progressing
disability inclusion and child protection which have been standout features of the program. This has been
brought about as a result of DFAT requirements in these areas and of Oxfam in South Africas appropriate
engagement of technical support to partners provided consistently over the course of the partnership. While
men and womens access rates to WASH have increased at comparable levels, tackling gender issues has been
acknowledged as a challenge among Oxfam and partners. While efforts were made to address gender gaps,
sufficient progress and success has not resulted in this area.
10
Bongi Zuma from from CREATE who
visited the Plan Uganda field visit.
Photo Wendy Lubbee

Key learnings
Time was invested in supporting partners to explore and understand the AACES WASH theory of change
(ToC), with partners discussing the pathways and how their work contributes to the desired outcomes.
This process supported partners to develop a shared understanding of the program and see the connections
between activities which supported an integrated approach.
All capacity building outcomes were systematically and carefully linked to program outcomes, with the ToC
being viewed when determining capacity building inputs and priorities.
Tailoring capacity building assistance to enable partners to integrate WASH into their programs with a relative
focus on both software and hardware aspects of WASH, and strengthen their wider organisational capacity,
has been an effective strategy.
Partners placed a high degree of trust in Oxfam at the beginning of the program, allowing it to support and
guide partners to meet donor requirements, with partners initially not knowing how requirements could be met.
Oxfams readiness may not be the same as partners readiness partners may be in different places in terms
of starting work, as well as in the ability to navigate requirements when defining capacity building activities
and pace of work; knowing when Oxfam was overstepping and needed to pull back was essential to
maintaining a successful partnership.
The design of the program with a focus on collective learning and sharing between partners has been
essential. The level of trust built among partners through the program has supported learning, as partners
have engaged in an open and non-competitive manner.
Providing ongoing linking and learning activities such as annual reflections and planning meetings, the
WASH group and field visits has enabled rich exchanges between partners which have led to new ideas and
ways of implementing WASH programs.
Use of external national consultants has brought a wealth of knowledge relating to their experience of other
program approaches and strategies in South Africa that were of great benefit to the program.
Engaging national consultants to work with and mentor partner organisation staff continuously over time
has helped to embed learning and capacity, both individually and institutionally.
The program provided a platform between partners to meet and develop strategic partnerships
the forming of strong partnerships between AACES NGO partners supports the potential sustainability of
outcomes, as support networks have been created which may endure.
Including an advocacy component is essential to ensuring the sustainability of the program; it has been
challenging to introduce advocacy activities due to an initial negative perception around this term and form
of engagement with government.
Having a specific objective around equity (ie reducing inequalities in WASH) has focused partners in working
towards inclusion.
Giving technical service providers in disability inclusion and child protection the freedom to decide what
supports are necessary, rather than providing blanket requirements, has been appropriate.
Having Oxfam staff learn with and alongside partners demonstrates leadership and commitment to
supporting inclusion.
12
Key recommendations
Recommendations for the remainder of the program period:
Consolidate data across all AACES partners to provide a comprehensive picture of the impact of the program
and the extent to which it has benefited different groups in the final report.
Document the most and least appropriate and cost-effective WASH technologies and reasons for this
and capture important technical learnings that can be applied in future WASH programs.
Work with the disability inclusion partner CREATE more strongly, so as to review the disability content in
partner reports and strengthen Oxfams own understanding of disability issues and ability to provide technical
analysis, and take greater steps to progress its own disability inclusive practice at all levels within the
organisation as partners have done.
Undertake a sustainability mapping of WASH infrastructure which outlines the requirements and avenues of
each partner to receive, maintain and repair WASH hardware in the future.
Support partners to form links with external WASH hardware suppliers (government and private sector)
exploring a range of avenues for sustainability.
Continue working with advocacy partners that support and mentor AACES partners to continue to build their
advocacy WASH work.
Recommendations for future programming:
Ensure a strong induction/orientation process in the early interactions between partners which gives them
opportunity to raise questions and develop an understanding of the program.
Provide partners with an annual plan of events in advance to allow for better planning and coordination with
field activities.
Support additional exchange visits at the community level to allow partners to see first-hand examples of
how effective WASH committees operate and pose questions to stakeholders directly, such as: how they
began; how they were formed; how they operate; what works well and what does not.
Bring Oxfam in South Africas internal expertise in gender (gained through its experience of delivering
other programs in South Africa over the last decade) to bear on AACES, in addition to putting in place other
strategies to address gender issues.
Ensure outcomes can be realistically achieved in the program period (i.e. ability of community and NGO
partners to influence duty bearers to provide and maintain WASH infrastructure) by outlining what is inside
and outside of the programs sphere of influence, and what changes may require a longer time frame.
Factor sustainability of WASH outcomes from the outset and ensure appropriate and sufficient investment
in partner capacity to source and receive WASH services and infrastructure at the end of the program.
13
Program information
Program name: Australia Africa Community Engagement Scheme (AACES)
Program type: Capacity building for inclusive water, sanitation and hygiene (WASH) programming
Region: Southern Africa
Country: South Africa
Program period: 20112016
Stage in program cycle: End of the program cycle
Theory of change in place: Yes (initial ToC and revised updated ToC in 2014)
Program cost: AUD $2,628,779 (ZAR 6,469,709) capacity building support
Improving the health and quality of life of the poor and vulnerable people
Goal of program:
in targeted areas of Zambia and South Africa.
Investment located
NGO
(Individual, community, NGO):
Attachments: Financial summary
Overall VfM rating
35
30
25
20
15
10
Poor VfM Low VfM Fair VfM Good VfM High VfM
15
Assessment summary
The overall Value for Money rating for the program component is excellent. Oxfam in South Africa has invested
and appropriately used resources to ensure it has consistently and successfully built the capacity of its NGO
partners to integrate WASH into their programs. It has used resources to support partners to link and learn
collectively to integrate WASH (appropriately programming in relation to Oxfams expertise and value-add) and
engaged appropriate local consultants to provide technical expertise in WASH and other forms of support. The
program was managed responsively; when gaps were identified in relation to required inputs, or existing inputs
were found to be inappropriate, the investment was redirected to support needs and achievement of program
objectives. The large investment in non-WASH related organisational capacity building support was central to
strengthening the organisational architecture that supports and enables partners WASH work.
The strong and respectful partnership established (between Oxfam and NGO partners, and between NGO partners)
has supported both the efficient delivery of the program and a focus on program delivery and collaboration
for practical program improvement. Internal leveraging of partner knowledge, skills and resources within the
program has been extremely high, with several partners going on to form collaborations outside of the program.
In some instances, the integration of WASH into partners programs and the capacity building inputs provided by
Oxfam has enabled partners to leverage additional external support.
There is clear evidence to indicate that partners understand WASH and have contextualised it to their
programming contexts and communities in which they work. This is significant given that initially partners
reported having little knowledge of WASH and how to integrate it, and encountered resistance to it within their
own organisations. The capacity building inputs have clearly cascaded to benefit communities, with partners
bringing about a range of WASH outcomes for communities, including changes in WASH knowledge and practices
and increased access to WASH services. The indications that WASH will continue to be integral to NGO partner
programs/organisations is high. The program design supports a high level of sustainability by using an integrated
approach which aims to ensure NGO partners can continue to deliver WASH after the partnership with Oxfam has
come to an end. Some constraints to sustainability were identified, with partners ability to ensure duty bearers
provide and maintain WASH infrastructure unclear at the point of this assessment. Oxfam has made efforts to
ensure sustainability, such as supporting partners to develop sustainability plans, developing an advocacy policy
and forming links with advocacy organisations. The assessment highlights opportunities to continue to address
sustainability issues in the final phase of this program.
Overall, the program has performed well in supporting partners to ensure WASH programs effectively target
and reach vulnerable groups. Significant progress has been made in relation to progressing disability inclusion
and child protection, which have been standout features of the program. Gender has been acknowledged as a
challenge among Oxfam and partners throughout the program. While efforts were made to address gender gaps,
the program has not made sufficient progress in this area. Reduced inequalities in WASH is a program objective
and has clearly been facilitated by the program, with comprehensive data provided on the many benefits
experienced by women, children and people with disability across the program.
16
Summary of VfM component rankings
10
High Economy High Efficiency Good Effectiveness Good Equity
Economy high (6/6) 6 Informed judgement

Sub-component 1: theory of change 3/3 well supported by a
Sub-component 2: competitiveness 3/3 range of quality evidence
Efficiency high (6/6) 6 Informed judgement

Sub-component 1: productivity 3/3 well supported by a
Sub-component 2: leveraging 3/3 range of quality evidence
Effectiveness good (9/12) 9 Tentative judgement

Sub-component 1: reach and depth of change 6/6 made on the balance of evidence,
Sub-component 2: sustainability 3/6 noting some gaps
Equity good (9/12) 9 Tentative judgement

Sub-component 1: equity of process 3/6 made on the balance of evidence,
Sub-component 2: equity of outcomes 6/6 noting some gaps
17
Summary of VfM component rankings
COMPONENT 1: ECONOMY
(costs and inputs: what went in?)
Reasonable cost to acquire good quality inputs that will enable

realisation of the desired changes
Relevant contextual factors and implications for investment options

Government provision of WASH services in South Africa is relatively weak.
The South Africa component was part of a broader program also implemented in Zambia.
Economy sub-category 1: theory of change (ToC)
ASSESSMENT QUESTIONS:
Has Oxfam invested in the right types of partners to achieve the change sought?
Oxfam engaged seven NGO partners with different sectoral focuses to integrate WASH. Two partners exited the
program during the partnership. The other five partners have continued, and successfully integrated WASH. All
partners reported that WASH aligns well with their program activities. One partner stated, WASH marries perfectly
with the concept of addressing the holistic needs of the young children while at the same time ensuring that the
young child is accessing a quality learning environment. It has suggested and exposed us to areas of the young
childs needs that we would not previously have considered. This indicates that Oxfam has selected partners
with a shared vision and way of working and reinforces the appropriateness of the premise of the programs
integrated approach and the ability of the program to deliver this. Where this was not the case (of the two
partners), appropriate arrangements were made to exit the partnership.
Partners are NGOs who serve different target groups within different communities and work in different sectors.
As NGOs have groups of community workers, conducting home and school visits in the regions they serve, they
are able to reach a large number of people at the community level, often in remote areas. This has allowed Oxfam
to extend WASH outcomes to diverse community groups (as outlined in Component 3: effectiveness reach and
depth). Arguably, Oxfam has selected the right partners if NGO partners are able to deliver WASH outcomes to
vulnerable communities in an inclusive manner, and if these NGOs continue to provide WASH services.
Challenges around the ability of partners to engage government and duty bearers to deliver WASH services were
identified during the program. An objective in this area was subsequently integrated into the revised ToC, with
the program aiming to support women and men to influence and claim their rights and a WASH influencing strategy
being developed in 2014 to further this objective. Anecdotal discussions with staff indicate that partners have started
to advocate for WASH services and infrastructure on a local level with community members taking some responsibility
in this area, but that this has not resulted in communities ability to effectively claim their rights in this area.
It is not currently clear to what extent sustainability of WASH outcomes is contingent upon the ability of AACES
NGO partners to advocate for and receive WASH services. If WASH infrastructure cannot be maintained, a
gap in the ToC may be evident (i.e. advocacy outcomes may be unrealistic or investment insufficient). This is
discussed in greater depth in the section below: effectiveness sustainability. When this challenge emerged,
Oxfam management responded by making greater investment mid-way through the program (and thereafter) to
strengthen advocacy work. In year four, Oxfam brought Equal Education an advocacy organisation that focuses
18
on the school sector into the partnership. One of their current programs is around school infrastructure, and
they have proven to be a good organisation to engage with to bring about advocacy level outcomes.
Has Oxfam used the right resources to support the changes sought? (Has it invested appropriately and sufficiently?)
Oxfams investment has involved 1) using internal resources to support partners to link and learn collectively
to integrate WASH, and 2) engaging consultants to provide technical expertise in WASH and other forms of
support to partners. Oxfams role and value-add in the program has been: holding an overarching vision of
the program; bringing partners together to learn and share; supporting and evolving the partnership; guiding
partners to deliver aspects of the program (WASH and other program requirements); and providing organisation
strengthening support.
As outlined in greater detail in both Component 2: efficiency, and Component 3: effectiveness, the inputs and
resources used to bring about change (linking and learning and engaging consultants to provide technical
support) have been largely appropriate. While the inputs sources were the right ones needed to make the
change, some inputs were not delivered appropriately as planned, resulting in diminished levels of change
in some areas. Inputs identified by some partners as being inappropriate and ineffectual include: the gender
training (which was found to be too theoretical with little relevance to work on the ground); Participatory Rural
Appraisal and fundraising activities (as insufficient time was allocated); and story gathering processes. In
regards to gender training, a pilot exercise with a gender expert was implemented and because the results were
extremely poor Oxfam cancelled this contract and did not roll it out any further due to the large disconnect.
The program was not only about WASH, but was designed to strengthen the capacity of partners in a range of
ways, having added benefits outside of WASH. There was a large investment in non-WASH specific organisational
capacity building support, such as MEL (ie data collection, development of baselines, knowledge, attitude and
practices (KAP) methods), community engagement, participation, awareness raising, disability inclusion, program
management, reporting and advocacy, which supported partners ability to deliver, monitor and report on WASH
work by strengthening the organisational architecture that supports WASH work. It also allowed partners to apply
this new capacity across other thematic areas, having an extended impact beyond WASH. Arguably, this model
was more effective in creating interest and buy-in among partners than a model with an exclusive focus on WASH
capacity building inputs, which while may have been a less resource-intensive approach, is unlikely to have
produced the same level of outcomes.
LEARNING FOR IMPROVEMENT QUESTIONS:
What learnings does the program hold for constructing the ToC and ensuring investment is appropriately aligned?
While the programs ToC was clearly articulated in the program design document, it was not socialised well
with partners because WASH was new to both Oxfam in South Africa and the partners and it was not clear
what areas of WASH partners would want to focus on in their communities across two countries. However, as
it was not socialised upfront time was wasted early on as partners struggled to understand the program.
After the mid-term review it was clear that the ToC had to be re-visited and mapped against the program
objectives with partners identifying their own pathways of change and mapping them. This process supported
partners to develop a shared understanding of the program and see the connections between activities
which supported an integrated approach.
Time was invested in supporting partners to explore and understanding the AACES WASH ToC and high-level
indicators, with partners discussing the pathways and how their work contributes to the desired outcomes.
All capacity building outcomes were systematically and carefully linked to program outcomes, with the ToC
being viewed when determining capacity building inputs and priorities.
As AACES was part of the wider No Longer Vulnerable program, partners were linked in to a larger program
and network and received additional support.
19
RECOMMENDATION QUESTIONS (CURRENT/FUTURE PROGRAM):
How could resources be better directed to achieve the programs intended outcomes?
Factoring sustainability of WASH outcomes from the beginning and ensuring appropriate and sufficient
investment in partner capacity to source and receive WASH services and infrastructure at the end of the program.
Ensuring outcomes can be realistically achieved at the end of the program (i.e. capacity of men and women
to influence duty bearers) by clearly outlining what is in the sphere of influence of Oxfam and partners and
what is outside of it (and what may require a longer time frame).
Advocacy type partner brought into the program from the outset to share their experiences early on and
provide further support many of the partners were traditionally involved in service delivery in the areas of
health, social protection, and food security so their focus was more on filling the gap when government was
failing to provide essential services. Oxfam could have provided more training earlier in the program on the
rights based approach, assisting partners to shift their thinking regarding their roles and responsibilities
versus the governments.
Economy sub-category 2: competitiveness
Have costs been kept reasonably low while maintaining quality? (Outline in relation to other comparable
options available to support the same changes OR the way in which the program has been implemented so
reducing or increasing costs/inputs).
Costs have been reasonable and appropriate and on the whole a high level of program quality has been
maintained. Internal Oxfam resources have been used to support linking and learning which has been highly
effective. These activities have used 1.24% of the total budget (this does not include investment of staff time
and partner staff time). This demonstrates how a relatively small financial investment can have a significant
impact on effectiveness. High quality national consultants have been engaged to provide technical support and
local South African NGOs were engaged to provide support on child protection and disability; no international
consultants were engaged, keeping costs down.
Of the capacity building budget, 3.12% has been spent on capacity building in MEL, which has been a relatively
small area of investment which has been of high value to all partners. The bulk of capacity building funds have
been spent on joint technical training of staff from multiple organisations (40.82%) and coaching and mentoring
of NGO staff (37.32%); noting a few exceptions, the technical support provided by partners through these
activities was reported to be instrumental in assisting partners to strengthen their organisational capacity.
Exchange visits were provided in region, allowing partners to learn practically from other WASH programs
implemented in similar contexts.
Have funding arrangements allowed costs and inputs to be used as necessary according to the program context?
The AACES program afforded a good amount of flexibility. Oxfam staff noted that they were able to cancel
planned activities if they no longer made sense, and be responsive to needs and requests from partners.
DFAT did not institute compliance guidelines which enabled Oxfam and partners to determine their own
implementation pathways.
20
Economy rating
Overall economy rating: high economy 6/6

(2 poor economy, 3 low economy, 4 fair economy, 5 good economy, 6 high economy)
Theory of change: 3/3 Competitiveness: 3/3

How well has the program directed resources to Have costs been kept reasonably low while
bring about planned outcomes? maintaining quality?
It is not clear if resources have been closely linked It is not clear if resources have been kept low and
U U
to outcomes; a judgement cannot be formed quality maintained; a judgement cannot be formed
Investment directed to achieve changes is Program costs are either unreasonably high or
1 1
largely inappropriate too low, affecting quality
Some forms of investment directed to achieve Some forms of investment are reasonable, some
2 2
changes are appropriate, with scope for improvement costs could have been reduced or redirected
Investment directed to achieve changes is Investment is largely reasonable and quality

3 3
largely appropriate maintained
Strength of evidence
Poor evidence: Judgement made on best available data, not well supported by evidence
Average evidence: Tentative judgement made on the balance of evidence, noting some gaps
Excellent evidence: Informed judgement well supported by a range of quality evidence
List the primary sources of information used to inform the assessment

Program budget
Theory of Change document (diagram and narrative)
MEL capacity building training report
AACES mid-term review
Operational plans (detailing capacity building support activities)
Oxfam WASH influencing strategy
Focus Group Discussion with staff
List the strengths in the information and how they enabled assessment
Comprehensive financial data received with a breakdown of what investments were made in different
capacity building strategies
2014 ToC is robust and detailed, supporting a clearer understanding of the program and the perceived
pathways for achieving change
Note gaps in MEL which made it difficult to make judgement, or what additional information would have
helped to make a more robust and nuanced assessment
Investment of staff time would provide a more comprehensive assessment of investment in linking
and learning
Limited documentation of a sustainability assessment which outlines which WASH services and
infrastructure each partner can source and maintain internally (and is therefore sustainable), and what
external support is required by partners to maintain and continue WASH work (and is therefore potentially
unsustainable if there is not a clear process in place to ensure this is received)
21
COMPONENT 2: EFFICIENCY
(Inputs to outputs: What happened?)
A measure of productivity how much you get out in relation to what is

put in (increasing output for a given input, or minimising input for a given
output, with a regard for maintaining quality)
Relevant contextual factors and implications for efficiency

Oxfam had worked with AACES NGO partners prior to the program and had already established good
working relationships allowing for a relatively high degree of trust.
South Africa has a high level of national technical expertise making it possible to engage quality
local consultants.
Oxfam in South Africa was engaged in the Oxfam 2020 process and was informed of the closure of their
office during the implementation of ACCES resulting in management having to navigate internal agency
processes and manage its own exit during the implementation of the program.
A relatively high number of donor requirements were attached to AACES funding requiring Oxfam and
partners to navigate and develop capacity in a range of new areas for the first time (i.e. disability inclusion
and child protection).
AACES partners are working in different regions and sectors with different target groups and different levels
of capacity making it difficult to standardise capacity building support.
Efficiency sub-category 1: productivity
How has the project created efficiencies?
The outputs sought relate to building partner capacity to integrate WASH into their organisations and
programs. Oxfams two main delivery approaches are 1) supporting partners to link and learn collectively
to integrate WASH, and 2) engaging consultants to provide technical expertise in WASH and other forms
of support to partners.
With the exception of one partner who reportedly exited the program mid-way due to the burdensome
level of requirements, all partners surveyed noted that the partnership had involved a heavy investment
of their time in capacity building processes, but that this was worth it given the benefits received. All Oxfam
partners stated that on most occasions, the support provided aligned with their needs. There is a breadth
of evidence which demonstrates that Oxfam has been genuinely responsive to partner needs, responding
to specific requests for capacity development support. For example, partners initiated a WASH group in
which they meet every two months to find common ground. This group was reported by partners as being
pivotal to the programs effectiveness and has been supported by Oxfam at the request of partners. Oxfams
flexible and supportive approach has created efficiencies by ensuring capacity building investments are
relevant to partners.
Oxfam has contracted out the majority of capacity building inputs rather than bringing in or using in-house
expertise. Oxfam noted they chose consultants carefully and took their time during the selection process to
make the right choice. This approach of engaging external national consultants appears to be appropriate, given
the need to deliver tailored and flexible technical support to different partners in different areas. With a few
22
exceptions (ie gender consultants), the consultants engaged were reported by partners as being highly skilled
with a good knowledge of the context and an ability to work well with partners. Partners commonly identified
appropriate selection of quality consultants as a key factor underpinning the success of the program.
Oxfam has not seen a high turnover of Oxfam staff and partner staff involved in the program. This has prevented
inefficiencies and allowed for a continuity of programming and a high level of comfort and trust to develop
between Oxfam and partners. Oxfam in South Africa management made a conscious decision not to involve AACES
program staff heavily in the Oxfam Single Management Structure change process. As a result, staff attention
was not diverted and program quality was maintained over the course of the program. Despite the impending
closure of the Oxfam in South Africa office, Oxfam staff have remained dedicated to the program and focused on
supporting partners to complete the program well, working to maximise impact and sustainability.
Partners trialed the use of different WASH infrastructure and technologies in different communities and
undertook exchange visits to share their experiences. This has allowed partners to compare technologies,
avoid making the same mistakes, and determine which are the most suitable and cost-effective solutions.
For example, the construction of the enviroloo was considered too expensive for a mass rollout, and the
installation of boreholes failed three times, while the dripping water system and tippy taps were seen to be
both affordable and effective. Oxfam supported partners to attend the World Toilet Summit, and participate in
exchange visits to Zambia and Malawi to visit other WASH programs implemented through AACES in collaboration
with Oxfam and Water Aid. These were collaborative platforms which facilitated information and knowledge
sharing between Oxfam and various partner NGOs.
How has the project created inefficiencies?
Inefficiencies appear to have been present at the beginning of the program, with several partners reporting
not to have had a clear vision of what they were expected to achieve and what was needed to implement the
program. One partner stated, there was quite a bit of time wasted in the early days with us all foundering around
trying to understand what this new program was and what was expected of us. This indicates that greater
investment was needed in the design and startup phase to ensure a clear and shared understanding of the
program among all partners.
Some partners reported a rush to start implementation of WASH work within their organisations, caused by funder
urgency. This purportedly led to some resistance within their own organisations to the idea of integrating WASH
into programs with a different sectoral focus, with staff not initially seeing the relevance of the approach. This
potentially created inefficiencies as partner staff had to work to overcome resistance created, which may have
been countered by allowing more time for awareness raising and discussion within partner organisations.
What can be learned about trying to produce certain outputs in relation to the context and program type and
the implications of this for investment?
Partners identified the following as key strengths of Oxfams approach: aligning themselves to what NGOs
were doing; agreeing on expectations and targets mutually; being responsive to requests for support; good
communication and willingness to listen; being aware of the capacity of partners; allowing failures; and
assisting partners in supporting each other.
The need to balance software and hardware capacity building support some reported a lack of alignment
with their needs, due to an initial focus on hardware which was not seen as relevant to their context and
therefore not benefiting early on; this was later addressed.
23
The need to take the levels at which partners work, their target groups and implementation approaches into
consideration when defining WASH mainstreaming approaches. One partner noted that as the program had an
initial focus on WASH governance, they perceived Oxfam to be using a top down approach that did not align
with their work at the household and community level.
Trust in Oxfam to navigate requirements both partners and Oxfam reported that a high degree of trust was
placed in Oxfam at the beginning of the program, particularly in relation to supporting and guiding partners to
meet donor requirements (i.e. MEL, disability, child protection, technical aspects of WASH) with partners
initially not knowing how these could be done. One partner stated we trusted that Oxfam knew what was
needed, and went along with the flow in the early years.
Oxfams readiness may not be the same as partners readiness being mindful that partners may be in
different places in terms of starting work and the ability to understand and navigate requirements when
defining capacity building activities and pace of work; knowing when Oxfam was overstepping and needed to
pull back was essential to maintaining a successful partnership.
Engaging consultants to work with and mentor partner staff continuously having consultants continue to
work with partners (often a group of people, not just one staff member) over time helped to embed learning
and capacity, both individually and institutionally.
How could Oxfam produce more or better outputs with a commensurate level of investment?
A stronger induction/orientation process in the early interactions between partners, giving them the
opportunity to raise questions and issues and develop an understanding of the program, would create efficiencies.
Involving partners in the selection of consultants while partners often cited high-quality consultants
as being key to the programs success, they also stated the program would have benefited from their
participation in the selection of all consultants.
Engaging the disability Inclusion support partner more strongly CREATE was engaged to provide technical
support to partners, however it could also have been used to review the disability content in partner reports
and strengthen Oxfams understanding of the issues and ability to provide technical analysis and feedback.
Providing partners with an annual plan of events in advance to allow for better planning and coordination
with field activities.
Documenting the most and least appropriate and cost-effective technologies and reasons for this would
capture important learning that could be applied in future WASH programs.
Efficiency sub-category 2: leveraging
How well has Oxfam strategically used and mobilised resources (i.e. the ideas, skills, funds and resources
of other individuals/organisations, and its own skills, knowledge and networks)? Outline how it has leveraged
and missed opportunities to leverage.
A key strategy of the program has been to provide a platform for NGOs to meet and develop strategic
partnerships. The program has facilitated a high level of engagement between partners, allowing them to learn
and benefit substantially from each others skills and experience. This has been a highly successful strategy
for achieving outcomes and is a program outcome in itself. The partnership (between Oxfam and partners
and between ACCES partners) has developed strongly. This has strengthened WASH outcomes, but has also
facilitated a range of multiplier effects of benefit to partner organisations and communities.
24
Internal leveraging of partner knowledge, skills and resources within the program has been extremely high.
Partners have shared these on a regular basis through the working group and reflection processes where
they have come together to discuss each others work. Partners developed a comprehensive charter to
outline the ways in which they will collaborate with each other. One section includes a commitment to
performing pro bono work for each other and reciprocating services. Partners have shared their internal
systems and processes with each other to support collective improvement, such as innovative WASH
systems, MEL and financial systems. Several AACES partners noted the value of knowledge and experience
shared by Save the Children of WASH in early childhood development centres and of Woza Moya in the
building of tippy taps.
In several instances, NGO partners have gone on to form collaborations outside of the program. For example,
disability inclusion partner CREATE has formed formal partnerships with two AACES NGO partners. For CREATE,
their inclusion in AACES has enabled them to develop a new model for stimulating disability inclusive
development within NGOs, rather than solely functioning as a service provider. One partner has established
a formal link with Save the Children which has led to the establishment of a local ECD forum, independent of
Oxfam funding. This has allowed the organisation to reach more ECD centres (which are isolated and receive
very little government support) than it previously did in a more cost-effective way. This is expected to
result in significantly more children being equipped and prepared for school. AACES partners have also recently
submitted joint funding proposals.
The integration of WASH into partners programs and the capacity building inputs provided by Oxfam has enabled
partners to leverage external support. One partners WASH work in schools has enabled it to link with the
Expanded Public Works Programme based in the municipality. Another partner noted that as a result of Oxfams
capacity development support on collection and presenting of stories, they have become better able to share
information about their program and have obtained direct funding support as a result.
What can be learned about trying to leverage in relation to the context and program type?
The design of the program and focus on collective learning and sharing between partners has been essential
to maximising program efficiency and effectiveness.
The level of trust built among partners through the program has supported learning partners have
engaged openly and in a non-competitive manner which is reportedly rare in the context.
Use of external national consultants consultants reportedly brought a wealth of knowledge relating to
their experience of other program approaches and strategies in South Africa, which was of great benefit
to the program.
How could Oxfam have better leveraged?
The program has facilitated a high degree of internal leveraging (of partners and consultants); there are
therefore no recommendations in this area.
It is not immediately clear if and how Oxfam could have better leveraged internally within Oxfam (such as
by drawing on Oxfam Australia). Possibilities include WASH technical support (which was reportedly requested
and not received) and support with documentation of aspects of its model.
25
It is not immediately clear if and how Oxfam could have better leveraged externally, such as through
engagement of the private sector in support to WASH hardware, as discussion of this issue is not present
in program documentation. It is recommended that Oxfam and partners consider this in discussions
regarding sustainability of WASH infrastructure (as noted below, this has been raised as a potential constraint
to sustainability).
Efficiency rating
Overall efficiency rating: fair efficiency 6/6

(2 poor efficiency, 3 low efficiency, 4 fair efficiency, 5 good efficiency, 6 high efficiency)
Productivity: 3/3 Leveraging: 3/3

How well have efficiencies been created during How well has the program strategically mobilised
program implementation? resources?
It is not clear how the way in which the program
It is not clear what leveraging occurred;
U was implemented has supported efficiency; a U
a judgement cannot be formed
judgement cannot be formed
Program implementation was largely inefficient,
1 1 No/very limited leveraging occurred
affecting program quality
Some efficiencies have been created by the way
Some leveraging occurred and some
2 in which the program has been implemented, 2
opportunities for leveraging were missed
with room for improvement
The ways in which the program has been
Strong leveraging occurred, extending and
3 implemented have been largely efficient and 3
enhancing outputs
supported program quality

Focus Group Discussion with Oxfam staff
Interviews and questionnaire from AACES partners
Operational plans (detailing capacity building support activities)
Program reports and reviews
Documentation of learning events
Partner Charter
Learning events were well documented, outlining the challenges experienced by partners during
implementation and actions taken and achievements
Clear evidence was outlined regarding how partners would work together making it easy to understand
points of intersect and how this would support program delivery
The questionnaire enabled partners to clearly identify the value-add of Oxfam and what had supported
productive working relationships
26
Oxfam in South Africa AACES partners visit
the Nyanga village in Mongu Zambia to
learn about the WASH interventions.

COMPONENT 3: EFFECTIVENESS
(Inputs to outputs: what happened?)
Is the program generating positive and sustainable outcomes?
Relevant contextual factors and implications for effectiveness
Partners had no/limited knowledge of WASH and of how to integrate WASH services as a result partners
were starting from a relatively low base
AACES partners are connected with a wider program (no longer vulnerable) in which capacity building and
reporting are connected
Partners are working in different regions and sectors and with different target groups making it difficult
to standardise and aggregate outcomes
Effectiveness sub-category 1: reach and depth of change
Did the program bring about the outcomes it had sought to?
The capacity development inputs provided through the program have significantly increased the capacity of
AACES partners in a range of ways (related to WASH and other areas). Partners were asked to rate the level of
impact of Oxfams capacity building work on their organisation. One partner rated it as having a medium impact;
the other four partners reported it as having a high impact. Several partners noted it stretched and developed
the capacity of individual staff members and their organisations. As stated by an AACES partner, the capacity
building, networking, linking, and learning together has had a lasting influence. WASH will stay, partnerships
have been formed and minds have been opened.
Most significantly, partners understand WASH and have contextualised it to their programming contexts and the
communities in which they work. Initially they reported having little knowledge of WASH and how to integrate it,
and encountered resistance to it within their own organisations. These issues have clearly been navigated and
addressed, with partners reporting greater confidence and expertise to implement WASH independently. There is
a range of evidence to indicate that partners have successfully mainstreamed WASH by integrating it into their
organisational policies and procedures, and into their existing programs. As one partner stated, empowering
our community care workers around WASH issues has had a direct impact in improving health outcomes in the
homes. Our community care workers now understand the direct link between improved hygiene practices like
hand washing, water storage, waste disposal and the improved health outcomes. They have taught the families
under their care this and we have seen the incidence of diarrhea decreasing in the community.
Did the program produce reasonable reach (numbers of people benefiting) and depth of change (systematic
change) in relation to the level of investment?
The capacity building inputs have clearly cascaded to benefit communities. A range of examples which support
this are documented in reports including: increased community knowledge of safe and clean water and other
WASH aspects; changes in WASH practices resulting in improvements in health; increased access to WASH
services and infrastructure; reduced inequalities to WASH services (outlined in the section below); establishment
of functioning WASH governance structures; and collective local responsibility taken to access, protect, use and
maintain WASH infrastructure by community members.
28
The qualitative data provided indicates a level of systematic change has been achieved. For example,
institutional change in capacity to integrate WASH has occurred among all partners. Quantitative data reveals the
reach facilitated by the program. A total of 11,306 additional people (6,551 women and 4,755 men) are reported
to have access to sustainable and safe water over the life of the partnership. The majority of people (8,082)
received services in July 2012June 2013, with a smaller but consistent number of people obtaining access in
the following two years. This is likely due to the delivery of WASH hardware/infrastructure that took place at the
beginning of the program. A total 9,793 people (5,595 women and 4,198 men) experienced access to appropriate
sanitation. The majority of people (7,757) experienced these benefits from July 2014June 2015, with numbers of
people lower in the previous year of July 2012June 2013 (1,777) and even lower in the subsequent year (259).
After the AACES MTR, the program emphasised the objective related to increasing the capacity of men and women
to claim their rights, respond to external trends and shocks, and influence those with power and to hold duty
bearers to account. While this objective was always present, partners had not made the conceptual shift from
service delivery to building the capacity of their communities to claim their rights. At the time of the assessment,
the extent to which this has been achieved could not be discerned due to lack of data as this assessment was
undertaken prior to the final evaluation.
What lessons can be learned regarding the resourcing of critical pathways/strategies to support change in this
program context?
Providing ongoing linking and learning activities (such as annual reflections, planning meetings, WASH
group and field visits) allowed for rich exchanges between WASH partners, often leading to new ideas and
ways of implementing WASH programs and helping partners to learn from each others challenges and
mistakes and share experiences and expertise.
Providing partners with exchange visits, and the opportunity to see how WASH was being practically taken
forward by others in different countries, was highly valuable these experiences motivated partners and
enabled them to look at what they were doing with new eyes. Several partners noted being impressed by the
proactive roles women were playing in WASH construction in Zambia, and how Oxfam had used the WASH
program to improve womens livelihoods. The actions taken by the Zambia program in skilling communities to
sustain boreholes also impressed partners.
Supporting partners to attend the World Toilet Summit enabled partners to become more familiar with global
innovations in technologies and legislation on sanitation and human rights, with organisations practically
taking forward some of this learning in their own organisations.
The program provided a platform between partners to meet and develop strategic partnerships, as illustrated
by the new formal partnerships formed and joint funding submissions.
Tailored capacity building assistance to enable partners to integrate WASH into their programs (with a
relative focus on both software and hardware aspects of WASH) and strengthen their organisational capacity
in other areas (such as digital media, storytelling, video making and photo stories).
Providing strong support to improve MEL the consultant was able to relate concepts simply and practically,
and taught partners a behavior change model which was found to be useful.
Providing targeted and sustained disability inclusion support has supported partners to understand the need
and importance of integrating disability inclusion and take practical steps to facilitate inclusion.
29
Could Oxfam have invested differently to enhance the quality, scale or depth of outputs/outcomes? Are there
alternative ways in which the program could be implemented that Oxfam should explore? (other models/
strategies used by Oxfam or other agencies)
Partners identified the following ways in which effectiveness may have been enhanced:
Additional exchange visits (ie on the ground, community to community) to allow partners to see first-hand
examples of how effective WASH committees operate and ask questions to stakeholders directly such as:
how they began; how they were formed; how they operate; what works well and what does not.
Undertaking more frequent support visits this would, however, have required additional investment
by Oxfam by either using other internal staff to provide this additional support or engaging short term
external expertise.
Encouraging joint projects between partners earlier while this was noted by a partner, it is not clear if
the level of trust required to enable this would have been present earlier on in the partnership.
Effectiveness sub-category 2: sustainability
Are the outcomes sustainable, or is there evidence to indicate likely sustainability?
The indications that WASH will continue to be integral to NGO partner programs is high. This is because of 1)
institutional integration of WASH among all agencies, 2) knowledge and capacity built around WASH; 3) the
perceived value of WASH held by partner organisations. There is evidence that partners have institutionalised
WASH within their programs. For example, organisations have incorporated WASH content into their learner
manuals and curriculums. Changes in knowledge of WASH have led to attitude and practice changes among
NGO practitioners whereby the community workers of some NGO partners have incorporated WASH teaching and
monitoring into their community visits. Partners have noted the high value of WASH, and how it marries well with,
supports and reinforces their other program activities. This indicates that partners are likely to continue to invest
in WASH without Oxfams external support as they see the alignment and benefits. Quantitative data reveals that
11,306 additional people are reported to have access to sustainable and safe water over the life of the partnership.
Some possible constraints to the sustainability of WASH outcomes are identified by two partners with regards to
maintaining WASH infrastructure and obtaining funding to meet WASH demands. Partners ability to ensure duty
bearers provide and maintain WASH infrastructure is unclear. While the program has created demand for WASH,
the extent to which partners are positioned to advocate and obtain this is not clear. This issue was identified
and invested in towards the end of the program (in 2014) with the development of a WASH influencing strategy.
However, at the time of this assessment the gains the program has made in supporting people to engage with
duty bearers and advocate for WASH services are not known. The other two NGO partners did not foresee these
issues, reporting that they expect to continue to implement WASH in a sustainable way.
30
From the design of the program Oxfam had its own sustainability plan in relation to partners which influenced
what it did and how it rolled out the program (participation, partnership and capacity building) however on a
partner level more support was needed to assist partners to have their own sustainability plans. Towards the end
of the program, Oxfam began to carry out sustainability and exit planning with partners. Partners noted the value
of this work in supporting them to plan and work in a more strategic way. They noted its particular importance in
the context of a large donor exiting within a constrained funding environment. There is, however, concern that
this work started too late in the program. Furthermore, the fundraising support did not materialise as partners
could not agree on how to complete it within the short time allocated.
As previously noted, strong partnerships have been formed by partners facilitated by collective linking and
learning activities. Partners noted the momentum that has been built through AACES in this area, and how rare
such strong collaboration is between NGOs in South Africa. While some formal partnerships have been formed (in
non-WASH areas), partners have identified a risk that they will revert back to a more isolated way of working without
Oxfam playing the role of convener, losing momentum and access to collective knowledge and experience.
What can be learned about ensuring sustainability of program outcomes in relation to the context and program type?
The integrated approach of working with partners in a tailored and flexible way to mainstream WASH at
institutional and program levels has fostered sustainability, as partners have bought into WASH and have
the capacity to implement it.
The inclusion of an objective related to the establishment of stakeholder forums around partners identifying
which government departments are responsible for ensuring delivery has supported partners to know who
to target for future WASH advocacy.
The forming of strong partnerships between AACES NGO partners (as discussed in the Efficiency Leveraging
section above), supports the potential sustainability of outcomes, as support networks have been created
and may endure if partners continue to use these.
Including an advocacy component is essential to ensuring the sustainability of the program it has been
challenging to introduce advocacy activities due to an initial negative perception around this term and form
of engagement with government.
RECCOMENDATION QUESTIONS (CURRENT/FUTURE PROGRAM):
What measures could have been taken to enhance sustainability and would this require additional investment?
On a partner level more support was needed to assist partners to have their own sustainability plans
institutional sustainability and exit planning work would have been more effective if it had begun mid-way
through the program rather than towards the end.
Partners form links with external WASH hardware suppliers (government and private sector) while
influencing the ability of government to deliver WASH services and infrastructure is largely beyond the
control of Oxfam and partners (as it is very difficult to gain traction and influence in this area), making clear
the requirements and avenues of each partner to receive, maintain and repair WASH hardware would support
consideration of sustainability and its assessment.
Partners continue links with advocacy partners such as Equal Education who could support and mentor
partners who want to continue to build their advocacy WASH work in schools.
31
Effectiveness rating
Overall effectiveness rating: high effectiveness (9/12)

(2 poor effectiveness, 4 low effectiveness, 6-7 fair effectiveness, 9 good effectiveness, 12 high effectiveness)
Reach and depth of change: 6/6 Sustainability: 3/6

How well is the program achieving its To what extent are program outcomes likely to
intended outcomes? be sustained beyond the program?
It is not clear to what extent the program has
It is not clear if the program is achieving its
U U addressed sustainability; a judgement cannot
intended outcomes; a judgement cannot be formed
be formed
Sustainability has not been well considered and/
Outputs led to a small number of scale and
1 1 or it is not clear how program outcomes are likely
breadth of outcomes
to be sustainable
The program has been designed and
Outputs led to a good level of scale and breadth
3 3 implemented with sustainability in mind, though
of outcomes
it is not clear if sustainably is likely
Strong efforts have been made to ensure
Outputs led to a high level of scale and breadth
6 6 sustainability and evidence of likely sustainability
of outcomes
is strong

AACES MTR
AACES snapshot reports
Partner reports
Oxfam WASH influencing strategy
AACES partner Questionnaire
MEL report
Reports provided rich qualitative data, allowing changes in relation to each AACES objective to be analysed
In the questionnaire, partners provided clear information on which inputs had supported effectiveness
and why
The MTR and reports contained rich analysis of the implementation process including issues encountered
and how they were addressed
Quantitative data to show the reach of partners
Limited documentation of a sustainability assessment which outlines which WASH services and
infrastructure communities can source and maintain (and is therefore sustainable), and what external
support is required by partners to maintain and continue WASH work (and is therefore potentially
unsustainable if there is not a clear process in place to ensure this is received)
32
Nyanga Village, Barotse Flood Plains,
Zambia- Kabu Kabu Kalauka, chairperson
of the village WASHE commitee, and the
Nyanga community offer a warm welcome
with music and dancing as the Oxfam
team and the visiting partners arrive for
the site visit by boat.

COMPONENT 4: EQUITY
(equal inclusion of vulnerable groups)
Is the program appropriately engaging and producing equal benefits

for different groups?
Relevant contextual factors and implications for equity
Partners had not previously worked to incorporate disability into their programs meaning that they were
starting from a low base
Oxfam had not previously worked in disability inclusion, and did not have internal expertise in this area
Gender inequalities are entrenched with partners experiencing challenges in this area
Equity sub-category 1: equity of process
How well is the program appropriately targeting and ensuring the participation of vulnerable groups?
(ie women, men, children, people with disability, people from different ethnic groups)
The focus on disability inclusion has been a standout of the program. The support provided by CREATE has been
nuanced, targeted and high-quality and has been highly valued by NGOs. As stated by one partner, the support
and training provided by Oxfam via CREATE was no less than revolutionary in our organisation and community.
Including people with disabilities in the training was a very effective way of changing perceptions around
disability both within our own team and in the community. The disability inclusion focus came about through a
donor requirement. Despite an initial perception among partners of a top down requirement, partners began to
see this as a need and have demonstrated genuine commitment to the concept and the process. Interestingly,
while one original partner dropped out of the AACES program due to the heavy requirements, they have stayed in
contact with CREATE and are seeking to continue this relationship to progress disability inclusion. As illustrated
in the section below, partners have developed a range of capacities to include people with disability in their work.
Gender has been acknowledged as a gap and challenge among partners throughout the program from beginning
to end. As stated in one report, gender issues, as mentioned by some partners, remain the big elephant in the
room. It appears that most partners found it easy to tackle issues of inclusion of people with disabilities but
struggled on issues of gender. The rural and traditionalist nature of Kwa-Zulu Natal contributes immensely to this
topic being difficult to tackle. Oxfam invested in this area, engaging consultants to provide support to partners
to strengthen the gender outcomes of their work. After a pilot activity, both Oxfam and partners acknowledged
that these consultants were inappropriate. Partners found their approach to be overly theoretical, not culturally
appropriate, not male inclusive and to have created more harm than good. Oxfam battled to find an alternative
consultant and decided to use their MEL consultant in the recent annual workshop to highlight some of the
gender issues and how they intersected across program objectives. While actions were taken to progress
gender, the program appears to have resulted in a missed opportunity to progress gender by not being able to
address this issue earlier on; partner program reports do not provide sufficient information of what gender issues
partners have faced and how they have tackled these.
34
Several partners work in rural areas which has also enabled the program to ensure remote populations have
access to clean water and other WASH outcomes. The program has targeted a range of age groups by virtue of
the target groups served by partner organisations. Of the five partner organisations, three partners are child
focused agencies; one works directly with youth; and one serves all age groups. This has ensured children and
youth are reached through the program. Oxfam has engaged the partner Resources Aimed at the Prevention of
Child Abuse and Neglect (RAPCAN) to work with partners to strengthen their ability to integrate child protection
across their programs. RAPCAN conducted a baseline with partners which allowed them to identify what their
value-add was. RAPCANs training on child protection went further than instituting organisational child protection
policies and safeguards by supporting partners to develop a child rights-based approach to programming.
What can be learned about supporting equity in relation to the context and program type?
Having a specific objective around equity (reducing inequalities in WASH) has focused partners in working
towards inclusion.
Providing ongoing support to partners by working in relation to the pace and capacity of partners, rather than
approaching inclusion from a language and approach of compliance has been an appropriate strategy.
An integrated approach to disability and child protection whereby support agencies worked with partners
in tailored ways to build understanding but also helped the different partners to relate and integrate these
concepts and approaches within their own programming contexts.
Engaging two specialised partners (child protection and disability inclusion) to provide ongoing and targeted
support has been effective.
Giving service providers the freedom to decide what supports are necessary, rather than providing blanket
requirements has been appropriate.
Having Oxfam staff learn with and alongside partners demonstrates leadership and commitment to
supporting inclusion.
A good balance on reaching children, youth and adults has been achieved by selecting organisations that
work with a range of target groups.
How could equity be more strongly integrated into the ToC, and the design and implementation of the program?
What resource implications would this have?
Instead of only focusing on supporting partners to facilitate inclusion, Oxfam itself could have taken greater
steps to continue to progress its own disability inclusive practice at all levels within the organisation as
partners have done.
While equity was well integrated into the ToC conceptually, alternative strategies to address gender issues
should have been put in place early on, with other options sourced when planned work did not eventuate or
was inadequate.
Bringing Oxfams internal expertise in gender to bear on AACES, gained through its experience of delivering
other programs in South Africa over the last decade, may have benefited the program.
35
Equity sub-category 1: equity of outcomes
How equitably have different groups benefited?
Reduced inequalities in WASH is a program objective and has clearly been facilitated by the program. The AACES
MTR and program reports provide clear evidence that partners have improved their capacity to understand and
navigate inclusion issues, particularly in relation to disability and children (noting gaps in gender capacity
identified above). A range of data is provided to show the organisational capacities partners have developed to
support inclusion (i.e. identify WASH needs for people with disability, understanding of the differing needs and
access to WASH services between rural communities and urban communities). A range of data is also presented
on the adaptations/inclusion strategies used to promote inclusion (i.e. design and building of accessible toilets for
people in wheelchairs, and training of community health and care workers in identifying people with disability).
Comprehensive qualitative data is provided on many of the benefits experienced by women, children and
people with disability across programs, detailing the changes in peoples lives as a result of the program.
Comprehensive gender disaggregated data is provided for the life of the partnership. Figures provided in the
effectiveness section above in relation to the additional people accessing sustainable and safe water and
appropriate sanitation indicate that increases in men and womens access have been comparably equitable, with
women experiencing slightly higher access rates.
A total of 230 additional people with disability reportedly gained access to services. Some partner reports reveal
anecdotal evidence regarding the reach of their programs in relation to people with disability. For example, one
partner stated, our community care workers began to do home visits with a keen new set of eyes finding many
more people and children with disabilities in the community than we were previously aware of; the numbers of
children identified and assistance to the monthly disability clinic tripled. Disaggregated data is not provided
in relation to the number of boys and girls and people with disability accessing WASH services. However, given
the number of partner organisations serving children, it can be expected that children have been well reached
through the program.
RECOMMENDATIONS QUESTIONS (CURRENT/FUTURE PROGRAM):
The final evaluation offers an opportunity to provide a more comprehensive picture of the impact of the
program and the extent to which it has benefited different vulnerable groups.
Program documents reveal partners have used a range of MEL tools to capture and analyse how their
WASH interventions have affected different groups; this information could be reported on and fed into the
final evaluation.
36
Equity rating
Overall equity rating: equity unknown (9/12)

(2 poor equity, 4 low equity, 6-7 fair equity, 9 good equity, 12 high equity)
Targeting of vulnerable groups: 3/6 Facilitating equitable outcomes

How well is the program appropriately targeting for vulnerable groups: 6/6
and benefiting vulnerable groups? To what extent have vulnerable groups benefited?
It is not clear if equity has been integrated; It is not clear who has benefited; a judgement
U U
a judgement cannot be formed cannot be formed
Equity was not sufficiently considered

1 1 The most vulnerable groups have not benefited
and integrated
Equity was integrated and resourced to some Some vulnerable groups have benefited
3 3
extent; there is scope to strengthen and others have not
Equity was strongly integrated into the programs Vulnerable groups have benefited significantly
6 6
design and implementation and equitably

AACES MTR
Partner reports
Annual snap shot reports
AACES partner questionnaire
Rich qualitative data on the changes (particularly in relation to people with disability)
Information on the strategies and adaptations used by partners to bring about inclusion
Disability disaggregated data on the number of people with disability accessing services and
infrastructure delivered through the program
Lack of child disaggregated data
Analysis of gender issues including strategies to address them
37
Oxfam Staff and South African partners
meet in Livingstone for the Annual
Reflection and Planning workshop.
Photo Oxfam
Value for money assessment appendix
Summary of costs and inputs (ZAR)
Year 1 Year 2 Year 3 Year 4 Total
Innovation 0 103,659 0 0 103,659
Research 108,000 210,700 108,514 79,800 507,014
Monitoring, evaluation and learning 119,850 342,766 215,061 32,319 709,996
WASH conferences 0 17,309 77,914 0 95,223
Exchange visits 0 3,600 33,243 0 36,843
Technical training 30,000 310,615 356,300 450,000 1,146,915
Cross cutting themes 664,828 525,000 394,137 320,300 1,904,265
Photographs 20,400 34,200 16,500 13,000 84,100
Storytelling and video making/photostories 0 204,055 141,600 85,710 431,365
Annual reflection and planning meeting 488,522 387,525 257,134 317,148 1,450,329
6,469,709
39
Purpose
This document outlines the theory behind the VfM assessment tool used to assess the Oxfam in South Africa
AACES program. It aims to orientate the reader as to why the particular approach has been taken and why certain
elements have been included. The assessment framework draws on a range of VfM research and analysis that
has taken place over the last 10 years. This document provides an outline of what the VfM tool does and does not
aim to do and why, with reference to this literature.
The approach
The approach takes Oxfams definition of VfM the best use of resources to contribute to positive significant
change in the most vulnerable peoples lives1 as the starting point for the tool. It aims to articulate the links
between resources invested and outputs and outcomes achieved (with reference to the four Es: economy,
efficiency, effectiveness and equity) and examine if and how Oxfam could achieve greater change given the
level and type of investment and operating context. It draws on aspects of the Basic Efficiency Resource (BER)
approach by comparing outputs and outcomes to resources.
The approach recognises that Oxfam delivers rights-based programs in complex settings, and that assessment
of VfM must recognise the context specific factors that determine VfM options and considerations. It draws
on Bonds proposition that one way of approaching VfM is for NGOs to build a robust and defensible case for
how an intervention balances economy, efficiency and effectiveness and delivers the most value for poor and
marginalised people.2 Oxfam sees VfM as integrated throughout the program management cycle, and believes
that if these processes are followed, its projects will deliver VfM. The tool provides a practical way to assess
initiatives to see if this holds true and consider how VfM can be enhanced. It seeks to progress Oxfam Great
Britain (OGB)s finding of needing to find ways to demonstrate VfM that are more than good management but may
not go as far as monetary measurement of impact.3
Defining the value in VfM
The approach does not aim to assign value in the way approaches such as Social Return on Investment (SROI)
do, using proxy indicators to come up with an overall value proposition. This focus does not systematically
answer the fundamental VfM question: could the use of resources be improved. The approach does not seek
to determine or rate the value of the changes made and ask the question was it worth it in relation to the
investment put in as some other approaches do. This is because Oxfam has not taken this approach and there
are competing interpretations of what value is, or should be, and who ought to define it4 and because value
takes time to deliver, especially at scale5, and may only be realised after the program has completed. In addition,
Oxfam works to progress human rights enshrined in national and international conventions (which arguably
must be progressed regardless of their perceived value). Oxfam recognises that pathways to achieving rights are
non-linear, context specific and must be strengthened through the application of strong MEL, and has therefore
focused on better understanding and improving its change pathways with a VfM lens.
1 Value for Money Discussion Paper, Oxfam Australia, Program Quality Unit, Sept 2013, Page 1.
2 Bond for International Development, 2012, Integrating Value for Money into the programme cycle.
3 OGB VfM training slide pack, June 2013, Page 78.
4 LSE, Value for Money: Current Approaches and Evolving Debates, 2011, Page 3.
41
Placing theory of change at the heart of VfM
The central role of Theory of Change in VfM is being increasingly acknowledged as development agencies seek
to apply VfM concepts at a programmatic level. London School of Economics (LSE) research conducted in
relation to VfM approaches and debates notes that many stakeholders mention a Theory of Change as it
reveals the organisations understanding of value, illustrated through their rationale connecting inputs,
outputs, outcomes and impact6. Findings of an OGB evaluation that included a VfM assessment highlighted the
importance of focusing on ToC to advance VfM. It states, by carefully identifying the point at which success
is measured (between activity and impact), and carefully tracking the types of interconnected inputs into the
activities which contributed to such change, the assessment of Value for Money and programme quality itself
can gradually be strengthened7. The assessment tool takes this suggested approach forward, using ToC to
locate these elements, examining VfM in relation to the change processes Oxfam and its partners contribute to.
As highlighted by Oxfam, this is important to ensure that the value is not considered in the achievement of an
activity, but in the occurrence of change, progress towards the outcomes and final vision as expressed in the
theory of change8.
Using ToC is also important to ensure VfM assessment is strongly located in the programming context and is not
reductionist and disconnected from context, which very much influences level of cost and investment required
to achieve specific changes. As highlighted through the OGB evaluation, The monitoring of Value for Money
needs to be able to link outcomes to inputs and those to financial resources. It is important to build in ways to
relate these elements together without losing the complexity and nuance of how change happens9.
Comparison as a means of assessment
An intervention can only be VfM compared to a different option, not by itself10; Nothing is good or bad, except
in comparison to something else11. While the need to build comparison into VfM assessment is acknowledged
in the literature, challenges related to this undertaking such as determining comparable data sets, comparing
programs that are implemented in different contexts, and finding available data to enable comparisons are
also widely noted. The approach recognises that it is difficult to do this in any precise way, but aims to trial
some comparative methods in a flexible way. As observed by a director of NGO performance, We have to encourage
comparisons. Were going to lose some nuance thats OK. We cannot make decisions without losing some nuance.12
The tool aims to document the costs and inputs in relation to the different strategies/activities used to effect
change in different thematic and sectoral programs. It also aims to document the contextual factors that
affect VfM considerations to support comparison. In time, this may support Oxfam to compare across its own
programs with similar desired outcomes13. The approach to comparison draws on work undertaken by OGB14 which
identifies three forms of comparison: 1) benchmarking measurement (comparing program achievements with
similar achievements outside the program external agencies); 2) trend measurement (which shows progress
6 LSE, Value for Money: Current Approaches and Evolving Debates, May 2011 Page 3.
7 Womens Right to Be Heard: An evaluation of Oxfam GBs Raising Her Voice portfolio, June 2013, Page 59.
10 LSE, Value for Money: Current Approaches and Evolving Debates, 2011 Page 24. According to Nicholles the objective of understanding VfM is to make
decisions and these are usually between more than one thing. This is powerful and useful within an organisation, but not necessarily between organisations
because often it is hard to compare like for like.
11 OGB Page 3, Bond OGB VfM Discussion Paper BER, Clair Hutchins, Page 3.
12 LSE, Value for Money: Current Approaches and Evolving Debates, 2011 Page 25.
13 This approach is also supported by LSE research which states discussions with interviewees, as well as the analysis of the methodological underpinnings
of VfM, suggest that one purpose of VfM could be to compare interventions with the same desired outcome. As such, VfM would refer to a way of achieving
the same outcomes with more efficient use of inputs (LSE Value for Money: Current Approaches and Evolving Debates, 2011, Page 26. Ideally its programs
could be compared to external agencies, however this requires other organisations to document their models and associated investments and make these
publically available. Concerns have been raised regarding the competitive environment this approach would create and the race to the bottom. LSE Value
for Money: Current Approaches and Evolving Debates, 2011, Page 26.
42
over time can demonstrate cumulative effect or show comparative improvement over time); and 3) stand-alone
measures (show what has been achieved within a reporting period and can be compared against the plan for that
period). Those steering OGBs work argue that programs often undertake comparison in relation to stand alone
measures, but need to move towards external benchmarking. In the early stages of the VfM assessment process,
the assessor tried to work with teams to identify available external comparators. However, in practice when
undertaking the three assessments, this could not be achieved at this early stage of formal VfM assessment and
trend measurement and stand-alone measures were used.
Leveraging as an indicator of VfM
The VfM framework explicitly integrates leveraging as defined by Oxfam International in a 2014 paper Leverage:
reaching scale in our work: Leverage is working strategically with others in a clever way, in order to lever a
bigger change than we could ever achieve on our own. It depends on developing a rich web of mutually beneficial
relationships and alliances at country, regional and global level. Leverage emerges out of that connectivity.15
Leveraging is not new and is something Oxfam does widely throughout its programs. However, integrating
leverage in a VfM assessment aims to help program teams to consider how they are and can more explicitly
harness the networks, resources, ideas and assets of others to achieve goals more efficiently and effectively
and create larger change enhancing VfM. It also ensures an assessment of VfM includes what others bring to
the change process, rather than explicitly focusing on Oxfams actions and resources. As highlighted in Oxfam
Internationals paper, a leverage approach means we must systematically strengthen these networks at all
levels. Leverage is about being far sighted as to what we want to achieve, clear sighted and strategic about
who will do it and especially, astute about understanding what kind of actions will create the alliances and
momentum we need in order to bring about the big changes we seek.16
Assessing scale and depth of change in VfM
Christian Aid has developed a considered and eloquent approach to VfM. Its approach is about achieving the
best results it can with the money and resources it has. It defines best results as the scale (numbers of people
benefiting), depth (intensity and sustainability of change) and inclusion (in other words, a change has greater
impact if it benefits people who are more excluded and marginalised)17. Oxfam shares this approach, and the
VfM tool seeks to capture numbers reached, level/depth of change, and levels of equity within outcomes. While
it is challenging to define and measure depth of change, the framework seeks to assess the extent to which
changes in practice and structural/institutional changes brought about are expected to be sustained beyond
the life of the program. Noting that impact can be difficult to measure and may not be realised until years later,
the framework seeks to assess the strength of outcomes produced.
15 Oxfam International Leverage: Reaching scale in our work, 2014, Page 1.

16 Oxfam International Leverage: Reaching scale in our work, 2014, Page 1.
17 Christian Aid, How Christian Aid Assesses Value for Money in its Programmes, July 2012, Page 1.
43
Participation of partners and community in assessment of VfM
A VfM assessment must importantly consider from whose perspective value is defined. This VfM assessment
tool is based on evidence collected through participatory monitoring and evaluation processes. Community
participation in this VfM assessment is encouraged through the methodology. There are different ways in which
communities can be involved throughout the assessment and the approach aims to encourage teams to explore
different ways of engaging stakeholders. This approach draws on DFATs approach to VfM which incorporates
ethics (transparency and accountability) into its definition of VfM18.
Evidence-based VfM assessment
Assessment is based on evidence. This includes regular MEL data and program documentation and additional
data collected by the assessor in the event of gaps in available data in relation to any of the four Es. Assessment
is based on evidence obtained through traditional evaluation methods such as desktop reviews, stakeholder
interviews and questionnaires, and participatory reflection workshops. The approach recognises that a VfM
assessment can only be made on the basis of evidence, and does not seek to facilitate a process whereby
assessment makes an unsubstantiated judgement due to lack of evidence, or penalises projects by rating VfM
as poor due to a lack of existing data. It is recognised that in some cases it may be difficult to make a judgement
due to lack of data. As with many evaluation methodologies, the judgement is ultimately the subjective
interpretation of the evaluator. The approach uses VfM assessment as an opportunity to help teams strengthen
their MEL (and application of the wider program management cycle), and explore how they can better integrate
VfM into their MEL frameworks which is key to managing for VfM.
18 An Oxfam presentation 2012 states, Ethics is described by AusAID as gives attention to the way VfM assessment is conducted to ensure that the research
or data collection is undertaken with permission of participants and in a way that is honest and understood by those involved.
44
Mdumiseni Zungu greeting the Nyanga
community during a partner village.
Photo Wendy Lubbee

www.oxfam.org.au

Value For Money Assessment Oxfam in South Africa: Australia Africa Community Engagement Scheme (AACES) Capacity Building Component

Uploaded by

Copyright:

Available Formats

Value For Money Assessment Oxfam in South Africa: Australia Africa Community Engagement Scheme (AACES) Capacity Building Component

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Value For Money Assessment Oxfam in South Africa: Australia Africa Community Engagement Scheme (AACES) Capacity Building Component

Uploaded by

Copyright:

Available Formats

Oxfam in South Africa

Australia Africa Community Engagement Scheme (AACES)

Photo Alexia Webster

Published: April 2016

Author: Michelle Besley, Independent Consultant

Design: LUMO design & illustration (www.lumo.co.za)

Contact Details: Oxfam OXFAM AUSTRALIA

Development of a Value For Money assessment framework 6

Program Assessment Summary 8

Value For Money assessment process 9

Summary of key findings 10

Full Value For Money Assessment 14

Component 1: Economy (Costs and inputs: What went in?) 18

Component 2: Efficiency (Inputs to outputs: What happened?) 22

Component 3: Effectiveness (Inputs to outputs: What happended?) 28

Component 4: Equity (Equal inclusion of vulnerable groups) 34

Methodological Rationale and Theoretical Underpinnings

Defining the value of VFM 41

Placing theory of change at the heart of VFM 42

Comparison as a means of assessment 42

Leveraging as an indicator of VFM 43

Assessing scale and depth of change in VFM 43

Participation of partners and community in assessment of VFM 44

Evidence-based VFM assessment 44

ATSIPP Aboriginal and Torres Strait Islander Peoples Program

ACRP Australian Community Rehabilitation Program

BER Basic efficiency resource

ECD Early childhood development

ToC Theory of change

DFAT Department of Foreign Affairs and Trade

KAP Knowledge, attitude and practices

MDIC Maputa Land Development and Information Centre

MEL Monitoring, evaluation and learning

MTR Mid-term review

NGO Non-government organisation

OGB Oxfam Great Britain

RAPCAN Resources Aimed at the Prevention of Child Abuse and Neglect

SROI Social return on investment

VfM Value for Money

WASH Water, sanitation and hygiene

consideration of scale (the number of people benefitting);

VfM has four core dimensions referred to as the four Es:

Economy Costs and inputs what went in?

Efficiency Inputs to outputs what happened?

Effectiveness Outputs to outcomes what resulted?

Is the program generating positive and sustainable outcomes?

Equity Equal inclusion of vulnerable groups

Oxfam in South Africa Australia Africa Community Engagement Scheme (AACES)

Oxfam Sri Lanka Australian Community Rehabilitation Program (ACRP3)

Value for money assessment process

Photo Wendy Lubbee

Recommendations for the remainder of the program period:

Recommendations for future programming:

Program name: Australia Africa Community Engagement Scheme (AACES)

Region: Southern Africa

Country: South Africa

Program period: 20112016

Stage in program cycle: End of the program cycle

Program cost: AUD $2,628,779 (ZAR 6,469,709) capacity building support