PROVISION FOR POST PROJECT EVALUATIONS FOR THE UNITED NATIONS

DEMOCRACY FUND
Contract NO.PD:C0110/10

EVALUATION REPORT

UDF-MEX-08-279 – Civil society advocating for quality education and

healthcare with equity in Mexico

Date: 11 September 2012

Acknowledgements
The evaluators would like to thank all of those who took the time to provide their expertise
and insight on civil society participation in Mexico and health and education issues and on
the implementation of the Civil Society Advocating for Quality Education and Healthcare with
Equity in Mexico project. In particular, Rebecca Berner of Oxfam México and the entire
Oxfam México team who provided logistical support for the evaluation field work in Mexico as
well as their extensive knowledge on the project and its implementation.
All errors and omissions remain the responsibility of the authors.

Disclaimer
The views expressed in this report are those of the evaluators. They do not represent those
of UNDEF or of any of the institutions referred to in the report.

Authors
This report was written by Sue Nelson and Carlos González Martínez.
Landis MacKellar, the Evaluation Team Leader, provided editorial and methodological advice
and quality assurance. Ms. Aurélie Ferreira was Evaluation Manager and Mr. Eric Tourres
Project Director at Transtec.
Table of Contents

I. EXECUTIVE SUMMARY ........................................................................................................................... 1

II. INTRODUCTION AND DEVELOPMENT CONTEXT..................................................................................... 4
(i) The project and evaluation objectives ..................................................................................................... 4
(ii) Evaluation methodology ............................................................................................................................ 4
(iii) Development context ................................................................................................................................. 5
III. PROJECT STRATEGY ............................................................................................................................... 7
(i) Project approach and strategy .................................................................................................................. 7
(ii) Logical framework ....................................................................................................................................... 9
IV. EVALUATION FINDINGS ....................................................................................................................... 10
(i) Relevance ................................................................................................................................................... 10
(ii) Effectiveness .............................................................................................................................................. 11
(iii) Efficiency .................................................................................................................................................... 13
(iv) Impact ......................................................................................................................................................... 15
(v) Sustainability.............................................................................................................................................. 16
(vi) UNDEF added value .................................................................................................................................. 17
V. CONCLUSIONS ..................................................................................................................................... 18
VI. RECOMMENDATIONS .......................................................................................................................... 19
VII. OVERALL ASSESSMENT AND CLOSING THOUGHTS ............................................................................... 20
VIII. LIMITATIONS, CONSTRAINTS AND CAVEATS ....................................................................................... 21

IX. ANNEXES ............................................................................................................................................. 22

ANNEX 1: EVALUATION QUESTIONS: .............................................................................................................. 22
ANNEX 2: DOCUMENTS REVIEWED : ............................................................................................................... 23
ANNEX 3: PERSONS INTERVIEWED ................................................................................................................. 25
ANNEX 4 : ACRONYMS ................................................................................................................................... 27
 I. Executive Summary
(i) Project Data

The Civil Society Advocating for Quality Education and Healthcare with Equity in Mexico
project sought to strengthen the collective voice and action of civil society in the Mexican
states of Chiapas, Hidalgo and Guerrero. The focus was on increasing the ability of local civil
society organizations (CSOs) and women, indigenous groups and youth to demand equitable
and quality public education and healthcare services through 1) strengthening the ability of
CSOs to train, inform, and raise awareness of these rights, and 2) enabling their engagement
to advocate for these services as a basic human right. The project was implemented by
Oxfam México (formerly the Rostros y Voces organization) through subgrants to three CSOs:
the Colectivo de Atención para la Salud Integral de la Familia in Chiapas (CIFAM), the
Academia Hidalguense de Educación y Derechos Humanos (ACADERH) in Hidalgo, and
Café, Mujer y Comunidad (CAMCO) in Guerrero.

This was a 405,000 USD project, funded by a UNDEF grant of 375,000 USD (of which
25,000 USD was used by UNDEF for monitoring and evaluation) with 30,000 USD from
OXFAM Novib in co-financing. It was a two year project with a four month no-cost time
extension (November 2009 -February 2012). According to the project document, its main
activities were to:
 undertake a participatory diagnosis of the health and educational services;
 information and capacity building for civic actors on the right to these services and
advocating for them;
 developing public agendas for action and information; and,
 monitoring international commitments made by the Government of Mexico on basic
services.

(ii) Evaluation Findings

The project was implemented in general as described in the project document for activities at
the state level. The three CSOs received sub-grants at the start of the project and each
worked in networks within the 5 or more municipalities identified within their states. The
anticipated work at the national level was not undertaken with the exception of the national
diagnosis which was completed. The reasons for this are not clear to the evaluators as
conflicting reasons were given, but it appears to be related to an Oxfam México capacity
issue and/or to the conceptual view of the Oxfam México project team on how the project
should be implemented. The lack of work at the national level did affect overall project
performance and results.

The project objectives and activities were relevant given the poverty levels, the quality of
services in the three states targeted by the project, and the lack of attention given to health
and education as a basic human right. The women, youth and indigenous groups were
among the most marginalized within these areas. The project was also directly relevant to
the institutional mandate and vision of Oxfam México and their three CSO partners/sub-
grantees. Risks were identified for the project in terms of the politicization of local officials
and the Mexican electoral campaigns. But programmatic activities did not appear to
adequately account for these risks which Oxfam México credited for programmatic delays
and reducing its effectiveness and impact, especially in Guerrero. This was also cited as one
of the rationales for not implementing the national level activities.

1|Page
The project strategy of undertaking a local-level diagnosis of the problem, then providing
training on the issues identified and developing an agenda for action proved to be an
effective. The agenda provided a focus for the groups working in this area and allowed for
the development of constructive discussions with local authorities on issues in the sector.
Linking the provision of health and education to basic human rights was seen by participants
as an effective approach as it increased the importance of the advocacy efforts and
receptiveness of civil and political actors to the messages of the project. However, the
diagnosis focused on the quality of health and education and lacked information on policy
making, citizen interaction in that process and what needed to be addressed to strengthen
their voice and advocacy efforts to improve services which was the intended outcome for the
project. The cascade nature of the programme was a good means to reach larger numbers
of persons however, the extremely decentralized nature of the project resulted in the project
being implemented more as three separate efforts than one integrated programme which
limited its effectiveness.

This degree of decentralization and separation between the CSO partners also affected the
efficiency of the project as it did not take advantage of the expertise of the partners and
resulted in duplicated efforts. Administrative support from Oxfam México did appear help
ensure a smoother administration of the multi-level project, and most project participants felt
there had been a good use of time and resources. The three sub-grantees used networks to
help implement the activities which increased efficiency, but the lack of a formal agreement
on their working relationships affected the project as not all had the same sense of project
purpose or obligation to participate regularly.

The project reported on activities and outputs but lacked the performance indicators needed
to assess impact. Anecdotal information suggests that the project did make a difference,
particularly at grass-root levels, but the impact could have been greater had the project been
implemented as one and included the national component. It appears that the relationships
between the civil society groups and the local officials are better and more constructive now
than they were at the start of the project and that some public officials are more receptive to
their messages. Most project participants expressed a feeling of empowerment from the
training and work on the agenda. In some cases, the evaluators heard of health and
education issues being given more attention in local government planning and in several
cases, the activities were reported to have resulted in improved services.

The grass-roots nature of the project means the knowledge and experiences of the project
are likely to remain within the targeted communities. The CSO partners are still working on
the issue of equitable services in their respective areas of focus and see the agenda created
by the project as a long-term planning tool. Their main sustainability issue deals with the
lack of financial resources which is a critical issue for most of the CSOs that participated in
this project. There was UNDEF value-added to this project. Many of the participating CSOs
felt the funding from a UN agency gave their organization and project more credibility and
visibility with the government officials and increased their access and receptivity to their
messages.

(iii) Conclusions
The project’s focus and activities were relevant and important within the Mexican social
and democratic context as it addressed issues of social equity and empowerment of
marginalized populations. The use of local level CSOs was an effective approach to
implement community-based activities, but the lack of an integrated programme between
the states and with national efforts limited its usefulness and potential impact. The use
of a diagnostic to develop training and an agenda for action was a good technique, but

2|Page
required more focus on issues of voice and civic participation which were the main
objectives of the project. The project strengthened advocacy efforts for more equitable
health and education services in its targeted areas in Chiapas, Hidalgo and Guerrero. In
particular, it increased individual knowledge, capacity and leadership among participating
CSOs and community members, strengthened their relations with local authorities on these
issues and in some cases resulted in improved services. However, the extent of results is
unknown due to the lack of outcome data. Changes are likely to be sustainable at the
level of personal empowerment and relationships built with some public officials and
the agenda provides CSOs with some of the key health and education issues to raise with
officials in the future. UNDEF-funding provided significant value added as it provided a
sense of neutrality and legitimacy to the rights-based discussions and to the CSOs that
participated in the effort.

(iv) Recommendations
For similar projects in the future, the evaluators recommend that civic participation and
advocacy projects be more firmly grounded in the democratic governance context.
Civic participation is done in within the political context of a country and this aspect should be
integrated into the project from the design stage on. A more robust civic education
component would strengthen citizen understanding of their roles and responsibilities in a
democratic system and the means by which they can hold the government accountable for
its actions, including service delivery and policy making. The electoral process is a part of
this process area and should be seen as a key opportunity to hold public debates on agenda
items, gain candidate support for the agenda, and monitor their commitments once they take
office. Projects should continue to promote local ownership but within an integrated
framework that builds a cohesive, synergistic programme. This requires a more
engaged programmatic role for UNDEF’s grantee as well as more formalized
relationships between sub-grantees and their partners. A results-based performance
monitoring plan focused on achievement of outcomes should be used in addition to
outputs to track project progress and measure its achievements. Finally during project
implementation provide guidance to CSOs and participants on how they can continue
their work in the equity sector so that they have an action plan already in place by the end of
the project.

3|Page
II. Introduction and development context
(i) The project and evaluation objectives
The Civil Society Advocating for Quality Education and Healthcare with Equity in Mexico
project was a two-year USD 375,000 project implemented by Oxfam México. The project ran
from 1 November 2009 to 29
February 2012 which included a four
month no-cost time extension. The
project worked to strengthen the
collective voice of civil society to
demand quality and equitable
education and healthcare services
for women, indigenous people and
youth in three of the poorest states in
Mexico: Chiapas, Guerrero, and
Hidalgo. It worked through three
state-level Civil Society
Organisations (CSOs): Colectivo de
Atención para la Salud Integral de la
Familia (CIFAM) in Chiapas; Café,
Mujer y Comunidad (CAMCO) in
Guerrero, and Academia
Hidalguense de Educación y
Derechos Humanos (ACADERH) in
Hidalgo. Each organization
undertook a diagnosis of the problems within their targeted municipalities, provided CSO and
community-level training and used the agenda developed through their activities to advocate
for more equitable services and policies for their communities.

UNDEF and Transtec have agreed on a framework governing the evaluation process, set out
in the Operational Manual. According to the manual, the objective of the evaluation is to
“undertake in-depth analysis of UNDEF-funded projects to gain a better understanding of
what constitutes a successful project which will in turn help UNDEF devise future project
strategies. Evaluations also assist stakeholders to determine whether projects have been
implemented in accordance with the project document and whether anticipated project
outputs have been achieved” 1.

(ii) Evaluation methodology

The evaluation took place in July 2012 with the field work in Mexico done 2-6 July, 2012. The
UNDEF Round 2 evaluations are qualitative in nature and follow a standard set of evaluation
questions that focus on the project’s relevance, effectiveness, efficiency, impact,
sustainability and any value added from UNDEF-funding (Annex 1). This is to allow meta-
analysis for cluster evaluations at a later stage. This report follows that structure. The
evaluators reviewed available documentation on the project and on the issue of equitable
services in Mexico (Annex 2). Interviews were held with Oxfam México, its main partners,
participants, government interlocutors, Oxfam Novib, the National Council for the Evaluation
of Social Development Policy (CONEVAL), and other nongovernmental organizations
(NGOs) working in the sector. The evaluators interviewed those in Mexico City and in
Pachuca, Hidalgo in person, and the remainder by phone, skype, and e-mail (Annex 3).

1
Operations Manual for the UNDEF-funded project evaluations, p. 3.
4|Page
During the preparatory work, the evaluators identified several issues which they followed up
during the field work in Mexico. These included:

 Ability of the project to reach its anticipated outcomes as it had ambitious goals
and had not implemented most of the national level activities as anticipated in the
project document.
 Extent the project leveraged existing data and efforts as each partner CSO did
their own diagnosis of health care/education systems, and each developed their own
curricula, manuals and training programmes raising questions of duplication of efforts.
 Sustainability issues and the degree of ownership in the programme that Oxfam
México sought to build, and whether the agenda were adopted and resulted in
sustainable changes for the intended beneficiaries.
 Value added from Oxfam México as the three CSO partners had already been
active in the project areas according to the proposal, and the networking and national
level work anticipated in the design was limited according to project reporting.
 Value added by UNDEF-funding as the project document suggested UNDEF
funding would strengthen CSO advocacy efforts by legitimizing them in the eyes of
the Mexican government.

(iii) Development context

The project grounded its design within the context of the Mexican government’s commitment
to provide essential services to all citizens, and in particular to the right to quality education
and health care for marginalized groups. as guaranteed by the Mexican Constitution. The
Constitution states that everyone has the right to free and compulsory basic education
(preschool through secondary), access to health services and an adequate environment for
their development and well being. The government has followed up on those commitments
and expanded health and educational services as part of its social policy in recent years.
This has earned Mexico a ranking of 57 out of the 187 countries in the United Nation
Development Programme Human Development Index and places it in the high human
development category2. However, despite these improvements, the project diagnosis found
that Mexicans still do not have universal access to public health services, education,
infrastructure and housing assistance programmes. Also that the quality of these services
varies considerably according to income, social status and location.3 Mexico bans all forms of
discrimination, but social and economic discrimination has marginalized some populations, in
particular indigenous people, many of which live in extreme poverty in rural areas. According
to the National Council for the Evaluation of Social Development Policy (CONEVAL)4, 46.2%
of Mexico’s population lives in poverty, with more than 10% in extreme poverty. These
numbers have increased in recent years due to the global economic crisis and its
repercussions on the Mexican economy. Rural and poor populations are especially
marginalized, and within those groups- women, youth and indigenous people.

The three states targeted by the project are among the poorest states in Mexico. In Chiapas,
almost a third of the population lives in extreme poverty, 35% lack access to health services,
and 48% are behind educationally. In Guerrero, 28% of the population lives in extreme
poverty, almost 40% lack access to health services and 55% are behind educationally. In
Hidalgo, 12% of the population lives in extreme poverty, 31% lack access to health services
and almost 32% are behind educationally. Within these states, access to services varied
between municipalities and within municipalities according to gender and population type.5
2
UNDP, Human Development Index, 2011
3
Oxfam Mexico, Diagnóstico sobre la Situación de la Educación y la Salud en los estados de Chiapas, Hidalgo y Guerrero, p 6
4
Unless otherwise noted, statistics in this section are from CONEVAL, Statistical Annex, Poverty 2010.
5
Diagnostic information from Oxfam Mexico, Diagnóstico sobre la Situación de la Educación y la Salud en los estados de
Chiapas, Hidalgo y Guerrero, p 56 and 57
5|Page
Hidalgo was found to by the diagnosis to have the best levels of human development among
the three targeted states, but with problems in areas with high concentrations of indigenous
populations, such as San Bartolo Tutotepec. Here nearly half the population was illiterate,
and 77% of the adults had not completed basic education. A similar situation existed in the
Copalillo municipality in Guerrero. Chiapas had the lowest indicators for human development
in the municipality of Venustiano Carranza which had the highest proportion of indigenous
population (21.6%). Some of the key issues found that related to the quality of education was
the lack of infrastructure and inability of professional staff to meet the social and cultural
diversity of the students. There were also issues related to the poor working conditions for
teachers in rural areas, and to political issues related to the teachers union. In the health
sector, the primary issues found were the right to live free of violence and to make informed
decisions. Here some of the most serious health problems were linked to issues of maternal
death, teen pregnancy, the increase in sexually transmitted diseases and uterine cancer, in
addition to the problem of alcohol consumption and its impact on gender-based violence
against girls and women.

The project’s diagnosis also found that access to health and educational services were
closely related to the ability of citizens to access other basic rights- such as food, housing,
infrastructure, potable water, and employment. Among the key factors that helped ensure
that their basic rights were met, was having citizens understand and demand their rights-
through advocacy and civic participation. The government has also started to address these
problems by including civic participation and outreach components into its social
programmes and services. However, many Mexicans are unaware of these opportunities or
understand the benefits of participating.

Surveying by CIVICUS of Mexican citizens, show a low level of trust (38%) in their political
institutions. Most of those polled did not believe in political action, with only 3% saying they
had participated in some type of political action, while 13% said they would never participate
and 82% said they didn’t know.6 In Mexico, the main form of political participation is voting,
with turnout at around 59% and turnout is greater among higher income groups than those in
the lower brackets.7

This project worked through CSOs and in addition to using them as implementers, intended
to strengthen their understanding of the issues and ability to work effectively in this sector.
There are an estimated 20,000 to 35,000 CSOs in Mexico with about 19,000 of these
focused on helping others.8 Almost half of these focus on social support, with about 18%
working on community development, 8% on health and the remainder working on other
issues including education and human rights. Most CSOs are concentrated in the 20 most
populous cities including Mexico City. The environment for civil society participation has
improved significantly since the democratic transition, especially for issues of democratic
development and human rights. Almost half the CSOs surveyed (42%) undertake advocacy
work to influence implementation of a public policy, with about half of these petitioning the
executive or legislature and only 17% doing so with public officials. Advocacy to strengthen
civic participation and networking amongst CSOs to interact on these issues with authorities
were seen as their strengths, while the scarcity of financial resources, and low levels of
transparency, visibility and accountability seen as their primary weaknesses.

6
Civicus, A Snapshot of Civil Society in Mexico, 2011 p 35
7
OECD, Better Life Index Mexico, 2011
8
Statistics and information in this section on CSOs are from CIVICUS, A Snapshot of Civil Society in Mexico, 2011
6|Page
III. Project strategy

(i) Project approach and strategy

The “Civil Society Advocating for Quality Education and Healthcare with Equity in Mexico”
Project sought to strengthen the collective action and voice of civil society in Mexico through:
1) participatory assessments and analysis of the quality of essential services at the national
and state levels; 2) information-sharing and capacity building of civic actors to advocate for
and monitor basic services as a human right; 3) formulating and promoting public agendas
for more equitable services; and 4) monitoring progress of government commitments for
equitable social services.

Oxfam México had been working in the sector of economic justice, active citizenship, equity
and humanitarian action as Rostros y Voces, a Mexican partner of Oxfam Novib at the start
of the project. It converted to a full member of the Oxfam confederation as Oxfam México
early in project implementation during the project duration. The project intended to work at
several levels. At the local level, it targeted the states of Chiapas, Guerrero, and Hidalgo and
where Oxfam México had been working with local CSOs on similar issues for several years.
At the state level it intended to work through advocacy and informational campaigns
undertaken by multi-sector actors, and at the national level through consciousness-raising
and advocacy efforts. The project also intended to participate in and learn best practices in
fostering civic demand for essential services from global and regional initiatives, particularly
in Latin America.

The local and state level activities were implemented through the three CSO partners: the
Colectivo de Atención para la Salud Integral de la Familia in Chiapas (CIFAM), the Academia
Hidalguense de Educación y Derechos Humanos (ACADERH) in Hidalgo, and Café, Mujer y
Comunidad (CAMCO) in Guerrero. Each received subgrants of approximately USD 80,000
for this purpose. Oxfam intended to implement the national project activities itself, as well as
to provide the general programmatic guidelines for its CSO partners, monitor their work and
manage the overall administration of the grant.

The project was implemented in a decentralized manner, with programmatic details at the
state levels largely decided by the CSO partners. This was a deliberate approach adopted by
Oxfam México which felt this would increase local ownership for the project. The project
started with Oxfam México hiring a consultant to undertake the baseline assessment of
social conditions nationally and in the three targeted states and a national diagnostic study of
essential services and needs. Each CSO partner hired its own consultants to undertake the
state-level diagnostic studies and to develop the training curriculum and manuals for their
state. Each CSO worked through their own networks created from existing relationships with
local organizations and persons working in the sector. Representatives of these networks
served as trainers (“promoters”) and implementers of the activities at the community levels
(Diagram 1). All of these third-tier organizations and persons participated pro-bono, with
expenses covered by the CSO partners with their sub-grant funding.

7|Page
 National
Oxfam Mexico
Israel Guardarrama

Chiapas Hidalgo Guerrero

CIFAM ACADERH CAMCO
Alejandro Rivera Marroquin Irma Eugenia Gutierrez Mejia Reyna Edith Carbajal Mendez

16 organizations 6 organizations 25 organizations

as promoters as promoters as promoters

7 municipalities 5 municipalities 5 municipalities

Copainalá, Chiapa de Corzo, Acaxochitlan, Huautla, Copalillo, Iguala, Pedro
Osumacinta, San Fernando, Ixmiquilpan, Pachuca, San Ascencio Alquisiras, Taxco de
Suchiapa, Tuxtla Gutiérrez,
Bartolo Tutotepec Alarcón, Tepecuacuilco
Venustiano Carranza

The diagnosis studies on the quality of public health and education services were undertaken
at the national and state level in the period February - April 2011. These assessments were
used by the partner CSOs as a means to identify the critical issues within their areas and
develop their agenda for action. As part of this process, a training programme was designed
and implemented in each state to strengthen the ability of the participants in their networks
and communities, and to develop and advocate for the agenda. The four modules in the
training programme were:
 Civic participation for essential rights (Participación ciudadana a favor de los
derechos esenciales).
 Public campaigns and media access (Campañas públicas y uso de medios).
 Advocacy and Lobbying (Incidencia política y cabildeo).
 Monitoring and evaluation of public services and policies (Monitoreo y evaluación de
los servicios y políticas públicas)

The agenda for essential services provided the focus for the advocacy work and the basis for
discussions with communities and local authorities. The project targeted women, youth and
indigenous groups which were considered by Oxfam México as the most vulnerable groups
within their targeted states. Each CSO had their own focus based on their institutional focus
and experiences: all groups were targeted in Chiapas, while only women were targeted in
Guerrero and only indigenous groups were the primary targets in Hidalgo.

The national level advocacy and networking with regional and global partners did not occur.
Different reasons were given to the evaluators for not implementing this part of the design,
but in general the project’s objectives were ambitious and project staff felt working at the
national level required more time, attention and resources than they had available.

8|Page
 (ii) Logical framework

Project activities Intended outputs Medium-term Long-term development

impacts/outcome objective
s
Medium Term
PARTICIPATORY ASSESSMENTS ANDImpacts
COMPARATIVE ANALYSIS

 Develop 3 territorial & 1 More equitable services and

 3 state & 1 national Identification of needs &
national diagnostic increased access for
diagnostic analyses priorities in health/ ed sectors
analyses marginalized groups
Improved quality &
 Comparative analysis of  1 comparative analysis to Better understanding &
transparency of state
state-national problems use as baseline priorities in health/ ed sectors
institutions & services
More knowledgeable, skilled &
 150 local leaders & CSO active CSOs advocating for & Increased civic participation &
representatives trained as monitoring rights to education demand for equable serves
 Training of local leaders
trainers, 75% female & health care,& on use of
and CSO representatives
 4 outreach & capacity media for public information More equitable services &
as trainers
building manual developed increased access for
per state Strengthened voice of women, marginalized groups
indigenous groups & youth
 Inventory of contact &
Increased networking and Stronger civic demand for
initiatives
exchange of information equitable essential services
 Local, regional & global  Participation in regional/
among state/national/global
networking global forums
actors working on equitable More effective CSO strategies
 Funds raised for exchange social service
visit

INFORMATION SHARING AND CAPACITY BUILDING

 Agendas & priorities defined CSOs actively engage with

Increased demand for & more
 Develop agendas with beneficiaries in 3 states government to enforce rights
equitable services
 1 national agenda defined to education and health care
 Consultations in 3 states & More equitable social
Government committed to
nationally on agendas development policies
 Presentation agendas to address equity issues
 1 agency & 10 municipalities
local, state, national
per state adopted proposals Improved access to quality
authorities CSOs monitored
 CSO monitoring of implementation of new policies
health & education services for
commitments marginalized groups

CREATION AND DISSEMINATON OF PUBLIC AGENDA

 Messages on heath &

 Development of More informed citizens on Increased demand for more
education a human right
informational materials rights to heath & education equitable services
developed
 500 women, youth,
indigenous people Increased demand for more
 Awareness raising participated in mobilization More aware citizens & policy equitable services
campaign in 3 states and activities in 15 municipalities makers on rights to heath &
nationally  Public awareness campaign education State services are more
implemented in 3 states & equitable
nationally
MONITORING PROGRESS TOWARDS STATE COMMITMENTS
Use of lessons learned to
 Quarterly progress reports Project implemented as
strengthen similar projects
 Project results analyzed in planned
 Monitoring & analysis of
internet forum
progress made in 3 states Increased state compliance
 3 state & 1 national report Factors of success and
with social development
on achievements constraints identified
commitments

9|Page
IV. Evaluation findings

(i) Relevance
The project objective and activities were appropriate and relevant to the level of CSO
organization in Mexico and to the basic needs of marginalized women, youth and indigenous
groups who were the intended beneficiaries. The three targeted states were among the
poorest in Mexico and had some of the lowest levels of social services in the country. The
project strategy of identifying and addressing health and education issues at the municipal
level brought the project activities closer to the beneficiaries, giving it a more direct meaning
in their everyday lives. The approach of strengthening these services for marginalized groups
through training on their rights, advocacy for improved services and promoting civic
participation to demand their rights also made it relevant from a democratic development
perspective.

There was a good level of interest in the project expressed by most project participants and
participating local officials. The project exceeded its targets concerning the number of
participants and municipalities reached. Several participants attributed this to the inclusion of
“local leaders” into the project activities
as promoters. They felt this gave
credibility and increased relevance of
the project activities. This reportedly
attracted local media which recognized
the importance of the messages, and
which provided free coverage of events
and air time for the public information
campaigns.

The approach of starting with a

participatory diagnosis of the problem
which was then used to drive the
training and develop an agenda for
action seemed to be an effective
Project activities in Chiapas technique to focus the work. However,
for a civil society strengthening project, the diagnosis focused almost exclusively on health
and education issues which diverted focus from the anticipated outcome of a stronger
collective voice and ability to effect policy change.

The project’s decentralized nature allowed the three CSO partners to adapt the project to
their own local context during implementation. In some aspects, this increased the relevance
for the participating CSOs as well their participants. For example, ACADERH in Hidalgo
decided to target indigenous groups which comprised 25% of the local population and which
were seen as the most disadvantaged group. ACADERH also noted the diversity of the
indigenous groups and included this intercultural aspect in their agenda-- in particular
advocating for three indigenous universities in the state rather than the one which was under
discussion at the time. ACADERH as an institution worked routinely in networks, and
continued that practice in this project by created a network of six organizations to help
implement the activities. These NGOs were indigenous themselves which increased the
relevance of ACADERH’s relevance to its targeted populations. This is in contrast to
Chiapas, where the CSO partner targeted youth but opened up the project to others who
were interested, and worked with 16 organizations which provided representatives to serve
as promoters, but which did not function as a network. In Guerrero, the CSO partner was a
women’s organization focusing on gender issues. Its project activities targeted women and it

10 | P a g e
worked with 25 CSOs that also worked primarily on gender and women’s issues.

The project’s objectives were also directly relevant to Oxfam México’s institutional mandate
and vision of strengthening civil society and enabling people to exercise their rights to create
a more equitable and democratic
society. However, the extremely
decentralized approach used for
project implementation undermined
the programmatic relevance of Oxfam
México beyond providing the general
outlines for the project and serving as
a grant manager for the three CSOs.

The political risks faced by the

advocacy elements of the project and
the difficulties of working at the
national level were not adequately
identified in the design or addressed
Project participants in Guerrero
during implementation. The electoral
process was blamed by Oxfam México and CAMCO for delays and difficulties in
implementation. It apparently politicized the CSOs affiliated with the network in Guerrero and
disrupted project coherence during large periods of time. In Hidalgo and Chiapas, project
implementation did not appear to be affected by the political processes, but some promoters
mentioned the need to “start over” when new officials came into office because they felt that
each party had different agendas and would not necessarily continue the work started under
a previous administration. The national level activities were a part of the design but were not
fully implemented. This directly affected the relevance of Oxfam México and the
effectiveness of the programme. For the CSO partners, beyond the provision of resources,
Oxfam México’s role was to build a project network (both within and beyond the project),
develop the national agenda and advocate at the national policy level for policy changes that
could support efforts at the state level. This would have integrated the efforts into a more
synergistic programme and increased the relevance, effectiveness and sustainability of the
efforts at the local level.

(ii) Effectiveness
Oxfam México met most of the intended outcomes as stated in the project document. The
CSOs were the key actors in the project and provided information, training and awareness
raising on the rights to education and healthcare. These CSOs interacted with government
officials in the three targeted states to promote these rights and because of their efforts,
more women, indigenous people and youth from marginalized communities in their targeted
areas are more aware of their rights and better able to articulate them with officials. However,
the extent of these relationships, interactions and results are unknown as they were not
adequately captured in project reporting.

Most of this data in the diagnosis were available publically and were compiled by the
consultants hired by the CSOs. CIFAM felt it already knew the issues and wanted to develop
its agenda without the diagnosis. It did the diagnosis, but submitted the draft two months late,
and the final document six months late. Doing the agenda did help to focus the work of the
CSOs and gave them a solid basis for their discussions with communities and local officials.
This led to constructive civic participation and dialogue with public officials.

11 | P a g e
Participants credited the rights-
based approach and UN funding
with increasing the effectiveness of
their messages with civil and
political actors, a positive point for
UNDEF value added. This helped
to develop relationships with local
government officials that did not
appear to have existed before the
project. For example, close
relationships appear to have been
developed in Hidalgo with the State
Minister for Health, in Guerrero with
the State Minister for Social
Development, and in Chiapas with
several state agencies. .
Public meeting in San Fernando, Chiapas
Oxfam México’s decentralized
approach was intended to increase local ownership for the project by allowing its partners to
work out their own implementation details based on the broad project lines it provided. The
CSO partners and participants did appear to own most of the activities. In fact, very few of
the participants/promoters interviewed knew of Oxfam México’s involvement when asked
during the evaluation.9 However, the project was delegated to the CSO partners to such an
extent that it affected the effectiveness and coherence of the overall effort. The three CSO
partners worked in almost complete isolation from each other, meeting together in only a few
workshops organized by Oxfam México and in a National Meeting Forum held at the end of
the project (February 2012). Each developed their own activity plans, training programme,
training materials, advocacy campaigns and messages. There was no project branding to
identify the efforts as a national movement to increase a collective civic voice towards
common goals.

ACADERH, and CAFÉ worked in networks within their state, primarily with organizations or
their representatives that they already knew and had worked with before. CIFAM opened the
project up to those beyond its usual partners. The local CSOs and/ their representatives
provided their time pro bono with the project covering their expenses, an effective and
efficient approach. The networks included independent journalists and media organizations
which was extremely useful as it helped to ensure good media coverage of activities and
generous airtime for their messages. Since the local CSOs were also already working at the
community levels, the cascade nature of the project made it easier for the project to reach
the grass-roots. As most of these local CSOs were at a lower level of institutional
development than the CSO partners, working on the project’s agenda and activities did
appear to have strengthened them programmatically, with the agenda providing them with a
direction for future work in the sector. Project training appears to have given promoters and
participants information that helped them to better understand their rights, and the skills to
raise them more effectively with officials. The agenda articulated those demands and gave
the specificity needed to discuss the issues constructively with local officials and to develop
their public awareness campaigns.

9
Project materials and products were all clearly labeled with the logos of Oxfam Mexico, UNDEF and the local partners.

12 | P a g e
The lack of work at the national level
anticipated in the project document
reduced the effectiveness of the
programme. In addition, the project did
not take advantage of .the internet or
social media to develop a common
platform for the project to connect the
state networks and draw in supporters
from other locations and sectors on
project issues and action. Although many
of the beneficiaries may not have had
access to the internet, most of the
participating CSOs did, as did local
officials, and leveraging information
technology would have been a
programmatic and cost-effective way to
deepen participation, create a shared
sense of purpose and exchange
information-- both among the national
participants as well as with regional and
global actors as anticipated in the project
design.

Project participant in Huautla, Hidalgo

(iii) Efficiency
The original budget submitted by Oxfam México (USD 466,000 plus USD 200,000
counterpart in cash and in kind from Oxfam México) in its proposal to UNDEF was
substantially higher than what was actually granted (USD 350,00010 plus USD 30,000 in
counterpart funding from Oxfam Novib). Despite the reduced amount, the project activities
and intended outputs remained the same. The main changes to the budget were reducing
the staffing and travel line item amounts and increasing contractual services-- probably
intended to compensate for the reduced number of project staff. According to Oxfam México,
it had intended to hire an expert in
advocacy and another for monitoring.
The evaluation did find that these two
areas were among the weakest for the
project and that most of the work at the
national level was not done. However,
the link between the reduced amount
of funding and staffing and this finding
is not evident. Had more project
persons been hired and dedicated to
the advocacy component and
monitoring, it is likely that those
activities would have been much
stronger. At the same time, a lot more
could have been done for advocacy and on the national efforts within the existing resources,
so funding considerations were not the only factors in terms of project results.

Oxfam México and its three CSO partners felt there was a good use of time and resources.
Oxfam México kept 40% of the grant funding to implement its part of the project and divided

10
Plus another USD 25,000 was retained by UNDEF for evaluation costs.

13 | P a g e
the remaining 60% equally among its three CSO partners. The Novib funding was used to
help cover other Oxfam México project-related staff and administrative costs. They also felt
that a two-year timeframe was needed to implement these types of activities. However, even
though the project document was signed well in advance of the official start date of the
project (signed end September 2009 for a 1 November 2009 project start date) they were
unable to complete the project as scheduled and required a four-month no-cost time
extension. Memoranda of Understanding (MOUs) with the three CSO partners were not
signed until 5 February 2010 and the diagnostic reports not completed until 2011. Delays in
delivery of reports also resulted in a delay in the delivery of UNDEF’s second tranche of
funding, which in turn delayed subsequent activities.

In Guerrero, project activities appear to have

been implemented erratically. The reasons are I give the project a 9 (out of 10) for
unclear but the project coordinator was ill, efficiency. There wasn’t much money but
which might account for some of the problems there were a lot of enthusiastic people”
Participant Hidalgo
along with personality and partisan clashes
We wanted to work in more
reported within its network. This affected the municipalities, 5 was too limiting. So we
efficiency and coherence of the activities. For worked in 7 and reached more people.
example, a journalist interviewed was invited CIFAM
by CAMCO to cover the first two training
sessions and taped the sessions which he
later broadcasted through his network of community radio stations. This was both an
effective and efficient use of local interest and resources, but then the journalist said he
never heard from the project again and was not even aware that they had developed an
agenda.

Oxfam México did provide some basic administrative training and mentoring for its partners
to help them with their funds management and reporting. By serving as a grant manager,
Oxfam México saved the partners
from having to deal directly with a
donor and its reporting requirements
which most of the partners
appreciated. The partner CSOs did
not have a formal agreement with the
CSOs they brought into their network
to help implement the project.
Although they were volunteers and
many appeared to have worked
diligently throughout the project,
others were said to have lacked
professionalism and their participation Extract from the Hidalgo Manual
was unreliable.

As noted, the approach of having each partner CSO develop their own programmes
independently resulted in duplicated efforts especially in regards to capacity building and
project products. The quality of work and focus varied considerably among the different CSO
partners. The training material in Chiapas for example, was intended to result in a certificate,
and was more academically focused. It included group exercise within the text and testing at
the end. The three manuals in Hidalgo were geared towards a lower-educated indigenous
audience and included use of graphics. Their handbook for public information campaigns
was also translated into the Nahuatl language. The manual from Guerrero was made up
more of texts from different laws and agreements. Reporting from the partners on their
activities and achievements to Oxfam México also varied widely despite a common format

14 | P a g e
provided. But in general, there was not enough regularity or specificity in reporting to allow
for adequate performance monitoring. Some of the training materials and guidance were also
provided very late in the process which limited their use by partner CSOs for this project,
such as the advocacy manual and indicator guidance which were published in 2012.

(iv) Impact
The project did have an impact for some participants and participating CSOs. But the lack of
adequate data on project results makes it difficult to identify or to assess the extent of
results.. Oxfam México did make an initial effort to develop a baseline for this project, but
these were macro-level indicators on social conditions that were unlikely to capture results at
the level of this project. This baseline also did not include information on the level of civic
engagement to improve services, the citizens’ attitudes towards participation and advocacy,
their level of knowledge on their rights for services or on how to address their demands to
government. The baseline was also not repeated at the end of the project, making it
impossible to measure change. Reporting was primarily on activities and outputs and was
not detailed enough to extrapolate impact. To measure impact, indicators needed to be more
focused on achievement of the intended outcomes of increased civic awareness and
collective voice for equitable services and any policy changes resulting from the increased
demand. Oxfam México did issue an indicators guide that provided guidance on the different
indicators, such as the UNDP Human Development Index, that could be used to measure
progress, but this was produced at the end of the project (February 2012) so its use by
project partners will be for future projects.

In the interviews, the three CSO partners felt they had been strengthened by participating in
the project as working on an agenda gave their work more focus and the funding allowed
them to continue their work in the sector. It seemed evident that the process of developing an
agenda and the participatory manner in which it was done had helped to them to think more
strategically about the issues of equitable public services and the needs in their communities.
Those who took the training also thought it was empowering as they felt it provided them with
skills to interact with public officials that they could replicate for other community needs in the
future.

CSO partners felt the project, agenda and UN funding gave their organization more visibility
and access to public officials and increased official receptivity to hear their issues and
proposals. This led to more productive discussions and in turn increased perceptions of the
CSOs’ legitimacy and ability to effectively represent the interests of their constituencies. In
some cases, the evaluators heard of health and education issues being given more attention
by local government planning officials and in a few cases to specific action, such as the
cancer screening brought to the municipality of San Fernando as noted below.

Some of the examples provided to or noted by the evaluators included:

 Increased sense of institutional legitimacy and empowerment by the CSOs which had
a positive effect in the relations with government, ability to defend citizen rights and
articulate the interests of marginalized groups.
 Strengthened ability for the three partner CSOs in terms of financial and
programmatic reporting.
 Participation by state officials in project activities in all three states. According to
accounts this was a significant change as these organizations had had difficulty in the
past getting official attention. In some cases, this contact has continued after the end
of the project. The official participation also gave the agenda items more visibility.
 Introduction of the concept of three intercultural universities to state education officials
within the state of Hidalgo rather than one to better reflect the diversity of indigenous

15 | P a g e
 groups in the state, specifically for the indigenous regions of Nahuatl, Otomi and
Tephua. This type of discussion had been held at the national level before, but
reportedly not within the state. The indigenous network Hñahnu felt they had obtained
a state promise to develop these three universities and that this provided needed
recognition from local authorities that the indigenous groups were homogenous and
each had their own languages, cultures and needs.
 Changed perceptions of health and education as a fundamental human right reported
by the CSOs for project participants and beneficiaries of the training and advocacy
campaigns. This outcome seems likely to the evaluators, however, cannot be
independently validate due to the lack of data on perceptions.
 Inclusion of a Guerrero
agenda item on gender into a
state plan entitled “100
puntos para el avance de las
mujeres de Guerrero” (100
points for the advancement
of women in Guerrero).
 Inclusion of health issues into
the municipal agenda in
Yahualixo in Hidalgo after
project participants question
the lack of attention to health
issues in municipal planning.
 Inclusion of a Chiapas Participants and youth from Hidalgo whose
agenda item on education advocacy led to the creation of a community library
into a recent agreement
CIFAM signed with the State
Human Rights Committee to work on education issues.
 Creation of a youth group within the Hidalgo indigenous network “Hñahnu” that is
active on equity issues. They felt their town needed a library and advocated for one.
This resulted in local officials creating a small library for their community.
 Improved health services and cancer screening for a San Fernando municipality in
Chiapas after the project drew attention of health officials to high incidents of mother
mortality and cancer in the locality. The health department sent a team into the town
to provide screening for the local women and to provide treatment.

(v) Sustainability
Without knowing the extent of the project results, it is difficult to assess issues of
sustainability. However, as the project worked at the local level with local CSO and
community members it is likely that the knowledge and skills resulting from the project
activities will remain with them and within those communities. Increased knowledge and skills
and participating in a project such as this
are empowering, which is usually a I joined the project last year and saw it was
significant enough change that it makes a a productive partnership. I brought my
lasting impact. The skills learned for this friends and 13 of us created a youth group
project should serve the participants as they to help support project objectives.
continue their volunteer work and other Indigenous youth, Hidalgo
community involvement. New practices and
policies adopted by local officials are also The training program was good, and we
likely to remain, as once an item such as have a better understanding of issues. But
health care has been included in a municipal nothing came of it as we never used it.
Participant Guerrero
agenda, it is unlikely to be removed. This
16 | P a g e
contributes to the consolidation of democracy and improved governance in a country context
such as Mexico, where the level of public attention on access to public services for
marginalized groups and more participatory decision making is still uneven. The
strengthening of the three CSO partners through their implementation of the project is likely
to be sustained, especially with CIFAM and ACADERH. The evaluation found both
organizations still discussing issues of equitable services in their states and how to work with
the government and communities for reforms. They recognized that the agenda provided
them with a strategic planning tool for future work in the sector, but seemed unsure of next
steps. The post-project situation for CAMCO was less clear. The project developed a number
of products that still remain and will be
useful for the NGOs in their future work. The UN funding opened doors. It gave us
These include the diagnostic reports on the more visibility. This is very important for a
project trying to do public reform in the human
health and education sector, the different
rights sector. I give UNDEF a 10.
sets of training manuals and materials, and CIFAM
the guidance manuals done by Oxfam
México for indicators and advocacy.

The CSOs’ main sustainability issue is the lack of financial resources which is a critical
constraint for many local CSOs. They saw the agenda as a useful means to raise funds to
continue although none appeared to have started that process as of the time of the
evaluation. At the same time, in speaking to Transparency Mexico and CONEVAL which are
working in the social development sector, both felt
funds were available, and/or other programmes
that CSOs such as these could work with. As an
example, they pointed to the Instituto Nacional de
Desarrollo Social (Indesol) which provides
government funding to CSOs for social
development and other projects. Transparency
Mexico has also mapped 1,164 social
programmes so far at the federal and local levels
in the directory it is developing with the Secretary
of Social Development and the United Nations
Development Programme. All of these government
projects have components that require civic
participation.

The electoral process (campaign and change of

officials following election of a new government)
was cited by both CAMCO and Oxfam México as
a constraint to more sustainable results. CIFAM
and ACADERH did not feel that this was a
significant factor, although ACADERH mentioned
the need to sensitize any new officials that took
office. This is indicative of the need to institutionalize the gains made by the project so they
are not dependent on personal relations and will continue regardless of who is in office.

(vi) UNDEF added value

There was widely recognized UNDEF-added value to this project. Oxfam México and the
main CSO partners felt that funding from a UN agency gave their organizations and project
activities more credibility and visibility with the government officials. It increased their access
to public officials and the receptivity of that official to project messages and work. Advocating
against social discrimination, especially in regards to indigenous rights, is a sensitive issue in

17 | P a g e
Mexico, and all felt that having a UN flag attached to the effort helped to present it as a non-
political and neutral endeavour.

V. Conclusions

Based on the evaluation findings, the team concludes:

(i) The project’s focus and activities as designed were relevant and
important given the social and democratic context with Mexico. It addressed issues of
social equity and the marginalization of large segments of society (women, youth and
indigenous groups) within the three targeted states. However, it did not adequately factor in
the political context and how this affected a civic participation/advocacy project. This reduced
its relevance at the national level and within Guerrero and limited its potential impact. This
conclusion follows from findings (i), (ii), and (iv).

(ii) The use of state- and local-level CSOs to implement the state-level
activities was an effective approach for a community-based project as it used
organizations with local networks and that knew the local context. However, the lack of
synergistic programming between the partners and with efforts at the national level
limited the usefulness of this approach as it resulted in duplicated efforts, inconsistent
programming, and isolated activities. The relationship between the CSO partners and their
networks of CSO promoters that participated in project implementation should have been
formalized and used as an opportunity for institutional strengthening. The lack of activities at
the national level was also a lost opportunity to build wider support for the changes
advocated at the state levels. This conclusion follows from findings (i), (ii), (iii) and (iv).

(iii) The use of a diagnostic to develop a training programme and

agenda for advocacy was a good programmatic technique, but required more focus on
the main of objective of strengthening the collective voice and action of civil society.
This diagnosis gave the project the statistical data from which it could select the most
affected municipalities in terms of service delivery, but this drew focus of the participants to
the two sectors within which the project decided to work (health and education) rather than
on its principal goal of strengthening civic voice and demand for more equitable services.
This affected project coherence, effectiveness and impact. This conclusion follows from
findings (ii) and (iv).

(iv) The project made a difference for individual participants and for
some collective interests, but the extent is unknown due to the lack of outcome data.
The project primarily tracked activities and outputs which is insufficient to determine
outcomes. However, anecdotal information indicates that the project had a positive impact,
especially for individual participants in terms of personal growth, improved relations between
some CSOs and local officials, and in a few cases to policy changes. This conclusion follows
from findings (ii), (iv), and (v)

(v) Project results are likely to be sustainable at the level of personal

empowerment and relationships built with some public officials. The changes to public

18 | P a g e
policy reported to the evaluators, such as the inclusion of health in municipal planning, are
also likely to be sustained. The agenda developed also provides CSOs and government with
a roadmap for future action, but the project activities were not sustainable due to the financial
conditions of the participating CSOs. The conclusion follows from findings (iv) and (v).

(vi) UNDEF-funding provided significant value added to this project.

The UN is respected in Mexico and in this case provided a sense of neutrality and legitimacy
to the rights-based discussions and to the CSOs that participated in the effort. This
conclusion follows from finding (vi).

VI. Recommendations

To strengthen similar projects in the future, the team recommends:

(i) Ground civic participation and advocacy projects more firmly in

the democratic governance context. Civic participation and advocacy are done within the
political and democratization context of a country and this aspect should be recognized and
integrated into project design and implementation. A stronger civic education component
could help participants understand their roles and responsibilities in a democratic system and
the means by which they can hold the government accountable for its actions. This includes
the issues of equitable service delivery and policy making. The electoral process is a part of
the political process, and should be seen as an opportunity for public debate on agenda
issues, obtaining candidate endorsements should be done for agenda items and following up
on those commitments once they enter office. State and local governments work within the
larger policy context set by the federal government and work at the national level should be
done to support these lower-level state efforts. This recommendation follows from
conclusions (i), (ii) and (iv).

(ii) Continue to build local ownership in projects but within an

integrated framework that builds a cohesive, synergistic programme. In a project such
as this, the UNDEF grant recipient should bring the different implementers together into one
integrated programme, by designing activities together, producing joint products and
branding the project to provide a rallying and unifying point for supporters and agents of
change. It should also leverage the different institutional strengths of each of the partners to
improve the project and its products. For example, CSOs could adapt the common materials
according to their institutional expertise- such as using ACADERH to adapt training materials
for indigenous groups or languages, CAMCO could strengthen the gender aspects and
CIFAM could develop a certificated version. Certificates can be valuable for those working in
the sector, especially in rural areas, as it demonstrate some level of knowledge or
professional competence and gives participants a tangible sign of their participation. This
recommendation follows from conclusions (ii), (iii) and (iv).

(iii) Use a results-based project performance monitoring plan to track

progress made towards achievement of outcomes and to measure results. This should be
used in addition to tracking activity outputs. Examples of performance indicators could

19 | P a g e
include: increased scores on a knowledge, attitude and practices survey of
promoters/participants/officials at the end of the project compared to scores at the start of the
project (or compared with a control group of non-participants); number/type of policy changes
resulting from project activities and of persons affected by this change; and increase in
services for the marginalized group by the end of the project. This recommendation follows
from conclusions (iii), (iv), and (v).

(iv) Formalize relationships between subgrantees and their

implementers with a written agreement that clearly articulates the roles and responsibilities
of each, the project purpose, timelines, targets and performance indicator and reporting
requirements even if this is only to be done verbally. This would strengthen the programme
as well as the participating CSOs. This recommendation follows from conclusions (i), (ii) and
(vi).

(v) Build a long term vision for work in the equity sector during the
project so the project leaves an-after project strategic action plan with partner CSOs and
participants so that they have a direction to continue their work after the end of the project.
This would help maintain the momentum generated by the project and sustain some of the
gains made. This recommendation follows from conclusion (iv).

VII. Overall assessment and closing thoughts

Overall, this project was a worthwhile use of UNDEF funding. It worked to empower
marginalized groups and protect their rights which are important elements in a consolidating
democracy. It implemented most of its planned activities and met and/or exceeded most of
its anticipated outputs. It seemed to have made an impact despite the lack of indicators to
measure it. It has beneficiaries still talking about the issues and waiting for new initiatives
that can help to continue the work started. However, several issues affected its ability to
make greater achievements. One was the uncertainty over its purpose. The project straddled
different sectors with its outcomes stated in terms of democratic development and its
activities funded by a democracy fund, but most of the activities and indicators were more
indicative of a social development programme that and one usually funded by social
development agencies such as UNICEF.

The other issue is the more hands-off programmatic role adopted by Oxfam Mexico. This
raises the question as to the value added for a donor to go through an umbrella organization
for a project such as this, especially when the sub-grantees appear relatively advanced and
capable of running their own parts of the project with minimum supervision, albeit with some
issues in some cases. The project design foresaw a much more active role for Oxfam
Mexico, providing the networking and linkages between them and working at the national
level in complementary activities, which is the valued added. But in practice, it only assumed
a small portion of this.

This illustrates the importance of clearly thinking through and articulating a project’s purpose
before it is designed, of working as an integrated team during project implementation, and in
providing the programmatic leadership and synergies to maximize project impact.

20 | P a g e
VIII. Limitations, constraints and caveats

The evaluation took place well after the end of the project and relied on project documents
and interviews to make its assessments. Reporting tended to be general and lacked data on
results. Interviews also provided some anecdotes that the team used to extrapolate findings.
There was difficulty reaching the persons who participated in Guerrero. The contact
information for all but the project coordinator was questionable. The team worked through
nine persons listed on the contact sheet for Guerrero as project promoters or participants
before finding one that knew the project or the CSO partner. The evaluators were also only
able to reach a limited number of public officials who knew of the project in the three states
which also made assessment of the advocacy and dialogue activities with these officials
problematic. However, overall the information provided from the different sources and
locations was consistent, which with all of the documents provided by Oxfam Mexico, which
helped to validate the evaluation’s findings.

21 | P a g e
 IX. ANNEXES

Annex 1: Evaluation questions:

DAC Evaluation Question Related sub-questions
criterion
Relevance To what extent was the  Were the objectives of the project in line with the needs and
project, as designed and priorities for democratic development, given the context?
implemented, suited to  Should another project strategy have been preferred rather
context and needs at the than the one implemented to better reflect those needs,
beneficiary, local, and priorities, and context? Why?
national levels?  Were risks appropriately identified by the projects? How
appropriate are/were the strategies developed to deal with
identified risks? Was the project overly risk-averse?
Effectiveness To what extent was the  To what extent have the project’s objectives been reached?
project, as implemented,  To what extent was the project implemented as envisaged
able to achieve by the project document? If not, why not?
objectives and goals?  Were the project activities adequate to make progress
towards the project objectives?
 What has the project achieved? Where it failed to meet the
outputs identified in the project document, why was this?
Efficiency To what extent was  Was there a reasonable relationship between project inputs
there a reasonable and project outputs?
relationship between  Did institutional arrangements promote cost-effectiveness
resources expended and accountability?
and project impacts?  Was the budget designed, and then implemented, in a way
that enabled the project to meet its objectives?
Impact To what extent has the  To what extent has/have the realization of the project
project put in place objective(s) and project outcomes had an impact on the
processes and specific problem the project aimed to address?
procedures supporting  Have the targeted beneficiaries experienced tangible
the role of civil society in impacts? Which were positive; which were negative?
contributing to  To what extent has the project caused changes and effects,
democratization, or to positive and negative, foreseen and unforeseen, on
direct promotion of democratization?
democracy?  Is the project likely to have a catalytic effect? How? Why?
Examples?
Sustainability To what extent has the  To what extent has the project established processes and
project, as designed and systems that are likely to support continued impact?
implemented, created  Are the involved parties willing and able to continue the
what is likely to be a project activities on their own (where applicable)?
continuing impetus
towards democratic
development?
UNDEF To what extent was  What was UNDEF able to accomplish, through the project,
value added UNDEF able to take that could not as well have been achieved by alternative
advantage of its unique projects, other donors, or other stakeholders (Government,
position and NGOs, etc).
comparative advantage  Did project design and implementing modalities exploit
to achieve results that UNDEF’s comparative advantage in the form of an explicit
could not have been mandate to focus on democratization issues?
achieved had support
come from other
donors?

22 | P a g e
Annex 2: Documents Reviewed :

ACADERH. Agenda por la Educación y Salud con Calidad y Equidad para las Mujeres, Jóvenes e
Indígenas en Hidalgo. Hidalgo, 2011

ACADERH Informe Final de Proyecto. Iniciativas Temáticas y Sectoriales. Hidalgo, octubre de 2011.

ACADERH. Informe de resultados de la Primera Escuela de Participación Ciudadana a Favor de los

Derechos Esenciales. Hidalgo, junio de 2010.

ACADERH. Manuales para las Escuelas de Participación Ciudadana a Favor de los Derechos
Escenciales. Hidalgo, 2010.

Amnesty International, Annual Report 2011, Mexico. http://www.amnesty.org/en/region/mexico/report-

2011

Becerril Albarrán, Nahela. Diagnóstico sobre la Situación de la Educación y la Salud en los estados de
Chiapas, Hidalgo y Guerrero. Oxfam México. Abril 2010.

Becerril Albarrán, Nahela. Guía práctica para seguimiento de indicadores en educación y salud.
Oxfam México. Marzo 2012.

CAMCO. Agenda Preliminar: Educación y Salud con Calidad y Equidad para las Mujeres, Joóvenes e
Indígenas en Guerrero. Guerrero, marzo 2011.

CAMCO. Informe Final de Proyecto. Iniciativas Temáticas y Sectoriales. Guerrero, octubre de 2011.

CAMCO. Manuales para las Escuelas de Participación Ciudadana a Favor de los Derechos
Esenciales. Guerrero, 2010.

CAMCO “Producción Colectiva del Conocimiento - Taller de Reflexión y Análisis “El acceso de las
mujeres a los servicios de salud y educación, en cinco municipios del estado de Guerrero”. Junio
2010

CIFAM. Agenda para Incidencia a favor de la Salud y Educación con Calidad y Equidad. Chiapas.
Undated
CIFAM. Diagnóstico Participativo para la Agenda de Incidencia a favor de la Salud y Educación con
Calidad y Equidad en Chiapas. Chiapas, 2011
CIFAM. Informe Final de Proyecto. Iniciativas Temáticas y Sectoriales. Chiapas, octubre de 2011.
CIFAM. Manuales para las Escuelas de Participación Ciudadana a Favor de los Derechos Esenciales.
Chiapas, 2010.
CIFAM. Rueda de prensa para presentar campaña. Tuxtla Gutiérrez, Chiapas. 26 de mayo de 2010

CIVICUS, A Snapshot of Civil Society in Mexico, Analytical Report on the CIVICUS Civil Society Index,
2011 https://civicus.org/images/stories/csi/csi_phase2/mexico%20acr.pdf

CONEVAL, Anexo estadístico- Pobreza 2010,

http://www.coneval.gob.mx/cmsconeval/rw/pages/medicion/Pobreza_2010/Anexo_estadistico.es.do

Forum on Universal Health Care Mexico City Political Declaration on Universal Health Care Coverage:
Coverage, Sustaining Universal Health Coverage, Sharing Experience, and Promoting Progress, April
2012
http://www.who.int/healthsystems/topics/financing/MexicoCityPoliticalDeclarationUniversalHealthCover
age.pdf

Freedom House, Freedom in the World, Country Report: Mexico, 2011,

http://old.freedomhouse.org/template.cfm?page=22&country=8091&year=2011

23 | P a g e
Global Health Workforce Advocacy Initiative, Guiding Principles for National Health Workforce
Strategies, http://www.who.int/healthsystems/round9_6.pdf
Herrera Gutiérrez, Tonatiuh. Las razones y los elementos, diagnóstico para la evaluación de la
Educación y la Salud en el estado de Hidalgo, los casos de Acaxochitlan, Huautla, Yahualica,
Zimapan, Ixmiquilpan y Pachuca. ACADERH, Hidalgo (without date)

Indesol, Our Mission, http://www.indesol.gob.mx/en/web_indesol/Nuestra_Mision

Monroy, Adriana. Diagnóstico de la calidad de los servicios públicos de salud y educación en cinco
municipios de la Región Norte del estado de Guererero: Copalillo, Iguala de la Independencia, Pedro
Ascencio de Alquisiras, Taxco de Alarcón y Tepecoacuilco. CAMCO, Guerrero (without date)
OECD, Better Life Index, Mexico Civic Engagement, http://www.oecdbetterlifeindex.org/topics/civic-
engagement/

OXFAM México. Incidencia de la Sociedad Civil a favor de la Educación y Salud con Calidad y
Equidad. Chiapas, Hidalgo y Guerrero. Informe Público. México, 2012.
OXFAM México. Memoria del Encuentro Nacional de Incidencia de la Sociedad Civil a favor de la
Educación y Salud con Calidad y Equidad. México DF, 27-28 de febrero de 2012.
Oxfam México , UDF-MEX-08-279, Proposal, Civil society advocating for quality education and
healthcare with equity in Mexico, 2008
11
Oxfam México , UDF-MEX-08-279, Project Document Civil society advocating for quality education
and healthcare with equity in Mexico, September 2009
Oxfam México , UDF-MEX-08-279, Mid-Term Report, Civil society advocating for quality education
and healthcare with equity in Mexico, March 2011
Oxfam México , UDF-MEX-08-279, Project Extension Request Form, October 2011
Oxfam México , UDF-MEX-08-279, Milestone Financial Reporting Report, March 2011
Oxfam México , UDF-MEX-08-279, Milestone Financial Reporting Report, September 2011
Oxfam México , UNDEF y Oxfam México dan voz a la sociedad civil de Chiapas, Guerrero e Hidalgo,
Comunicado de Prensa, 29 February 2012
http://www.oxfammexico.org/oxfam/descargas/comunicado_oxfam_undef.pdf
OXFAM, Policy and Practices: Water, Health and Education, http://policy-practice.oxfam.org.uk/our-
work/water-health-education

Santibanez, Lucrecia; Vernez G., Razquin, P. Education in Mexico, Challenges and Opportunities,
RAND Education, 2005

Transparency International, Corruption Perception Index 2011,

http://cpi.transparency.org/cpi2011/in_detail/

Transparency Mexico, with UNDP, Iniciativa para el Fortaleximiento de la Institucionalidad de los

Programas Sociales, http://www.programassociales.org.mx/

UNDP, Human Development Index, 2011, http://hdr.undp.org/en/statistics/

UNICEF, Indigenous adolescents push for recognition and equity in Mexico,

http://www.unicef.org/infobycountry/mexico_61007.html
UNICEF, Radio project gives a voice to indigenous children in Mexico,
http://www.unicef.org/infobycountry/mexico_47386.html

UNESCO, Expanding equitable early childhood care and education is an urgent need, Education for
All Global Monitoring Report, Policy Paper 03, April 2012

World Bank, Additional Financing for Mexico’s Oportunidades Helps Millions Gain Access to
Education, Health, 2010, http://www.worldbank.org/en/news/2010/11/10/additional-financing-mexicos-
oportunidades-helps-millions-gain-access-education-health

11
Formerly Rostros y Voces

24 | P a g e
Annex 3: Persons Interviewed

1 July 2012
Arrival, international consultant
2 July 2012
Meetings with Oxfam México
Rebecca Berner Institutional Funding Manager
Israel Guadarrama Project Coordinator
Rodrigo Galindo Caldedras Programme Division Manager
Miriam Reyna Administrative Manager
3 July 2012
Travel to Pachuca, Hildalgo
Irma Eugenia Gutierrez Mejia Project Coordinator, ACADERH
Pablo Elias Vargas Gonzalez President, ACADERH
Reyna Torres Project team member, ACADERH
Marcela Hernandez Project Promoter, Trainer, ACADERH
Elvia Beltran Vileda, Ixmiquilpan municipality Setas y Champinones
Alejandro Rendon, Ixmiquilpan municipality Red Indigena Hnahnu
Silvano Hernandez Herndez, Yahualica
Organizacion Nahual Campesina de la Huasteca
municipality
Tanya Meza, ACADERH – « Desde abajo » Member ACADERH, Independent journalist
Alberto Rodriguez, ACADERH – « Desde abajo » Member ACADERH, Independent journalist
Pedro Luis Noble Ministry of Health
Diana Reyes Ministry of Health assistant
Return to Mexico City
4 July 2012
Alejandro Rivera Marroquin Director, CIFAM, Chiapas
Guadalupe Coutino Pereyra Project Coordinator, CIFAM, Chiapas
Walter Roblero Lao Administrative Officer, CIFAM, Chiapas
Mireya Albores Herrera Community Leader, Chiapa de Corzo, Chiapas
Jaqueline Herandez Cruz Community Leader, San Fernando, Chiapas
Agustina Valencia Abarca Beneficiary, Apaxtla de Castrejon, Guerrero
Jose Luis Tenorio Bello Fundacion Domi Bello Tenorio, Chiapas
Ubali Guerrero Gonzalez Mujeres Indigenas en Lucha, Copalillo, Guerrero
Beneficiary, Asociacion de Ejidatarias de
Rufina de Salatiel
Ixcapuzalco, Guerrero
Organizacion Para El Desarrollo Integral de Los
Luis Zacaria Quiroz Pueblos Indigenas Campesinos Y Popular,
Chipancingo, Guerrero
5 July 2012
Rafael Reygadas Autonomous Metropolitan University Professor
General Director at National Council for
Thania de la Garza Navarrete Evaluation of Social Development Policies
(CONEVAL)

25 | P a g e
Paola Palacios Programme Director, Transparency Mexico
Erika Lopez Transparency Mexico
Monsterrat Hernandez Transparency Mexico
Israel Balderrama Project Director, Oxfam México
Mirian Reyna Administration and Finance, Oxfam México
Rodrigo Galindo Programme Division Director, Oxfam México
6 July 2012
Theo Bouma Oxfam Novib, Co-funder of Project
Director, Café, Mujeres y Comunidad (CAMCO),
Reyna Edith Carbajal Medez
Guerrero
Eva Albavera Project Coordinator, former CAMCO, Guerrero
Project Consultant (local diagnostic and training
Adriana Monrroy
program)
Departure, international consultant
9 July 2012
Silvia Romero Ministry of Education, Guerrero
Ministry of Social Development Assistant,
Liliana Valiente
Guerrero
Diana Reyes Ministry of Health Assistant, Hidalgo
Beneficiary, Organización de la Mujer Taxqueña,
Lorena Marisa Pineda (by e-mail)
Guerrero
Benificiary, Mujeres Campecinas de
Delfina Bartolo Carbajal (by e-mail)
Tepecoacuilco, UGOCEN.
Yasmin Vilchis Garcia (by mail) Benificiary, Colectivo Red Abierta, Chiapas
Coyolxauhqui Valencia Eligio (by mail) Benificiary, Visión Mundial, Chiapas
Benificiary, Asociacion “Skolta’ el Yu’un Jlumaltic
Robina Soria Vargas (by mail)
(SYJAC)

26 | P a g e
Annex 4 : Acronyms

ACADERH Academia Hidalguense de Educación y Derechos

CAMCO Café, Mujer y Comunidad, A.C.
CSO Civil Society Organization
CONEVAL Consejo Nacional de Evaluación de la Política de Desarrollo Social
CIFAM Colectivo de Atención para la Salud Integral de la Familia, A.C.
NGO Non-Governmental Organizaiton
UNDEF United Nations Democracy Fund

27 | P a g e