TOR Impact Evaluation
TOR Impact Evaluation
TOR Impact Evaluation
Terms of Reference
A. Background
1. The Government of Indonesia has received a loan from the Asian Development
Bank (ADB) for the Urban Sanitation and Rural Infrastructure (USRI) Support to PNPM
Mandiri Project. The Project aims at (i) upgrading basic rural infrastructure in poor villages
in four provinces, and (ii) promoting community-driven sanitation development to
contribute to achieving the Millennium Development Goal (MDG) target to halve, by 2015,
the proportion of people without sustainable access to basic sanitation.
3. The USRI is financed by a $100.0 million loan from ADB’s Ordinary Capital
Resources. The Project was approved on 5 August 2011, declared effective on 15 November
2011, and is scheduled to close on 30 June 2015. The project is implemented in about 800
rural communities in four provinces, and 1,400 poor urban neighborhoods in 34 cities in five
provinces. The project design monitoring framework is in Appendix 1, project locations are in
Appendix 2.
4. The impact of the Project is reduced poverty in the project areas; the expected
outcome is improved access to service delivery and healthy live conditions for the poor, near
poor, and women in the project communities. The project has three outputs: (i) strengthened
capacity for community planning and development, (ii) improved rural basic infrastructure,
and (iii) improved sanitation services.
8. Infrastructure upgrading financed by the project include building and repairing rural
roads, pathways, small bridges, irrigation facilities, water supply systems and public
sanitation facilities. Supported by CFs, the CIO takes the lead in developing village
investment plans in close cooperation with the community members. After the village
investment plans, which include the final technical designs and cost estimates, have been
approved at a village assembly and submitted to the DPIU for approval; a contract will be
executed between the CIO and the district Satker and the first installment of the block grant
will be released. Civil works are undertaken by communities with technical guidance from the
CFs, DPIUs, and consultants. Block grants are transferred directly to community bank
accounts managed by the CIOs.The first installments of 40% of the approved block grant are
transferred directly into the CIO’s bank account as an advance payment. The remaining
funds, subject to certification of progress by the DPIUs, are disbursed to communities in two
additional installments.
10. The objective of the assignment is to assess and evaluate the overall impact,
outcomes and outputs of the project compared to the targets and performance indicators
specified in the DMF, gender action plans and other targets specified in project documents.
1
Regional working unit of MPW.
2
The original target was 600 villages to receive 2 cycles of block grants.
3
Originally 1,350 neighborhoods were targeted to be covered, the minor change added 89 more villages to the
target.
C. Scope of Services, Tasks and Expected Deliverables
13. The expected deliverables are: (i) evaluations of the project results, impacts, lessons
and detailed recommendations for the project exit strategy including the consolidation and if
applicable the replication of the project approach and achievements; (ii) advice on ADB’s
future involvement in the sector. The findings and recommendations should be presented in
a reader friendly and professional manner and may include illustrations and photos,
interviews with stakeholders/beneficiaries, etc. The team will also be required to present
results of the assignment during a national workshop. The consultant should ensure that data
from a proper number neighborhoods and villages (proper sampling method) are obtained so
that conclusions are validated.
14. The services of the consultant will be required for a total of 3.5 months. The
consultant team will be selected and engaged in accordance with the ADB’s Guidelines on
the Use of Consultant (2012, as amended from time to time). The NMC will be recruited
through selection of national firms. The quality and cost-based selection method, which
requires the submission a simplified technical proposal within 25 days,4 will be used to select
the consultant. A total minimum of 18 person-months of consultant inputs will be required,
including 14.5 person-months of national consultants and 3.5 international consultant inputs
are required for this assignment.
4
ADB’s procedures of recruitment of consultants allow shorter period for preparation of proposals, if the electronic
system is applied.
(xii) review the quality of community sanitation improvement action plans (CSIAPs)
and other community sanitation plan documents;
(xiii) assess the quality of project manuals (JUKLAK/JUKNIS etc.), technical/design
standards, particularly for the sanitation part;
(xiv) assess the sanitation block grant size and effectiveness of their fund
channeling;
(xv) assess quality of built sanitation facilities;
(xvi) assess the quality of community facilitator selection;
(xvii) assess the selection process of participating cities and neighborhoods;
(xviii) assess the quality of training for sanitation community facilitators;
(xix) assess the project’s impact on participating cities’ sanitation development and
target achievements;
(xx) assess the project impacts on environment and sanitation sector
development;
(xxi) assess the quality of project monitoring and supervision by the Executing Agency,
participating cities/districts, consultants and other related stakeholders (sanitation
part);
(xxii) Assess the quality of management consultants supporting DGHS in implementing
and managing the project (sanitation part);
(xxiii) Assess the project management of data/information system including flow of
reporting;
(xxiv) lead in implementing other related tasks as requested by ADB and the EA, to
ensure the timely completion of the assignment.
19. The specialist is expected to have extensive experience in project impact evaluation
activities. He/She is preferable to hold a bachelor degree (S1) in civil/environmental
engineering or other related fields. Experience in evaluating rural infrastructure/development
related projects, applying the community-driven (e.g. PNPM Rural) development approach is
preferable. The expert is preferable to have experience in working in project areas (villages
in southern part of Sumatra). The expert is expected to have good English oral and writing
skills, to be a good team player and have excellent communication skills.
21. The specialist is expected to have extensive experience in data collection (surveys)
and analysis assignments. Experience in evaluation of similar project impacts would be an
advantage. He/She is preferable to hold a bachelor degree (S1) statistics or other related
fields. The expert is preferable to have experience in working in project areas. The expert is
expected to have good English oral and writing skills, to be a good team player and have
excellent communication skills.
23. The expert is preferable to hold a bachelor degree social science or other related
fields with extensive experience in projects/activities involving community in urban and rural
areas. The expert is expected to have excellent communication skills, including in preparing
high quality reports, both in the English language and in Bahasa Indonesia. He/She is
expected to (i) be familiar with the ADB’s and Indonesia’s gender and development policies,
(ii) be a team player, and (iii) have experience in working with international agencies.
24. Nine (9) non-key experts will be required to supervise and coordinate the data
collection process in the nine project provinces. The total allocated input for the non-key
experts is 6 person-months. Each coordinator will be responsible to ensure the completion of
data collection in his/her assigned province. The coordinators are tasked to support the TL in
mobilizing, training and assigning survey areas of surveyors. He/She will be responsible to
ensure the high quality of collected data. The consulting firm should properly allocate the
inputs of supervisors in each project province considering the coverage and number of
villages and neighborhoods to be visited by surveyors (e.g. in South Sulawesi and North
Sulawesi only one city participating in the project, while in Central Java 19 cities/districts).
25. The following are the required key qualifications for data collection supervisors: (i)
hold a bachelor degree in engineering (civil or sanitary engineering is preferable) or other
related fields, (ii) have experience in data collections/surveys, (iii) have excellent
communication skills particularly verbal communication skills, (iv) familiar with local culture
and structure.
26. It should be noted that non-key experts are NOT subject to technical proposal
evaluations. However, payments of non-key experts will be included under the remuneration
category (FIN-3 form).
27. The following reports should be prepared and submitted by the consultant:
(i) An inception report outlining the evaluation work plan and field work
methodology and analytical framework. The report should be submitted within
2 weeks of the issuance of the notice to proceeds (NTP).
(ii) Interim report providing progress of assignment implementation including
preliminary analysis of project impacts. The report is to be submitted within 1.5
months after the issuance of the NTP.
(iii) Draft Final Report, to be submitted within 2.5 months after the issuance of the
NTP. This report contains a comprehensive account of project evaluation
results and outcomes, lessons-learned, experiences gained, and conclusions
as well as recommendations, as specified in the scope of assignment. The draft
final report must be presented to and discussed with the ADB, Executing
Agency and other project stakeholders in a national workshop.
(iv) Final Report, to be submitted within 2 weeks after the completion of the
assignment. The final report should contain comments and inputs provided
during the discussion of the draft final report.
(v) Other reports as required by ADB related to the assignment.
28. The following activities are included in the provisional sums allocation:
(i) Training of surveyors and
(ii) National Workshop.
29. Term of references and estimated budgets for activities under the provisional sums
will need to be endorsed by ADB prior to their implementation.
30. Office spaces and other equipment will not be provided. ADB cost estimate under
the OPE category has included budgets for office rentals, the running and maintenance
cost of the office including electricity, telecommunications and cleaning cost etc. The
consulting firm should also allocate sufficient budget for duty trips/travels to
cities/neighborhoods/villages required in carrying out the assignment.
31. DGHS and ADB will provide at no costs, access to all project documents and data
available at DGHS and ADB to support the implementation of the assignment. DGHS will
also assign counterpart staff at the CPMU and at the national Satker to work with the
consultant. Costs for these counterpart staff will be borne by the DGHS. At the local level, the
project cities/districts will appoint staff at the DPIUs and city/district satkers as counterpart
staff at the city/district level. Costs for these counterpart staff will be borne by the respective
project city/district. The project cities/districts will also provide access to
data/information/documents to support the execution of the work at no costs.
I. Urban Sanitation
A. Central Java:
1. Kota Semarang
2. Kota Pekalongan
3. Kabupaten Cilacap
4. Kabupaten Banyumas
5. Kabupaten Purbalingga
6. Kabupaten Kebumen
7. Kabupaten Purworejo
8. Kabupaten Klaten
9. Kabupaten Sukoharjo
10. Kabupaten Sragen
11. Kabupaten Rembang
12. Kabupaten Blora
13. Kabupaten Kudus
14. Kabupaten Brebes
15. Kabupaten Jepara
16. Kota Magelang
17. Kota Surakarta
18. Kota Tegal
B. DI Yogyakarta
1. Kabupaten Bantul
2. Kabupaten Sleman
C. East Java
1. Kabupaten Blitar
2. Kabupaten Bojonegoro
3. Kabupaten Gresik
4. Kabupaten Jombang
5. Kabupaten Malang
6. Kota Batu
7. Kota Blitar
8. Kota Kediri
9. Kota Malang
10. Kota Pasuruan
11. Kota Probolinggo
12. Kota Surabaya
D. South Sulawesi
1. Kota Makassar
E. North Sulawesi
1. Kota Manado.
A. Riau
B. Jambi
1. Kabupaten Kerinci
2. Kota Sungai Penuh
3. Kabupaten Sarolangun
4. Kabupaten Tanjung Jabung Timur
C. South Sulawesi
D. Lampung
1. Kabupaten Tanggamus
2. Kabupaten Lampung Selatan
3. Kabupaten Tulang Bawang
4. Kabupaten Lampung Tengah
5. Kabupaten Lampung Timur
6. Kabupaten Pesawaran