This checklist provides guidance for conducting an evaluability assessment of a project or program. It focuses on 3 key areas: 1) assessing the project design, including clarity, relevance, plausibility, and other factors; 2) evaluating the availability of key information like baseline data, indicators, and documentation; 3) considering the practical and utility factors of the institutional context, including things like timing, resources, stakeholder buy-in, and risks. The checklist aims to help evaluators determine if a project is ready for evaluation and what limitations or challenges may exist.
This checklist provides guidance for conducting an evaluability assessment of a project or program. It focuses on 3 key areas: 1) assessing the project design, including clarity, relevance, plausibility, and other factors; 2) evaluating the availability of key information like baseline data, indicators, and documentation; 3) considering the practical and utility factors of the institutional context, including things like timing, resources, stakeholder buy-in, and risks. The checklist aims to help evaluators determine if a project is ready for evaluation and what limitations or challenges may exist.
This checklist provides guidance for conducting an evaluability assessment of a project or program. It focuses on 3 key areas: 1) assessing the project design, including clarity, relevance, plausibility, and other factors; 2) evaluating the availability of key information like baseline data, indicators, and documentation; 3) considering the practical and utility factors of the institutional context, including things like timing, resources, stakeholder buy-in, and risks. The checklist aims to help evaluators determine if a project is ready for evaluation and what limitations or challenges may exist.
This checklist provides guidance for conducting an evaluability assessment of a project or program. It focuses on 3 key areas: 1) assessing the project design, including clarity, relevance, plausibility, and other factors; 2) evaluating the availability of key information like baseline data, indicators, and documentation; 3) considering the practical and utility factors of the institutional context, including things like timing, resources, stakeholder buy-in, and risks. The checklist aims to help evaluators determine if a project is ready for evaluation and what limitations or challenges may exist.
Download as DOCX, PDF, TXT or read online from Scribd
Download as docx, pdf, or txt
You are on page 1/ 3
An Evaluability Assessment checklist
Rick Davies, Friday, 23 January 2015
This checklist has been extracted from pages 19-23 of the following report: Davies, R., 2013. Planning Evaluability Assessments: A Synthesis of the Literature with Recommendations. Report of a Study Commissioned by the Department for International Development. Available on at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40- planning-eval-assessments.pdf Please refer to that report for information on the origins and rationale for this checklist.
1. Project Design (as described in a Theory of Change, Logical Framework or narrative)
Clarity? Are the long-term impact and outcomes clearly identified and are the proposed steps towards achieving these clearly defined?
Relevant? Is the project objective clearly relevant to the needs of the target group, as identified by any form of situation analysis, baseline study, or other evidence and argument? Is the intended beneficiary group clearly identified?
Plausible? Is there a continuous causal chain, connecting the intervening agency with the final impact of concern? Is it likely that the project objective could be achieved, given the planned interventions, within the project lifespan? Is there evidence from elsewhere that it could be achieved?
Validity and Are there valid indicators for each expected event (output, outcome and reliability? impact levels)? I.e. will they capture what is expected to happen? Are they reliable indicators? I.e. will observations by different observers find the same thing?
Testable? Is it possible to identify which linkages in the causal chain will be most critical to the success of the project, and thus should be the focus of evaluation questions?
Contextualised? Have assumptions about the roles of other actors outside the project been made explicit? (both enablers and constrainers) Are there plausible plans to monitor these in any practicable way?
Consistent? Is there consistency in the way the Theory of Change is described across various project multiple documents (Design, M&E plans, work plans, progress reports, etc.)
Complexity? Are there expected to be multiple interactions between different project
components? [complicating attribution of causes and identification of effects] How clearly defined are the expected interactions? Agreement? To what extent are different stakeholders holding different views about the project objectives and how they will be achieved? How visible are the views of stakeholders who might be expected to have different views?
2. Information availability Is a complete set of …relative to what could have been expected? E.g. Project proposal, documents available? Progress Reports, Evaluations / impact assessments, Commissioned studies
Do baseline measures If baseline data is not yet available, are there specific plans for when exist? baseline data would be collected and how feasible are these? If baseline data exists in the form of survey data, is the raw data available, or just selected currently relevant items? Is the sampling process clear? Are the survey instruments available? If baseline data is in the form of national or subnational statistics, how disaggregated is the data? Are time series data available, for pre-project years?
Is there data on a Is it clear how the control group compares to the intervention group? Is control group? the raw data available or just summary statistics? Are the members of the control group identifiable and potentially contactable? How frequently has data been collected on the status of the control group?
Is data being collected Is it with sufficient frequency? Is there significant missing data? Are the for all the indicators? measures being used reliable i.e. Is measurement error likely to be a problem?
Is critical data Are the intended and actual beneficiaries identifiable? Is there a record available? of who was involved in what project activities and when? Is gender disaggregated In the baseline? For each of the indicators during project intervention? data available? In the control group? In any mid-term or process review?
If reviews or Are the reports available? Are the authors contactable? Is the raw data evaluations have been available? Is the sampling process clear? Are the survey instruments carried out… available?
Do existing M&E Where data is not yet available, do existing staff and systems have the systems have the capacity to do so in the future? Are responsibilities, sources and capacity to deliver? periodicities defined and appropriate? Is the budget adequate? 3. Institutional context Practicality
Accessibility to and Are there physical security risks? Will weather be a constraint? availability of Are staff and key stakeholders likely to be present, or absent on leave or stakeholders? secondment? Can reported availability be relied upon?
Resources available to Time available in total and in country? Timing within the schedule of all do the evaluation? other activities? Funding available for the relevant team and duration? People with the necessary skills available at this point?
Is the timing right? Is there an opportunity for an evaluation to have an influence? Has the project accumulated enough implementation experience to enable useful lessons to be extracted? If the evaluation was planned in advance, is the evaluation still relevant?
Coordination How many other donors, government departments, or NGOs need to be
requirements? or want to be involved? What forms of coordination are possible and/or required?
Utility
Who wants an Have the primary users been clearly identified? Can they be involved in evaluation? defining the evaluation? Will they participate in an evaluation process?
What do stakeholders What evaluation questions are of interest to whom? Are these realistic, want to know? given the project design and likely data availability? Can they be prioritised? How do people want to see the results used? Is this realistic?
What sort of evaluation What designs do stakeholders express interest in? Could these work process do given evaluation the questions of interest and likely information stakeholders want? availability, and resources available?
What ethical issues Are they known or knowable? Are they likely to be manageable? What exist? constraints will they impose?
What are the risks? Will stakeholders be able to manage negative findings? Have previous evaluation experiences prejudiced stakeholder’s likely participation?
