Afghanistan Basic

Package of Health
Services (BPHS) Study:
Cost-Efficiency, Quality,
Equity and Stakeholder
Insights into Contracting
Modalities

JULY 2013
Suggested citation: Blaakman A and Lwin A. 2013. Afghanistan Basic Package of Health Services (BPHS)
Study: Cost-Efficiency, Quality, Equity and Stakeholder Insights into Contracting Modalities. Washington, DC:
Centre for Development and Population Activities (CEDPA), Health Policy Project.

The Health Policy Project is a five-year cooperative agreement funded by the U.S. Agency for International
Development under Agreement No. AID-OAA-A-10-00067, beginning September 30, 2010.It is implemented
by Futures Group, in collaboration with the Centre for Development and Population Activities (CEDPA),
Futures Institute, Partners in Population and Development, Africa Regional Office (PPD ARO), Population
Reference Bureau (PRB), Research Triangle Institute (RTI) International, and the White Ribbon Alliance for
Safe Motherhood (WRA).

The information provided in this document is not official U.S. government information and does not
necessarily represent the views or positions of the U.S. Agency for International Development.
Afghanistan Basic Package of
Health Services (BPHS) Study:
Cost-Efficiency, Quality, Equity
and Stakeholder Insights into
Contracting Modalities

JULY 2013

The information provided in this document is not official U.S. Government information and does not
necessarily represent the views or positions of the U.S. Agency for International Development.
CONTENTS

List of Tables and Figures .......................................................................................................................... ii

Acknowledgements .................................................................................................................................... iii
Executive Summary ................................................................................................................................... iv
Abbreviations ............................................................................................................................................. vi
Background ................................................................................................................................................. 1
Primary Study Aim ..................................................................................................................................... 1
Methods........................................................................................................................................................ 1
Secondary Data Sources............................................................................................................................ 3
Quantitative Results.................................................................................................................................... 4
Costs and Cost-Efficiency......................................................................................................................... 4
Cost-Efficiency and Cost-Quality ............................................................................................................. 6
Equity ........................................................................................................................................................ 8
Qualitative Results .................................................................................................................................... 10
Stakeholder Survey (n=76): Central and Provincial Level (non-parliamentarian) ................................. 10
Component 1: Applying evidence for decision-making in BPHS implementation ............................ 11
Component 2: Influence of the Political Process on SM and NGO BPHS implementation ............... 11
Component 3: Application of Continuous Medical Education (CME) and Training for Staff ........... 11
Component 4: Linkages between BPHS and EPHS ........................................................................... 12
Component 5: Resource Allocation and Management ....................................................................... 12
Component 6: Staffing Capacity......................................................................................................... 12
Component 7: Qualification and Referral Rates of Community Health Workers .............................. 13
Component 8: Pharmaceutical Supplies Availability ......................................................................... 13
Parliamentarian Survey Results (n=21) .................................................................................................. 13
Summary Descriptive Statistics .......................................................................................................... 13
Analysis of Difference of Means (SM and NGO) .............................................................................. 14
Strengths and Limitations ........................................................................................................................ 16
Discussion and Recommendations........................................................................................................... 16
Annex A: BPHS Variable List ............................................................................................................. …18
Annex B: Explanation About Relative Per Capita Cost and Relative Efficiency Values ................... 19
Annex C: Results From Independent Samples Test of Differences Between SM and NGOs
(Parliamentarian Survey) ........................................................................................................................ 20

i
LIST OF TABLES AND FIGURES
Table 1. Stakeholder List for the Qualitative Study
Table 2. BPHS General Contracted-Out Population Coverage and Annual Expenditure Estimates 2010
Table 3. Contracted-Out Unit Cost Estimates of BPHS 2010 by Implementer
Table 4. Key Cost-Efficiency Measures (Averages) – Relative BPHS Measure/Relative Cost Per Capita
Table 5. Key Cost-Efficiency and Quality Measures (Averages) – Relative BPHS Measure/Relative Cost
Per Capita
Table 6. Afghanistan Parliamentary Survey 2012 – Descriptive Statistics by SM and NGO

Figure 1. BPHS Relative Cost-Efficiency Measures by Implementer – 2010

Figure 2. BPHS Relative Cost-Efficiency/Cost-Quality Measures by Implementer – 2010
Figure 3. Relative BPHS Measure Cost-Efficiency Ranking Counts by Implementer – 2010
Figure 4. Average number of antenatal care visits during pregnancy (NGO and SM BPHS
implementation) by wealth quintile 2010
Figure 5. Percentage of Utilization of Health Facility for ANC by BPHS Modality (NGO/SM) and Type
of Care
Figure 6. Infant Mortality % by Wealth Quintile within NGO and SM-supported BPHS Provinces

ii
ACKNOWLEDGMENTS

I would like to acknowledge the financial and technical support of the United States Agency for
International Development (USAID)-funded Health Policy Project (HPP) in conducting the “Afghanistan
Basic Package of Health Services (BPHS) Study: Cost-Efficiency, Quality, Equity and Stakeholder
Insights into Contracting Modalities.” We especially thank our HPP colleagues Kathleen Sears,
Omarzaman Sayedi and Christine Kim.

Special thanks go to Aaron Blaakman and Aung Lwin for providing technical support to the MoPH
Health Economics and Financing Directorate (HEFD) throughout this study. Finally, I would like to
appreciate the efforts of Faridoon Joyenda, Husnia Sadat, Khwaja Mir Ahad Saeed, Mohammad Saber
Perdes, and Mohammad Younus Zawoli for their contributions in data collection and analysis.

Representatives of the Afghan Parliament and the Ministry of Finance (MoF), the Ministry of Public
Health (MoPH) Leadership; Director Generals; provincial staff in Jawzjan, Urozgon, Wardak, Parwan,
Kapisa, and Panjshir Provinces; Strengthening Mechanism (SM) Department Staff, including Atiqullah
Akbary and Farid Haidari; Representatives of USAID, the European Union (EU); the World Bank; non-
governmental organizations (NGOs) including the Social Health and Development Program (SHDP) have
had significant contributions to this study.

Ahmad Shah Salehi, MD, MSc

Director
HEFD

iii
EXECUTIVE SUMMARY
Background
One of the key innovations in the reconstruction of Afghanistan’s health care system since 2001 is the
design of the health service delivery structure, including the Basic Package of Health Services (BPHS).
The Ministry of Public Health (MoPH) manages, finances, and provides the BPHS through two service
delivery mechanisms: contracting-out to nongovernmental organizations (NGOs) and through the MoPH-
Strengthening Mechanism (SM) model (contracting-in). Despite the MoPH’s oversight, governance, and
stewardship over both delivery mechanisms, inevitably there are differences (as well as shared
experiences) between both contracting modalities. Although there have been a few studies conducted on
comparing these modalities, important policy and implementation questions remain, including: (1) Are
there differences in BPHS indicators (effectiveness and relative cost-efficiency) achieved under the two
models? (2) Are there differences in the relative cost and quality of BPHS services delivered under the
two models? (3) Is there a correlation between contracting mechanisms and equity in service utilization
among income quintiles of Afghanistan households? To address these questions, this study investigates
important issues related to cost, quality and equity of BPHS services under the two contracting
modalities.

Aim
The primary aim of this study is to understand the differences between the two BPHS implementation
models (contracting-in and contracting-out) with regard to cost, quality, and equity of services.

Methods
Multi-component quantitative and qualitative methods are applied in this study using both primary data
and secondary datasets from 2010. Primary data include qualitative survey data from 93 central and
provincial level stakeholders, while secondary data include data from the MoPH Health Management
Information System (HMIS), the Balanced Scorecard (BSC) 2004 and 2010, and the Afghanistan
Mortality Survey (AMS). Cost data were also collected from both the Strengthening Mechanisms (SM)
department of the MoPH and from the three main BPHS development partner implementers (USAID, EU,
and World Bank) to obtain updated cost estimates of SM for 2010. Quantitative analyses include a
comparison of all BPHS implementers (USAID, EU, World Bank and SM) in the 34 provinces of
Afghanistan. For comparison purposes between the BPHS implementers, measures are “normalized” (the
average raw score for the implementer divided by the national average) on a relative scale with the
national average = 1. Benefit Incidence Analysis (BIA) is applied to examine “equity” differences in
NGO and SM provinces.

Results
Costs. BPHS Implementers incurred expenditures accordingly from highest to lowest during 2010:
USAID ($37,869,817), EU ($24,564,432), WB ($19,780,567), SM ($5,026,838).

Unit Costs: The simple average cost per capita 2010 for EU ($4.69) is 15% higher relative to USAID-
supported provinces ($4.07), 2% higher (relative to WB-supported provinces/$4.61), and 1% higher
(relative to SM supported provinces/$4.64). These costs serve as the denominator for all relative cost-
efficiency analyses.

iv
Cost-Efficiency. When examining the relative cost efficiency of cost per visit per year, USAID generally
appears to be the most cost-efficient BPHS implementer, achieving the highest relative efficiency ranking
on 12 out of 14 indicators. Rankings for the other implementers appear more mixed with no clear
distinction overall between NGO and SM implementation.

Cost-Quality. The cost-quality measure “relative efficiency of 2010 Balanced Scorecard results” shows
that USAID (1.13) reflected slightly higher quality relative to cost than the other three implementers, EU
(0.99) and WB (0.88), and SM (1.05).

Equity. Equity measures indicate mixed results between NGO and SM mechanisms with regard to
implementing a “pro-poor strategy” within BPHS. SM appears to exhibit greater pro-poorness when
examining antenatal care visits, but also experiences higher amounts of infant mortality, particularly in
wealth quintiles 1 and 3 1.

Qualitative Analyses. Qualitative analyses highlight important issues related to the implementation of
BPHS and the two models. Respondents expressed that burdensome bureaucratic processes seemingly
impact the implementation of BPHS under SM (particularly procurement) and that the NGO model
manages human resources more efficiently. Stakeholders also highlight that both models face constraints
in achieving full BPHS implementation including political pressure, system referral and linkage
problems, as well as administrative barriers. In addition, an independent samples t-test of the
parliamentary survey results indicate that none of the means of 17 likert scale measures were statistically
significant at the p<.05 or p<.10 levels. Accordingly, we conclude that either the sample size is too small
to identify any differences or that there are no differences in responses related to the two BPHS
implementation mechanisms.

Discussion and Recommendation

This study involved an assessment of cost, cost-efficiency, cost-quality, and equity of BPHS services
under both SM and NGOs contracting mechanisms in Afghanistan using both quantitative and qualitative
research methods. From each of these perspectives, although some differences are highlighted and lessons
are learned, there appears to be no clear collective difference between SM and NGOs on the
implementation of BPHS. The authors recommend that as the MoPH and development partners move
forward with the BPHS, “best practices” are further examined and applied to overall implementation.

1
The dataset provided by the MoPH at the time of the analysis had some data cleaning limitations that restricted the
level of analysis.

v
ABBREVIATIONS
ANC Antenatal Care
AMS Afghanistan Mortality Survey
BHC Basic Health Center
BIA Benefit Incidence Analysis
BPHS Basic Package of Health Services
BSC Balanced Scorecard
CHC Comprehensive Health Center
CHW Community Health Worker
CME Continuous Medical Education
EU European Union
HEFD Health Economics and Financing Directorate
HMIS Health Management Information System
IRB Internal Review Board
MoPH Ministry of Public Health
NGO Non-governmental Organization
NHA National Health Accounts
NMC National Monitoring Checklist
NRVA National Risk and Vulnerability Assessment
PPHD Provincial Health Directorate
PNC Postnatal care
REAN Relative efficiency of coverage percent of first ANC of population
REAN2 Relative efficiency of coverage percent of first ANC of population
REDC Relative efficiency of cost per daily served client per year ($)
REDS Relative efficiency of coverage percent of daily served clients of the population
REID Relative efficiency of coverage percent of institutional deliveries of population
RELD Relative efficiency of coverage percent of live deliveries of population
REPENTA Relative efficiency of coverage percent of children <2 years received PENTA3
REPN Relative efficiency of coverage percent of first PNC of population
REPV Relative efficiency of coverage percent of visits of the population
REQ Relative efficiency of 2010
REU1 Relative efficiency of coverage percent of immunized children <1 year of population
REU1A Relative efficiency of percent of <1 year immunization of all visits
SM Strengthening Mechanism
USAID United States Agency for International Development
WB World Bank

vi
BACKGROUND
Afghanistan’s health care system has been undergoing reconstruction since 2001. One of the key
innovations in the reconstruction is the design of the health service delivery structure, including the Basic
Package of Health Services (BPHS). At present, the Ministry of Public Health (MoPH) manages,
finances, and provides the BPHS through two service delivery mechanisms: contracting-out to
nongovernmental organizations (NGOs) in 31 provinces, and the provision of services through the
MoPH-Strengthening Mechanism (SM) model, also known as “contracting-in” in three provinces.
Although a few studies have been conducted to understand the differences in cost and outcomes between
the two models 2, important policy and implementation questions remain. These questions include:

1. Are there differences in BPHS indicators (effectiveness and relative cost-efficiency) achieved
under the two models?
2. Are there differences in the relative cost and quality of BPHS services delivered under the two
models?
3. Is there a correlation between contracting mechanisms and equity in service utilization among
income quintiles of Afghanistan households?

To address these questions, the Health Economics and Financing Directorate (HEFD) of the MoPH, with
technical and financial support from the USAID-funded Health Policy Project, conducted a comparative
analysis of the strengths and weaknesses of the contracting-in and contracting-out modalities.

PRIMARY STUDY AIM

The primary aim of this study is to understand the differences between the two BPHS implementation
models (contracting-in and contracting-out) with regard to cost, quality, and equity of services.

METHODS
Multi-component quantitative and qualitative methods were applied in this study using both primary data
and secondary datasets. First, an updated cost analysis for Strengthening Mechanisms (contracting-in)
was conducted using data from 1390/2010, to compare with recent estimates of contracting-out. This
analysis involved examining the annual BPHS costs of SM as implemented in Parwan, Panjshir, and
Kapisa. Subsequently, these data were compared with contracting-out cost data conducted for the 31
donor-supported provinces of Afghanistan. As a result, cost analyses now reflect implementation of the
BPHS across Afghanistan during this period 3.

Secondly, an analysis of Health Management Information System (HMIS) and other secondary data was
conducted to examine differences in effectiveness and relative cost-efficiency of the NGO and SM

2
Blaakman, AP., Salehi, AS, and Boitard, R., A cost and technical efficiency analysis of two alternative models for
implementing the basic package of health services in Afghanistan. Global Public Health (in press 2013).

Arur, A, Peters, D, Hansen P, Mashkoor, MA, Steinhardt, LC, Burnham, G, (2010). Contracting for health
and curative care use in Afghanistan between 2004 and 2005. Health Policy and Planning, 25(2):135-
144.
3
Since a level of variation in the BPHS was implemented among BPHS partners (USAID, EU, WB, and SM) during 2010, the
comparative analysis separates out results among the NGO implementers to compare with SM while also indicating some
averages among NGOs.

1
models on 14 BPHS key indicators. This analysis involved the adaptation of SM provincial BPHS data
into a recently-developed NGO dataset related to the primary BPHS indicators within the MoPH’s HMIS
and other secondary data. Normalized measures were developed to examine comparative effectiveness
and relative cost efficiency using these data.

Next, an analysis of quality of care, comparing contracting-in and contracting-out modalities, was
conducted. This analysis involved an examination of Balanced Scorecard (BSC) data (2010) from both
SM and NGO provinces (similar to analyses of effectiveness and relative cost-efficiency outlined above).

To complement these analyses and to address important concerns of the MoPH with regard to equity,
comparative equity analyses between the two models were conducted using the Afghanistan Mortality
Survey (AMS). This analysis involved a specific examination of key BPHS variables according to the
economic status of patients (indicated by wealth quintiles) in both SM and NGO-supported provinces as
related to receiving BPHS services. The primary analytical approach applied was Benefit Incidence
Analysis (BIA).

Lastly, a qualitative study on stakeholder impressions regarding the relative strengths and weaknesses of
contracting-in and out of the BPHS in Afghanistan was conducted. This study involved the development
of two questionnaires that were administered to 97 respondents, including 76 key stakeholders (non-
parliamentarians) in the Afghanistan health system at central and provincial levels and 21
parliamentarians. These stakeholders are listed in Table 1. A convenience sub-sample of corresponding
NGO-supported provinces was selected for these interviews including Urozgan (EU-supported), Jawzjan
(USAID-supported), and Wardak (WB-supported).

Table 1 below indicates the list of stakeholder interviewed for the qualitative study. As there were
originally 97 stakeholders to be interviewed, the study achieved a response rate of 95.8%

Table 1. Stakeholder List for the Qualitative Study

Representative Category Number of Representatives
Central Level
MoPH Leadership 2
Director Generals (MoPH) 3
MoPH Directors 5
Ministry of Finance Representatives 1
Development Partners 3
Strengthening Mechanisms Reps 3
GCMU Staff 1
NGOs 3
Mustufiat 3
Parliamentarians 21
Central Total 45
Provincial Level
Provincial Public Health Directors (PPHDs) 6
NGO Offices 6
Health Facility Representatives 24
Community Individuals 12
Provincial Total 48
Grand Total 93

2
Secondary Data Sources
Secondary data sources used and examined in this study for either reference or detailed analysis include
MoPH HMIS, National Risk and Vulnerability Assessment (NRVA), Multiple Indicator Cluster Survey
data (MICS), AMS, BSC, National Monitoring Checklist (NMC), World Bank Aide Memoire, and other
Monitoring Reports and National Health Accounts (NHA).

The quantitative analyses outlined above were led by Dr. Aaron Blaakman and Dr. Aung Lwin, and
supported by staff at MoPH/HEFD. The qualitative study data collection was outsourced to a local data
collection firm based in Kabul. The international and MoPH/HEFD research team developed primary data
collection questionnaires and protocols for the qualitative study and submitted documentation as
necessary to the Afghanistan Internal Review Board (IRB) as standard research protocol. IRB approval
was received shortly after submission.

QUANTITATIVE RESULTS
Costs and Cost-Efficiency
Costs: Table 2 highlights the Afghanistan BPHS General Population Coverage and Annual Expenditure
Estimates for 2010 for all 34 provinces in Afghanistan. It is important to clarify that the BPHS coverage
of Kabul population at this time was 15% of the total Kabul population. As a result, the total population is
an estimate of the BPHS-covered population within Afghanistan, not the total population of the country.
USAID continues to cover the largest proportion of the BPHS-covered population (49%), while SM
covers the smallest proportion of the BPHS covered population (5%). BPHS implementers incurred
expenditures accordingly from highest to lowest: USAID ($37,869,817), EU ($24,564,432), WB
($19,780,567), SM ($5,026,838). It should also be noted that since contractual periods are different
among the three primary donors and MoPH/“strengthening mechanisms,” actual 12-month fiscal time
periods can range from over the period 2009-2012. Lastly, it is important to recognize that the EU was the
only BPHS implementer to apply the revised Afghanistan National Salary Policy to BPHS contracts in
2010. This notably increased the costs of EU relative to the other implementers. As a result, for
comparative purposes, an estimated reduction of $2.6 million USD (or approximately 10% 4) was made
from its total annual expenditures/costs for the proceeding analyses.

4
This estimate is based on the following calculation: .35 (NSP reduction) x .55 (human resource percent of total cost) x .5
(provincial application) x $USD 27,164,000

3
Unit Costs: Table 3 presents unit cost estimates of BPHS 2010 by implementer (donors and MoPH/SM),
including simple provincial averages and provincial population weighted averages for both cost per capita
and cost per BPHS visit. The simple average cost per capita 2010 for EU ($4.69) is 15% higher relative to
USAID-supported provinces ($4.07), 2% higher (relative to WB-supported provinces/$4.61), and 1%
higher (relative to SM supported provinces/$4.64). These proportions generally remain the same when
examining the BPHS population weighted 5 per capita cost per year estimates.

Cost-Efficiency: Table 4 shows “relative cost-efficiency measures” for 10 BPHS indicators relative to
cost per capita per year (see Annex A for all key BPHS variables). For comparison purposes between the
BPHS implementers, both the BPHS indicator and cost per capita measures are “normalized” (meaning
the average raw score of each measure for the implementer is divided by the national average), resulting
in a relative scale with the national average = 1 (see Annex B). Subsequently, for each implementer the
normalized BPHS indicator is then divided by the normalized cost per capita to obtain the final cost-
efficiency measure. Accordingly, results of the final cost-efficiency measure above 1 indicate higher
levels of cost, while results below 1 indicate lower costs relative to the national average. The mean for
NGOs is also computed to compare with SM. Accordingly, the implementer with higher relative cost-

5
The weighted average is similar to the arithmetic average where instead of each of the data points contributing equally to the
final average, some data points contribute more than others. In this instance, this contribution varies by the proportion of
provincial population to the total population of Afghanistan.

4
efficiency (on specific measures) is highlighted in grey. 6

Figure 1 provides a graphical representation of the above measures, with the corresponding cost-
efficiency measure by code (shown in Table 4) on the x-axis and the normalized measure on the y-axis.
Among the 10 measures, results indicate that USAID-supported provinces show greater cost-efficiency
relative to the other three implementers on most measures. Tighter distributions around the average (1) for
all implementers resulted as related to the following measures: relative efficiency of coverage percent of
visits of the population (REPV), relative efficiency of coverage percent of daily served clients of the
population (REDS), and relative efficiency of percent of possible dropouts <2 year immunization of
children <1 year old.

Figure 1. BPHS Relative Cost-Efficiency Measures by Implementer - 2010

6
The distribution of these measures for NGOs is skewed and as a result, the mean should be considered a collective
representation of the NGO implementers with caution.

5
Cost-Efficiency and Cost-Quality
Cost-Efficiency and Cost-Quality: As in Table 4, Table 5 indicates additional key efficiency measures
on a “normalized” scale with the national average = 1 for several BPHS measures from HMIS and
Balance Score Card (along with Raw BSC scores), related to implementer cost. Again, results above 1
indicate higher levels of efficiency, while results below 1 indicate lower efficiency relative to the national
average. In this table, cost-efficiency measures such as “relative cost efficiency of 2010 BSC results/cost
per capita” show that USAID (1.13) contracting-out is more cost efficient than EU (0.99) and WB (0.88),
and SM (1.05). Table 5 also indicates simple averages for key relative efficiency measures for remaining
BPHS indicators from the HMIS and BSC results as related to cost for all BPHS implementers (NGO and
SM). Again, the implementer with higher relative cost-efficiency (on specific measures) is highlighted in
grey. 7

It should be noted that raw BSC results changes from 2004-2010 were close (range .33-.40) among all
implementers (NGO and SM). Figure 2 provides a graphical representation of the remaining BPHS
relative cost-efficiency/cost-quality measures by implementer 2010. USAID-supported provinces show
greater relative cost-efficiency compared to other implementers on related variables REAN2 and REPN.
The cost-quality measure REQ shows much less difference between them.

Within the context of the limitations of comparing by means, it should be noted that NGOs, on average,
appear to be more cost-efficient on 8 of 14 BPHS measures, while SM appears to be more cost-efficient
on 5 of 14 BPHS measures. The two models are equivalent on one measure (REU2), relative efficiency of
percent of possible dropouts from under 2-year-old immunization.

7
Again, the distribution of these measures for NGOs is skewed and as a result, the mean should be considered a collective
representation of the NGO implementers with caution.

6
Figure 2. BPHS Relative Cost-Efficiency/Cost-Quality Measures by Implementer - 2010

Finally, Figure 3 shows the relative BPHS measure cost-efficiency ranking counts by implementer. For
example, comparing among BPHS implementers, EU ranked second among them on 6 out of 14 relative
cost-efficiency measures during 2010. The figure highlights USAID-supported provinces as an outlier,
while cost-efficiency among the EU, WB, and SM appear to be more mixed.

Figure 3. Relative BPHS Measure Cost-Efficiency Ranking Counts by

Implementer - 2010

7
Equity
In addition to the above described cost-efficiency and quality analysis of the BPHS during 2010, an
analysis of the equitability of the delivery of the BPHS was conducted using available AMS data for
several public health and BPHS variables as related to wealth (indicated by wealth quintiles) in all 34
provinces 8. In the equity-related analysis, we explored generally if SM and NGO mechanisms were more
or less “pro-poor” in their approach, focusing on the middle to lower income quintiles with regard to
service delivery. We also examined some associated public health indicators 9. It should be noted that this
analysis is limited also in the context of not fully understanding the variation of patient needs in different
communities and among different wealth quintiles. Such data limitations are outlined in the AMS report.
Subsequently, we can only make general interpretations with regard to the observation of the data. For
example, Figure 4 indicates the average number of antenatal care (ANC) visits during pregnancy by
patients to NGO and SM BPHS services throughout Afghanistan during 2010 10. Comparing NGO and SM
implementation, both NGOs and SM appear to provide a greater amount of ANC visits to the lower two
quintiles relative to other quintiles, while SM, on average, appears to provide a greater number of visits
relative to NGOs.

Figure 4. Average number of antenatal care visits during pregnancy (NGO and SM BPHS
implementation) by wealth quintile 2010

4.50
4.00
3.50
Total number of antenatal
3.00
visits during pregnancy NGO
2.50 Total number of antenatal
visits during pregnancy SM
2.00
1.50
1.00
0.50
0.00
Lowest 2 3 4 Highest
quintile quintile

8
The overall equity analysis conducted during January 2013 had limitations due to data constraints of the dataset at the time.
More recent, updated data, cleaned during May 2013, were not analyzed for this report and as a result, may show different
findings.
9
Summary statistics such as the Concentration Index (CI), a measure of understanding the magnitude of difference among wealth
quintiles, are not reported here due to data limitations within the available dataset at the time of this analysis.
10
At the time of this analysis, data related to the number of antennal care visits during pregnancy required adjustment due to
missing data and some data entry errors in the version of the AMS dataset that was provided to the research team. Adjustments
were made by the research team based on a relative assessment of the BPHS protocol for the number of antenatal care visits
conducted during pregnancy.

8
Figure 5 shows the percentage of utilization of a health facility for ANC NGOs and SM within type of
care. NGOs appear to be slightly more pro-poor as related to delivery to the lowest income quintile at
CHCs/Public Polyclinics, while both NGOs and SM appear to be pro-poor (as related to the lowest two
income quintiles) at the BHC level. Furthermore, both appear to be pro-poor at the health post level.

Figure 5. Percentage of Utilization of Health Facility for ANC by BPHS Modality (NGO/SM) and
Type of Care

Figure 6 highlights infant mortality by wealth quintile within NGO and SM-supported BPHS provinces
for 2010. In this case, we note that these public health indicators are clustered among quintiles for NGOs
while SM indicators show greater variation. It is important to highlight that stillbirth is most pronounced
within the lowest quintile of SM. Furthermore, the middle quintile experienced higher rates of neonatal
death and infantile death relative to other quintiles in NGO implemented BPHS provinces.

9
Figure 6. Infant Mortality % by Wealth Quintile within NGO and SM-supported BPHS Provinces

QUALITATIVE RESULTS
Stakeholder Survey: Central and Provincial Level (non-
parliamentarian)
To complement the quantitative comparative analysis between SM and NGO BPHS implementers, a
qualitative study on stakeholder impressions regarding the relative strengths and weaknesses of
contracting-in and out of the BPHS in Afghanistan was conducted. A questionnaire was administered to
76 key stakeholders in the Afghanistan health system, at both central and provincial levels, to examine the
following aspects of BPHS implementation by NGOs and SM:

1. Applying evidence for decision-making in BPHS implementation

2. Political influence in NGO and SM BPHS implementation
3. Application of continuous medical education and training for staff
4. Linkages between BPHS and EPHS
5. Resource allocation and management
6. Staffing Capacity
7. Qualification and referral rates of Community Health Workers
8. Pharmaceutical supplies availability

10
The following provides a summary of findings from a survey of central and provincial level stakeholders
(non-parliamentarians) as related to impressions of BPHS implementation under both SM and NGO
mechanisms.

Component 1: Applying evidence for decision-making in BPHS implementation

Stakeholders generally noted that every province should have a resource center for documenting evidence
for action or decision-making and encouragement should also be given for using data locally in making
decisions for the health sector. Overall, stakeholders indicate that client satisfaction should be evaluated,
that there are blockages in the government reporting system and that BSC indicators have their own
limitations and are not fully accepted.

Comparing SM and NGOs on this measure, stakeholders highlight that SM has more paperwork, and as a
result, this mechanism has more reliable accountability and transparency. SM has health councils for
providing informed evidence, while NGOs use new and more effective methods for reporting. A few
stakeholders also indicated that NGOs are not as transparent in their reporting as before. Alternatively,
some stakeholders found that the independent nature of NGOs allows them to report more independently
and transparently. Overall, stakeholders gave the impression that NGOs are more transparent relative to
SM.

Component 2: Influence of the Political Process on SM and NGO BPHS implementation

Stakeholders noted that political influence is an overall issue throughout the country. In the health sector,
stakeholders indicate that such pressures influence a health facility and its implementation, but primarily,
high-level persons influence the system as well as parties, commanders, directors of government
departments, provincial representatives, provincial Shura members, and provincial health directors.
Importantly, it was noted that political pressures can override the standardized criteria in activation of a
health facility and that those with political influence propose that clinics be built in certain locations. It
was noted that these authorities may also influence hiring practices.

Respondents also indicated that MoPH should define clear standards and strive to be more independent of
the political process. Comparing NGOs and SM, stakeholders suggested that there is less political
influence in NGO hiring of staff compared to SM. Furthermore, respondents noted that NGOs can reduce
political influence as it is part of civil society and that such a mechanism deals with lower level
authorities. Alternatively, others believe that SM has government power to minimize the problem.

Component 3: Application of Continuous Medical Education (CME) and Training for Staff

With regard to continuous medical education and training for staff, respondents indicated that different
providers operate separate packages making it difficult for continuous medical education and training to
be standardized.

Other elements noted for this component include:

• High-level staff have no incentive to visit lower levels.
• CME is less respected in SM compared to NGOs.
• SM's primary problem is that its focus is more on the documentation process than technical
implementation.
• SM government offices are slow in learning and coordinating.
• SM has procurement problems impacting CME.
• NGOs are more efficient for facilitating the communication between high-level staff with low-
level staff.

11
Component 4: Linkages between BPHS and EPHS

Stakeholders noted that if different “BPHS and EPHS packages” are offered by different providers,
linkage and coordination is more difficult in the system overall and that it is recommended that one level
of leadership should be conducted for both the BPHS and the EPHS. At present, coordination from top to
bottom is very weak and requires strengthening. For example, TB cases referred to lower level not
followed by Community Health Workers (CHWs) properly. Alternatively, it was noted that linkages from
bottom to up is good, particularly from Comprehensive Health Centers (CHCs) to upper levels. On the
other hand, it was noted that no proper system of feedback and follow-ups is available. For example, there
is no feedback for referred patients. Some stakeholders noted that complicated cases are not referred up in
the system. Lastly, for NGOs, it was highlighted that USAID-supported provinces provide better linkages
than EU-supported and WB-supported provinces.

Component 5: Resource Allocation and Management

Respondents highlighted some comparative differences between SM and NGO mechanisms on resource
allocation and management including the following:
• SM has recurring problems with salaries and payments.
• SM has recruitment problems.
• SM has problems with consistent staff attendance.
• NGO has additional financial resources compared with SM.
• NGO hires staff from neighbors around the NGO.
• Staff in both models is not skilled-enough.
• Many specialty services in both models are considered “weak.”

Component 6: Staffing Capacity

The following summary points where indicated with regard to BPHS staffing capacity at different levels
in the system:

Comprehensive Health Centers

• Two more staff members are necessary for adequate staffing.
• Two nurses should be recruited: one nurse should be attached to BHC.
• Female staff has challenges working during night shifts.
• Maternal care for staff should be extended.
• CHC is underutilized.
• CHC is overstaffed.
• One dentist necessary at this level.
• SM is better for coverage relative to NGOs.

Basic Health Centers

• Female staff is necessary to address women’s issues.
• Male nurses are preferred for night shifts and security concerns.
• One more nurse is necessary than is currently allocated.
• One pharmacist is necessary.
• One additional lab technician is necessary.
• BHCs should have a doctor on staff.
• In order to improve quality in both SM and NGO models it is necessary to in salary and support
necessary

12
Component 7: Qualification and Referral Rates of Community Health Workers

With regard to Component 7, several important items were to note from respondents, including:
• Overall, unqualified staff is often recruited for both SM and NGO mechanisms. (More training is
necessary.)
• Illiteracy is an issue among staff, including CHWs.
• Incentives for CHWs are not enough.
• Under SM, recruitment of CHWs is a long, enduring process.
• NGOs are better for human resource hiring and faster implementation.

Component 8: Pharmaceutical Supplies Availability

With regard to pharmaceutical supply availability, in summary, several important items were to note from
respondents including the following:
• SM has a much longer procurement process for pharmaceuticals compared with NGOs.
• CHWs' knowledge is not sufficient so that provision should be modest
• There are also often stock-outs of medicines, particularly under SM.

Parliamentarian Survey Results (n=21)

Summary Descriptive Statistics

In addition to the above qualitative assessment of central and provincial level staff regarding BPHS
implementation issues and SM and NGO mechanisms, a parliamentary survey (n=21) was administered to
compare impressions of SM and NGO BPHS implementation at this level. The survey inquired the extent
to which parliamentarians agreed with a series of statements related to BPHS implementation under SM
and NGO mechanisms (using a Likert scale of 1-4, where 4=strongly agreed, 3=agreed, 2=disagreed, or
1=strongly disagreed). Table 6 highlights the relevant descriptive statistics associated with the results of
the analysis of this survey. Overall there were 21 respondents, with 8 parliamentarians coming from SM-
related provinces, and 13 coming from NGO-supported provinces. Average/mean scores (along with
standard deviation and the mean standard error) were established for each mechanism (SM/NGO)
associated with each question.

SM representatives most strongly agreed with the statements “People’s access to health services has
significantly increased in my province in recent years (mean=3.75)”, and “the Ministry of Public Health
has been successful in controlling the private hospitals in my province (mean=3.75)”. NGO
representatives most strongly agreed with the statements “the Ministry of Public Health has been
successful in managing the private hospitals in my province (mean=3.46),” and “the Ministry of Public
Health promptly responds to requests and health needs of my province (mean=3.62).”

Alternatively, SM representatives most strongly disagreed with the statements “the Ministry of public
health should deliver health services directly by itself (mean=1.62),” and “the senior officials of the
provincial public health directorate are very competent in my province (mean=1.88).” NGO
representatives most strongly disagreed with the following statements “the Ministry of public health
should deliver health services directly by itself (mean=1.54),” “the senior officials of the provincial public
health directorate are very competent in my province (mean=2.00)” and the statement” the Ministry of
public health should continue delivering services though contracts with NGOs (mean=2.00)”.

13
Analysis of Difference of Means (SM and NGO)

An independent samples t-test was conducted to examine differences in means between the responses of
Parliamentarians from representative NGOs and SM provinces. Results indicate that none of the means of
17 likert scale measures were found to be statistically significant at the p<.05 or p<.10 levels.
Accordingly, we conclude that either the sample size is too small to identify any differences or there are
no differences in responses related to the two BPHS implementation mechanisms. The full table of results
can be found in Annex C of this report.

14
15
STRENGTHS AND LIMITATIONS
Although several important issues related to BPHS implementation were addressed in this study, it is
important to highlight both the strengths and weaknesses of the methodology to provide context for
discussion and recommendations.

With regard to strengths, the investigators of the study emphasize that the data for the quantitative
analyses come from all 34 provinces of Afghanistan. Prior cost-efficiency analyses were conducted in
only six provinces. Furthermore, the study builds upon previous work conducted by the investigators in a
comparative study of BPHS contracting-out implementers funded by the European Union in 2011. Lastly,
this study involves detailed verification of SM cost data from three sources of data including interviews
with the SM department within the MoPH, extracted SM invoice data, and provincial data.

The study also has limitations. First, although the study examines important ratios related to cost-
efficiency and cost-quality, explanatory variables are missing, largely due to a lack of data availability.
For example, this study could be advanced if the researchers investigated how differences in security,
remoteness, type of facility, and other geographical and social variables impact relative cost-efficiency,
cost-quality, and equity indicators. Furthermore, the equity analysis is limited due to data limitation
factors previously highlighted in this report. Additional data elements would be required to conduct a
more detailed equity assessment at the provincial level.

With regard to the qualitative study, the samples sizes are adequate for understanding collective
impressions of stakeholders of BPHS implementation, but the sample size of the parliamentary survey is
small and, as a result, may not fully reflect any existing differences in means.

DISCUSSION AND RECOMMENDATIONS

Applying key economic variables for decision-making and planning the next steps in the Afghan health
sector is critical for moving Afghanistan beyond an emergency phase and achieving a sustainable health
system. Furthermore, economic variables can be used for strengthening health system functioning overall.

This study involved an assessment of cost, cost-efficiency, cost-quality, and equity of BPHS services
under both SM and NGOs contracting mechanisms in Afghanistan using both quantitative and qualitative
research methods. From each of these perspectives, although some differences are highlighted and lessons
are learned, there appears to be no clear collective difference between SM and NGOs on the
implementation of BPHS. 11 The cost-efficiency analysis highlights that, among the NGO implementers,
USAID-supported provinces appear to be most cost-efficient while the other three implementers (EU,
WB, and SM) show mixed results depending on the BPHS related indicator. However, it should be noted
that several factors might contribute to cost-efficiency, including security, remoteness, type of contacting
mechanism, auditing procedures, level of monitoring and evaluation, strength of management and
administration, etc. These factors are not examined in this study because of lack of data and resource
constraints, but could be modeled and examined in future analyses.

With regard to cost-quality, the primary cost-quality measure “relative efficiency of 2010 BSC results
shows that USAID (1.13) reflected slightly higher quality relative to cost than the other three
implementers, EU (0.99) and WB (0.88), and SM (1.05). These data show no significant difference
between NGO and SM BPHS implementation on relative cost and quality at this point in time. Also,

11
This conclusion most likely would change if comparing USAID alone with SM given the relative cost-effectiveness of USAID.

16
quality measures are limited and additional data would strengthen an understanding of the relationship
between quality and cost.

Equity measures indicate mixed results between NGO and SM mechanisms with regard to implementing
a “pro-poor strategy” within BPHS. There are also limitations with regard to more complete equity
analyses including the a skewed sample towards NGO-supported provinces, a lack of knowledge or data
about community needs and limited understanding if NGOs/SM address the variation of health needs
across the country.

Qualitative analyses highlight important issues related to the implementation of BPHS and the two
models but also show not significant difference between the implementation modalities (NGOs and SM).
Stakeholders indicated that both models face constraints in achieving full BPHS implementation
including political pressure, system referral and linkage problems, as well as administrative barriers. As
previously noted, with regard to the independent samples t-test of parliamentary data, we conclude that
either the sample size is too small to identify any differences or that there are no differences in responses
related to the two BPHS implementation mechanisms.

As the MoPH and development partners move forward with the BPHS, “best practices” should be further
examined and applied to overall implementation. For example, as highlighted in this study, USAID shows
strength on relative cost-efficiency on numerous indicators. Components of USAID-supported BPHS
should be further examined, along with strengths of the other implementers (EU, WB, and SM).
Weaknesses of BPHS highlighted by stakeholders in the qualitative study should also be addressed as
necessary from policy and management levels.

Lastly, decision-making with regard to future BPHS implementation modalities can be further informed
with public health and economic evidence, but the way forward is also a political, administrative, and
social process. Taking into consideration these perspectives can also aid the government and development
partners in improving and strengthening the value for investment in the BPHS in Afghanistan in the
future.

17
ANNEX A: KEY BPHS VARIABLES LIST

Codes and titles of key BPHS variables associated with BPHS implementers:

1. REPV - Percent of visits of catchment population

2. REID - Percent of institutional deliveries of catchment population
3. RELD - Percent of live deliveries of the catchment population
4. REDC - Cost per daily served client per year ($)
5. REDS - Percent of daily served clients of the catchment population
6. REU1 - Percent of immunized Children under 1 year old of the
catchment population
7. REU1A - Percent of under 1 year old children immunized of all visits
8. REPENTA - Percent of children under 2 years old receiving
PENTA3
9. REU2 - Percent of possible drop-outs in the under 2 year population
of those having received under 1 year immunization
10. REAN - Percent of patients receiving the first ANC of the catchment
population
11. REAN2- Percent of patients receiving additional ANC of the
catchment population
12. UNCODED - Percent having received the first PNC of catchment
population
13. UNCODED - Percent of possible drop outs of PNC of the first ANC
patients
14. UNCODED - Raw BSC composite score 2004
15. UNCODED - Raw BSC composite score 2010
16. UNCODED - Raw BSC change, 2004-2010
17. UNCODED - Relative BSC score, 2010
18. REQ - Relative BSC Score 2010/per capita cost

18
ANNEX B: EXPLANATION ABOUT RELATIVE PER CAPITA COST
AND RELATIVE EFFICIENCY VALUES
We measured cost-efficiency by calculating two kinds of parameters: relative per capita cost and relative
efficiencies. The relative per capita costs is set as the only variable to serve as the denominator while
efficiency indicators of various kinds serve as numerators. The term “relative” indicates the process of
standardization of measurement among different provinces by calculating how much is the value of a
parameter in one individual province is relative to the overall mean of that parameter in all 34 provinces.

Step 1: Derivation of the denominator (cost parameter)

individual province’s cost per capita
Relative cost per capita =
mean cost per capita of all provinces

Step 2: Derivation of a numerator (efficiency parameter)

individual province’s BPHS indicatormean BPHS indicator of all provinces
Relative efficiency =

Step 3: Derivation of cost-efficiency of the province (cost-efficiency result)

Rel. BPHS indicator
Rel. efficiency of BPHS indicator =
Rel. cost per capita

As seen in the formula in Step 1, we divide the individual province’s cost per capita cost by mean cost per
capital of all provinces. This implies that if the value of relative per capita cost is above 1, this indicates
the province has a higher cost than the mean per capita cost of all provinces. Similarly, if the value of
relative per capita cost is below 1, this indicates the province has a lower cost than the mean per capita
cost of all provinces. In principle, a comparatively higher value in relative per capita cost in its position of
the denominator indicates the province is more likely to demonstrate a lower efficiency (cost-efficiency)
result.

If the efficiency parameter that occupies the numerator position is a “normalized indicator”, the higher
value over there will be more likely to demonstrate a higher efficiency (cost-efficiency result).

19
ANNEX C: RESULTS FROM INDEPENDENT SAMPLES TEST OF
DIFFERENCES BETWEEN SM AND NGOS (PARLIAMENTARIAN
SURVEY)

20
 For more information, contact:

Health Economics and Financing Directorate

Ministry of Public Health
Kabul, Afghanistan

or

Health Policy Project

Futures Group
One Thomas Circle, NW, Suite 200
Washington, DC 20005
Tel: (202) 775-9680
Fax: (202) 775-9694
Email: [email protected]
www.healthpolicyproject.com