Aggregating Governance With The Right Method
Aggregating Governance With The Right Method
Aggregating Governance With The Right Method
Pablo Zoido-Lobat6n
components.
Public Disclosure Authorized
Summaryfindings
In recent yearsthe growinginterest of academicsand governancethan any singleindicator,the standarderrors
policymakersin governancehas been reflectedin the associatedwith estimatesof governanceare still large
proliferationof cross-countryindicesmeasuringvarious relativeto the units in whichgovernanceis measured.In
aspectsof governance. light of these marginsof error, it is misleadingto offer
Kaufmann,Kraay, and Zoido-Lobat6nexplain how a very preciserankingsof countriesaccordingto their level
simplevariant of an unobservedcomponentsmodel can of governance:smalldifferencesin country rankingsare
be used to combinethe informationfrom these different unlikelyto be statistically- let alone practically-
sourcesinto aggregategovernanceindicators.The main significant.Nevertheless,these aggregategovernance
advantageof this method is that it allowsquantification indicatorsare usefulbecausethey allow countriesto be
of the precisionof both individualsourcesof governance sorted into broad groupingsaccordingto levelsof
data and country-specificaggregategovernance governance,and they can be used to studythe causesand
indicators. consequencesof governancein a much larger sampleof
Kaufmann,Kraay,and Zoido-Lobat6nillustratethe countriesthan previouslyused (seefor examplethe
methodologyby constructingaggregateindicatorsof companionpaper by Kaufmann,Kraay,and Zoido-
bureaucraticquality,rule of law, and graft for a sample Lobat6n, "GovernanceMatters,"PolicyResearch
of 160 countries.Althoughthese aggregategovernance WorkingPaper 2196).
indicatorsare more informativeabout the levelof
This paper - a joint product of Macroeconomicsand Growth, Development Research Group; and Governance,
Regulation,andFinance,World BankInstitute-is part of a largereffort in the Bankto studythe causesandconsequences
of governancefor development.Copies of the paper are availablefree from the World Bank, 1818 H Street NW,
Washington, DC 20433. Please contact Diane Bouvet, room G2-136, telephone 202-473-5818,fax 202-334-8350,
Internet address [email protected] ResearchWorking Papers are also posted on the Web at http://
www.worldbank.org/html/dec/Publications/Workpapers/home.html. The authors may be contacted at
[email protected], [email protected],or [email protected]. October 1999. (39 pages)
DanielKaufmann
Aart Kraay
FlabloZoido-Lobat6n
TheWorld Bank
1
of this conditionaldistributionprovidesa naturalestimateof the precisionof this
aggregategovernancemeasurefor eachcountry.
2
consequencesof relaxingthe secondassumptionby proposinga methodwhichsimply
aggregatesthe ordinalinformationacrossindicators.Althoughthis has the advantageof
simplicityand does not requireassumptionsof linearity,it is also much less precisethan
the unobservedcomponentsmethodsince it discardsthe cardinalinformationin the
data. The third assumptionof a normaldistributionfor unobservedgovernanceimplies
that our estimatesof governancewill be clusteredaroundthe mean of this distribution.
This raisesthe possibilitythat the!difficultyin distinguishingbetweencountriesis in part
drivenby this assumption.We thereforeexplorethe robustnessof our results by
consideringaltemativesto this assumption,andfind that our conclusionsare materially
unaffectedby our assumptionson the shapeof the distributionof unobserved
govemance.
2 Inourcompanion
paperwe explorethisideain moredetail,usingcross-country
regressions
of percapita
incomeonvariousgovernance measuresinstrumenting forgovemanceusingmeasures
of thelinguistic'
composition
of thepopulation
- in thespiritof HallandJones(1999).
3
illustrateour ideas. In Section3, we lay out and implementthe unobservedcomponents
frameworkfor estimatinggovernance,and presentthe main resultsin Section4. In
Section5, we discussthe consequencesof relaxingseveralof the assumptions
underlyingthe model.We concludewith a discussionof the implicationsof our findings
for researchand policyadviceregardinggovernance.
4
2. Indicatorsof Governance
5
indicatorshas implicationsfor howwe interpretthe errorterms in the relationship
betweenobservedindicatorscoresand the underlyingconceptsof governance.
6
for mostsurveysof citizensor entrepreneurswe havethe mean responseacross
respondentsof discretecategoricalscores. We re-orientdatafrom each sourceso that
highervaluescorrespondto betteroutcomes(i.e. strongerrule of law, moreeffective
govemment,and less graft). In addition,we rescaleeach indicatorby subtractingthe
minimumpossiblescore and dividingbythe differencebetweenthe maximumand
minimumscores,sothat each indicatoris on a possiblescalefrom zero to one. Since
we rescaleeachindicatorusingthemaximumand minimumpossiblescores(ratherthan
the maximumand minimumactualscoresin the sampleof countriescoveredby each
indicator),this is nothingmorethan a convenientchoiceof units.
7
3. EstimatingGovernance
The Model
8
to vary acrossindicatorsk, the fact that P(k)multipliesthe error term is an innocuous
rescalingwhichslightlysimplifiessomeof the expressionswhichfollow.
4 Toseethis,consider
thespecialcase!
wherea(k)=Oand13(k)=1
forallindicators.Wecanmakethe
likelihood
functionoftheobserved dataarbitrarily
largesimplyby estimating
go)as theobserved scoreona
particularindicator,
forexample ga)=y(j.K),
foreverycountryj, andsettingcr(K)=0.Kiefer(1980)provides
a
detaileddiscussionof thispoint.
9
"improperpractices")is perfectlymeasured,it mayneverthelessbe a noisy indicatorof
graft if there are differencesacrosscountriesin what "improperpractices"are
consideredto be.
10
particularaspectof governanceis betterthan in the worldas a whole (e.g. BERI,which
coversprimarilydevelopedcountries). Supposefurtherthat this sourcetends to score
countriesrelativeto each other,so that the worst(best)countryin the samplereceives
the lowest(highest)possiblescore?
of zero (one). Thiswould be reflectedin a relatively
highvalueof ,8(k),since relativelysmalldifferencesin true govemanceare magnified
into relativelylargedifferencesin observedscores.
11
These expressionshavea very naturalinterpretation.If the parametersa(k),
2 were known,a sensibleway to estimatego) would be to rescalethe
,(k) and acn(k)
observedscoresby subtractinga(k) and dividingby ,B(k),and then constructa weighted
( K(j)
V[g(j)ly(j)J = (1+ (k)- 2 ), whichis decreasingin the numberof indicators
k=1
availablefor that country,K(), and is increasingin the varianceof the errorterm in each
of these indicators,ue(k)2.
Estimatingthe UnknownParameters
12
indicators.To see why,
We cannotapplythis methodto non-representative
considerthe maximum-likelihood
estimateof a(k), which unsurprisinglyis the mean
score acrosscountrieson indicatork. It is straightforwardto see from Equation(1) that
the expectedvalue of the sampleimeanof scoreson indicatork is a(k) + ,B(k)* g(k),
where g(k) denotesthe averagelevelof governancein the sampleof countriescovered
by indicatork. Forrepresentativeindicators,our choiceof units for governance
normalizesg(k) = 0. However,for a non-representative
indicatorwherethe average
level of govemanceis differentfrom the worldas a whole, g(k) • 0 and the sample
meandoesnot providea consistentestimateof a(k).
13
It is worth notingthat this estimationmethodurewardsconformity",in the sensethat
indicatorsthat are highlycorrelatedwill havelow estimatedvariancesand hencewill be
perceivedas more precise. Given our assumptionthat the disturbancetermsare
independentacrossindicators,it makessenseto treat highlycorrelatedindicatorsin this
way. If on the other handindicatorsare correlatedsimplybecausetheir disturbances
are correlated,this interpretationwouldbe inappropriate.We take this issueup in more
detailin Section4, and arguethat it will result in even less preciseestimatesof
govemancethan thosewe obtainhere.
14
4. Results
Estimates of Governance
15
these confidenceintervalsdo not reflectthe samplingvariationin the point estimatesof
the unknownparametersa(k), ,8(k)anda.(k). If this uncertaintywere also taken into
account,the standarderrorswould be evenlarger.
AssessingCross-CountryDifferencesin Governance
16
quartilesthe situationis muchless clear,as veryfew countries'90% confidenceintervals
lie entirelywithin a givenquartile,for each of the three aspectsof govemance.
8 Sincethe ga)ly() and g(')ly(') are jointly normaland independentby assumption,this calculationinvolves
a straighfforwardintegrationof the areaunder a bivariatenormalprobabilitydensityfunction.
17
as dark pointson the verticalaxis. We also repeatthe exercise,but insteadreportthe
largerproportionof countriesfor whichthis probabilityis between25% and 75%,which
correspondsto a test at the 50%significancelevel. This proportionis shownas a light
dot in Figure2.
Intuitions
18
for each of the surveysk=1,2,:3.9Insertingthis into Equation(3),the varianceof the
aggregategovernanceindicatorbasedon this hypotheticaldatawill be the samefor
19
4. Extensions
CorrelatedDisturbances
10SeeKaufmanand Zoido-Lobat6n(1999).
20
Althoughit is not possibleto statisticallyidentifythe correlationof the
disturbancesacrosssources,it is straighfforwardto see the consequencesof positively-
correlatederrorsfor our results.If the errorsare correlatedacross indicators,each
additionalindicatorcontributesless informationto our estimateof governance.Thiswill
be reflectedin the varianceof the conditionaldistributionof governancein eachcountry.
In particular,it is straightforwaird
to showthat holdingconstantthe varianceof the
residualson each indicator,the varianceof the conditional distributionof govemanceis
11
increasingin the correlationbetweenthe errorson any two indicators.
21
Non-Linearities
13
The straighfforward solutionto this problemwouldbe to rely on an orderedmultinomialchoicemodelwith
individualeffectsin the latentvariable. HoWever,for sucha modelto be identified,it wouldagainbe
necessaryto assumethat the variancesof the errorsare identicalacrossindicators.
14This problemmaybe particularlyacutefor polls of expertswho considera largeset of countriesat once,
since in contrastto surveysof residents,expertsare muchmorelikelyto be awareof relativecomparisons
of countries.
22
and 3/6 to Mexicofor graft,while on a much less representativesurveysuch as BERI
whichcoversprimarilydevelopedcountries,Mexicoreceivesa rather poor ratingof 1/7.
23
indicators,this hypothesiscan be testedusingthe data on the proportionof indicatorsin
which countryj is rankedhigherthan countryj' in a simplebinomialproportionstest.
24
possiblethat it is easierto identifystatisticallysignificantdifferencesin governance
acrosscountries.
17
We computethese as follows. First,usinga methodof momentsargumentwe constructestimatesof the
parametersof the modelcorresponding to the assumeddistributionof govemance(notethat the meanand
varianceof this beta distributionchangeas we varythe parameters).We then constructthejoint distribution
of go)andy() as theappropriate
mixtureof a normalanda betadistribution,
andthenobtainthemarginal
distributionof yj) and conditionaldistributionof go) givenyG)by appropriatenumericalintegrationsof this
joint distribution.We numericallyevaluatethe meanof this distribution,and the 5e and 95 percentiles,and
reportthesefor eachcountryin Figure5.
25
number of countries with 90% confidence intervals falling entirely within particular
quartiles is essentially unchanged for each quartile, as we vary our assumptions on the
shape of the distribution of govemance.
26
5. Conclusions
27
In the longterm, however,our resultsalsopoint to the inadequacyof existing
governancemeasures.jt is very unsatisfyingthat existingdata,evenwith favourable
assumptions,allows us to identifyrelativelyfew statisticallysignificantdifferencesin
governanceacrosscountries. Moreover,existingdata providesat besttenuouslinks
betweenperceptionsof govemanceand objectivepolicyinterventionsthat govemments
interestedin improvingthe qualityof governancecan undertake.There is thereforea
needto improvethe qualityand quantityof govemancedata,both by improvingand
extendingcross-countrysurveywork of govemanceperceptions,as well as employing
1 8 Manyof the polls and surveyswe
country-specificin-depthgovemancediagnostics.
use sufferfrom deficiencies,such as poorly-wordedquestionsabout ill-definedand
excessivelybroadconcepts. There is roomto improvethese instrumentsby asking
respondentsabouttheir directexperienceswith well-definedeventsand using
transparentunitsto measuregovemance. However,these are time- and resource-
intensiveexercises,and intemationally-comparable
high-qualitydata of this sort is years
away.
28
References
29
Table 1: GovernanceIndicators
Code 6e Coe
(Sura C eptls
cir (P)
Pnli 12M-t
DRN Standard and P 106 developed 0.23 Governmentineffectiveness, Enforceability contracts, Comuptionamong pubic
Poor's DRI and developing instiutional hfaihe costs of crime officials, offectivenessof
countries antkorruption iitiativs
EIU Economist P 115developed 0.19 Insitutional efficacy, red tape Cme, corruption in Corruptionemong public
Intelligence Unit and developing bakng sector of4ciats
counties
FHNT Freedom House P 28 transiion 0.82 Quality of goverment and Rule of law Perep,tions of corrlp,tiOn
economies public administration in civil service, business
interests of polkymakers
GCS Gobal S 59 develped 0.42 Competence of pubic Cdizens can file awsuts Frequency of "kregular
Competitiveness and developing sector, polrtial pressures on against government payments" to officials and
Survey countries civi servants, time spent citizns acoept legal judiiary
with tbrreauciats e4udicln,
independence of judiciary,
costs of crime
GCSA Global S 23 African 0.73 Competence Ofpblic Citiwenscan file lawsuits Frequency of "iregular
Competitiveness countries servants, commitment to against goversnent, paymentsWto offiials and
Stivey, Afria polices of previous cdizens acceot legal judiciary
govefmments edudication,
independence of judiciary,
costs of mime
PRS Poitical Risk P 131 developed 0.10 Bureatic qulaity, policy Rule of law Corruptionin the political
Services and developing stability system sa "threat to
countries foreign investment"
WCY World S 46 primarily 0.59 Efflcien implementation of Tax evasion, confidence "Improperpratices in
Competitiveness devetoped goennrert decisions in atblity o ato ties to the publi sptere
Yearbook countries political pressures on civil protect property,
servants confidence in
admrinistrationof justice
WDR World S 74 developed 0.25 Efficiency of govemment in Unpredictabilityof the Corruption as "obstacle to
Deveopment and develoing delivenrig services, judiciary, ttheft end crime, busine, frequency of
Report countries predictabiliy of rules, time ability of state to protect "additiol payments" to
spent wih bureaucrats private propeity "get things done"
30
Table 2: CorrelationsAmongGovernanceIndicators
GovernmentEffectiveness
gee,u gedn goprs gewdr geberi geefint gegcs gogcs gswcy
gaeiu 1.00
114
gadri 0.77' 1.00
96 106
geprs o.60- 0.61 1.00
111 100 140
gewdr 0.78 * 0.68 0.36 1.00
58 57 65 74
gsberi 0.74 * 0.71 * 0.54 * 0.73 1.00
49 so 50 30 so
gent 066 * 0.71 024 0.62 - 0.67 1.00
19 24 21 20 6 28
gegcs 0.76 0.74 0.52 * 0.85 0.70 0.75 * 1.00
64 62 72 45 46 6 75
gagcsa 0.64 0.69 0.55 0.26 1.00 0.56 1.00
1 15 20 - 14 2 0 19 23
gewcy 0.55 0.53 0.48' 0.81 0.65 0.06 0.92 1.00
43 44 46 27 41 4 46 0 46
Ruleof Law
ddi diu rdMf rIprs rdwdr dberi dceer Mint dges dgsa ljz dwqcy iscore
rtdi 100
106
dwiu 0.73* 1.O
96 114
Mhf 0.73 0.686 1.00
105 112 16O
rtpus 0.75 0.75* 0.62 1.00
1oo' 111 137 140
dwdr 0.58 0.75 0.76 0.59s 1.00
57 58 72 es 74
rberi 0.73 0.70* 0.82* 0.62' 0.54 1.00
so 49 50 50 30 50
de 0.69 * 0.79 * 0.90 * 0.52 * 0.51 - .0.86 1.00
24 19 25 20 20 6 27
dfhrt 0.76' 0.66' 0.86 0.33 0.394 -0.75' 0.91 * 1.00
24 19 26 21 20 6 27 28
dg3 0.70* 0.768 0.78* 0.70 0.74* 0.77* 0.96 0.78 1.00
53 se 59 59 33 46 6 6 59
dugsa 0.43 031 0.03 0.27 0.61 * -1.00 0.87 1.00
15 18 23 20 14 2 0 0 3 23
rjlkz 0.63 - 0.45 0.37 0.57 0.13 0.64 0.19 0.00 0.50 0.16 1.00
6s 73 76 74 48 44 19 20' 49 8 77
dwCy 0.63 0.73 0.77' 0.68' 0.786 0.75' 0.98' 0.94' 0.94 0.47' 1.00
44 43 46 46 27 41 4 4 46 0 40 46
drscor 1.00 0.99 0.95 0.91 0.72 0.63 0.99 0.986 0.986 . 0.37 0.96' 1.00
5 5 S 5 5 5 5 5 5 0 5 4 5
Graft
gWrd grei grpn gWwdr guSeti grow grit gupilup grgrs grgca gWpercgnucy grscore
grdri 1.00
106
greiu 0.80 100
97 115
guprs 0.65 0.64 1.00
100 112 140
grwdr 0.69e 0.80* 0.46 100
57 5s 65 74
gubiei 0.57 0.58 0.48 0.78 1.00
50 50 50 30 50
gnroer 0.91 0.76 0.68 0.56' 0.38 100
24 19 20 20 6 26
grntd 0.79 o0.60 0.72n 0.58 0.40 0.92' 100
23 19 21 20 6 25 28
grgalkp 0.63 0.78- 0.62 0.81 0.46 - 0.71 0.69 1.00
42 42 44 26 29 9 9 44
grgcs 0.77' 0.88' 0.57 0.87 0.73' 0.98 0.80 * 0.60 1.0o
53 57 59 33 46 6 7 35 59
grgcsa 0.59 0.53 * 0.45 0.61' 1.00 . . 0.9 0.79 1.00
15 18 20 14 2 0 0 3 3 23
grperc 0.58 0.95' 0.33 0.96' 0.76' . . 0.84' 0.89 * 1.00
12 12 12 6 11 0 0 5 12 0 12
gr*cy 0.69 0.-84 0.65' 0.93 0.62 -0.32 -0.51 0.67' 0.856 . 0.93 100
44 44 46 27 41 4 4 31 46 0 11 46
grscere 0.91 0.92' 0.82' 0.869 0.74' 0.89' 0.67' 0.73 0.82 0.64 0.82- 0.82' 1.0o
51 51 51 51 30 13 13 25 33 11 6 27 51
Thistable reportspairwisecorrelationsbetweengovemanceindicatorswithineach
govemancecluster. The numbersbelowthe correlationcoefficientsindicatethe
numberof countriescommonto eachpair of indicators.* indicatessignificanceat the
90% level.
31
Table 3: ParameterEstimates
RepresentativeIndicators
DRI 0.539 0.239 0.588 0.668 0.179 0.583 0.539 0.221 0.618
EIU 0.432 0.226 0.396 0.442 0.285 0.445 0.312 0.309 0.322
HF 0.466 0.247 0.751
PRS 0.789 0.109 1.222 0.606 0.220 0.656 0.506 0.142 1.129
WDR 0.473 0.097 0.685 0.354 0.135 0.681 0.465 0.150 0.636
Non-RepresentativeIndicators
BERI 0.406 0.133 0.707 0.404 0.136 0.687 0.483 0.173 1.226
CEER 0.604 0.380 0.303 0.615 0.359 0.356
FHNT 0.624 0.396 0.440 0.566 0.397 0.454 0.561 0.513 0.511
GALLUP 0.470 0.149 0.709
GCS 0.398 0.151 0.600 0.526 0.148 0.594 0.493 0.247 0.457
GCSA 0.470 0.112 0.450 0.499 0.041 1.695 0.498 0.297 0.406
PERC 0.302 0.299 0.270
WCY 0.307 0.120 0.906 0.382 0.156 0.580 0.212 0.284 0.498
32
Table?
4: AssigningCountriesto Quartiles
GovernmentEffectiveness
First Quairtile 0.31 0.54 0.72
SecondQuartile 0.00 0.00 0.26
Third Quartile 0.00 0.13 0.31
FourthQuartile 0.59 0.69 0.79
Ruleof Law
First Quartile 0.31 0.43 0.65
SecondQuartile 0.00 0.05 0.39
ThirdQuartile 0.12 0.24 0.55
FourthQuartile 0.55 0.63 0.84
Graft
First Quartile 0.13 0.23 0.49
SecondQuartile 0.00 0.03 0.26
Third Quartile 0.08 0.21 0.36
Fourth Quartile 0.65 0.72 0.80
This table reports the fraction of all countries whose point estimate of govemance
falls in the indicated quartile for which the corresponding x%/o confidence interval
also falls entirely within that quartile, for each of the three govemance aggregates
and for a range of values of x.
Table 5: Consequences of Correlated Disturbances
AverageStandardErrorof GovemanceAggregate
Basedon RepresentativeIndicators
AssumedError Correlation:
p=0 p=0. 2 5 p=0.5
34
Figure1: Estimatesof Governance
Go.,m.t EfIhws.
zT I TTT I
T4Si L*. R
.< i:.1U._
~~~~~~~~t. ._
2Z5
2o5
35
Figure2: Significanceof PairwiseGovernanceComparisons
I GovenmentEffectvenes
0.9
Z0.
a - 10%SignificanceLevel
071
X0.3
0.03**8
0.9
c 0.4
0.
_! 02 80%S10%ne LevelficSn ev
0. 6 - *
&0.29-
w~~~~~~~~~~~~
'2E0 1 * GaL
. f Lo
0.8
point estimates
of governance. For e1achScountr c,nweLpvloto h etclaih
.O0.7 - B
e t. Sis gnificanceL e< >*E<,, .+
0-
O
E0.3
between
5%and 95% (ark dots) 1ri0eten2%
Sglandc70eL (vligtdt) o
reasons of space,0 conr nm sae niatd0.7freer itcuty
gO S~~~3
Figure 3: Precision of GovernanceAggregates
as a Function of PatirwiseCorrelations of Observed Data
90% Confidenceintervalfor
mediancountryif
correlationof observeddata is:
p=0 .7 5
90 \
p=O.
Distributionof Actual
Govemance
-3 -2 -1 0 1 2 3
Govemance Index
0.7 -
5 0.6
0.5
0.5
O S = \ S , CountryatMedian
o Z%0.2-
~E
to
-n
0.
37
Figure 4: Significance of Pairwise Governance Comparisons
Based on Ordinal Method
GovemmentEffectivenea
01
Oa * . . s. S *,-a -
0.7 *
}07~~~~~ *-a *-,'g @ "
0.7
0. SigrificanceLevel
1~50%
0.3,
IL L
Rule of Law
0.9 - -t* -
m 0.6 ,
j0.4 - #N 5
03 so%SJgnmcae50%SignificanceLevel
0.2
Grt
-V.
i 0.9
6]107~~ * . U **a. *
10.8 U** .
i 0 ° 5g * g , a.¢
0C3
16
li~~~~~~ ~~~~~~50%
° SignificanceLevel
a
0.2-
0.1
*- 0
38
Figure 5: Alternative Distributions for Governance
0.a
0.02 .
0.015
0.01 05-nMTE 9
0000~~~~~~~~~~~~~~WI
0.1
0.020
0.02 A >.
0.0210.
0.015 0000~~~~~~~~~~~~~~~~~~~~~~~~~.
02
0.02~~~~~~~~~~~~~~~~~~~~~~~~'
0.0Th 1
0.018~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
0.015
o 0.2 04 0.a 08 1 0
0.2
coum we2 s oAthe shap of thsdsrbto.I0 h ih oun eso h
0.1
0005
39
Policy Research Working Paper Series
Contact
Title Author Date for paper
WPS2186Contagion,BankLendingSpreads, Pierre-Richard
Ag6nor September1999 T. Shiel
and OutputFluctuations JoshuaAizenman 36317
AlexanderHoffmaister
Policy Research Working Paper Series
Contact
Title Author Date for paper