bayesian methods for social and political measurement

simon jackman stanford university http://jackman.stanford.edu [email protected]

Measurement Models
ei1 ei2 ... eiJ

measurement error

yi1

yi2

...

yiJ

indicators

xi

latent variable
simon jackman - aaas 2006

Latent Variables Everywhere

ideology of legislators & judges (e.g., roll call analysis, to obtain ideal points) preferences of political parties (e.g., Manifesto project) ability/intelligence: IQ, SATs, GREs. attitudes/psychological states. public opinion: support for candidates over an election campaign
simon jackman - aaas 2006

Latent Variables Everywhere

preferences of Congressional districts, or states (e.g., normal vote) democracy of country i at time t (e.g., Polity or Freedom House) concepts in international relations: threat, resolve, capabilities.

simon jackman - aaas 2006

Indicators Can Be Poor

few indicators lack of coverage (not all sources providing data at all times for all objects of interest) combinations of various levels of measurement (binary, ordinal, continuous) bias (esp with human sources) reliability (esp with human sources)
simon jackman - aaas 2006

Bayesian Inference
in Bayesian modeling, inference is via the posterior density of the model parameters

p(h|data) p(h) p(data|h)

posterior prior likelihood

simon jackman - aaas 2006

Bayesian Inference for Measurement Models

x are latent traits are parameters that tap quality of the indicators y are indicators (data)

p(x, b|y) px (x)pb (b) p(y|x, b)

posterior prior likelihood
simon jackman - aaas 2006

Bayesian Inference for Measurement Models

what should I believe about a latent variable given available indicators and priors? how uncertain am I as to values of the latent variable? important to know how uncertain we are when contemplating (costly) actions

simon jackman - aaas 2006

Why Bayes?
1. no nuisance parameters (inference for both latent traits and item parameters) 2. incorporation of any relevant prior information 3. computational ease, post-MCMC revolution

simon jackman - aaas 2006

Examples in Contributed Paper

1. legislators revealed preferences (analysis of roll call data; was John Kerry the most liberal senator?) 2. evaluating candidates for graduate school (combining an admission committees assessments with GREs) 3. tracking public opinion over an election campaign (published opinion polls are noisy and possibly biased indicators of actual vote intentions) 4. social networks: representing network ties as distances between actors in an abstract/latent space
simon jackman - aaas 2006

Analysis of Roll Call Data

ROL L C AL L DATA

simon jackman - aaas 2006

Analysis of Roll Call Data

legislators preferences are points in a low dimensional policy space (ideal points) each vote presents each legislator with a choice between two locations in the policy space (Yea and Nay locations)

simon jackman - aaas 2006

Quadratic Loss Utilities

Ui (Yj ) = -(xi - Yj )2 + eijY Ui (Nj ) = -(xi - Nj )2 + eijN Pr(yij = 1) = Pr[Ui (Yj ) > Ui (Nj )] = F (xi bj - j ) where bj = 2(Yj - Nj ) 2 2 j = Yj - Nj Yj + Nj j /bj = 2
simon jackman - aaas 2006

PERFECT SPATIAL VOTING

(Y+N)/2
Y Y Y Y Y Y N N N N N N

simon jackman - aaas 2006

Latent Variable (Utility Difference)

* yij * yij

ui (Yj ) - ui (Nj ) yij = 1 Yea yij = 0 Nay

>0 M0

* yij

simon jackman - aaas 2006

Latent Linear Regression

ui (Yj ) = -(xi - Yj ) + gij 2 ui (Nj ) = -(xi - Nj ) + mij
* yij 2

= ui (Yj ) - ui (Nj ) = -(xi - Yj ) + (xi - Nj ) + gij - mij 2 2 = 2xi (Yj - Nj ) - Yj + Nj + gij - mij
2 2

* yij

rj

= xi bj - j + eij
simon jackman - aaas 2006

Statistical Model
eij N(0, 1), i, j * Pr(Yeaij ) = Pr(yij > 0) = Pr xi bj - j + eij > 0 = U xi bj - j ,

simon jackman - aaas 2006

Statistical Model
Pr(yij = 1) = F (xi bj - j )
probit/logit with unobserved covariate corresponds to a 2-parameter itemresponse theory (IRT) model in educational testing, y are responses of students taking tests; x is latent ability; is item-discrimination; is item-difculty
simon jackman - aaas 2006

1.00

Item-characteristic curve
Pr(yij = 1) = F (xi bj - j )

Pr(yij = 1)

0.50

0.75

0.25

~ = (Y + N )/2 = /b xj j j j j
Y N

x1 0.00

x2

x3

x4

x5

x6

x7

x8

x9

x10

x11

x12

simon jackman - aaas 2006

Item characteristic curves

1.00 0.75 0.50 0.50 Y
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11

0.25

N
x12

0.25

0.75

1.00

Y
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11

N
x12

1.00 0.00

0.75

0.50

0.25

Y
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11

N
x12

0.25

0.50

0.75

1.00 0.00

Y
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11

N
x12

0.00

0.00

simon jackman - aaas 2006

legislators ideal points lie in 2 dimensional space

vote Aye and Nay on rollcall j

simon jackman - aaas 2006

IMPERFECT SPATIAL VOTING

x xb {~ : ~ j = j }
misclassication

ea Y

misclassication

ay
simon jackman - aaas 2006

Implementation Issues

Identication: invariance scale change and translation, and, in higher dimensions, rotation. Estimation: lots of parameters

simon jackman - aaas 2006

Formal Statement as a Bayesian Model

j N(0, 10 ) 2 bj N(0, 10 ) xi N(0, 1) yij |xi Bern(pi ) -1 U (pi ) = xi bj - j
iid 2

priors

} model (likelihood)
prior
simon jackman - aaas 2006

f (, b, x|Y) f (Y|, b, x)f (, b, x) posterior

likelihood

simon jackman - aaas 2006

Two dimensional representation of 105th Senate

.25

-.75 -.50 WELLSTONE

DASCHLE

-.25

BREAUX

CHAFEE

LOTT MCCAIN .25 .50 ASHCROFT

.75

-.25

simon jackman - aaas 2006

Supreme Court, 213 non-unanimous decisions, 1994-97

2 1 0 1

Thomas Scalia Rehnquist O'Connor Kennedy Souter Ginsburg Bryer Stevens

simon jackman - aaas 2006

9 t h Ci rcui t , Im m i grat i o n Appe al s

3 2 1 0 1 2 3 2 1 0 1 2

1 2 simon jackman - aaas 2006

Example: is Kerry the most liberal Senator?

National Journal 62 key votes for 2003 NJs statistical analysis nds Kerry to be the most liberal Senator; released in Feb 2004. Republicans tagged Kerry with this label through the 2004 presidential campaign

simon jackman - aaas 2006

High Rates of Absenteeism

Kerry missed 60% of NJs key roll calls in 2003 Lieberman missed 40% Edwards missed 35% Graham missed 24% Bush administrations announced positions, missing for 61%
simon jackman - aaas 2006

3 2 1 0

Snowe (RME) Chafee (RRI) Nelson (DNE) Breaux (DLA) Bayh (DIN) Baucus (DMT) Lincoln (DAR) Jeffords (IVT) Carper (DDE) Landrieu (DLA) Conrad (DND) Pryor (DAR) LIERBERMAN (DCT) Feinstein (DCA) Nelson (DFL) Inouye (DHI) Dorgan (DND) Reid (DNV) Biden (DDE) Bingaman (DNM) Kohl (DWI) Schumer (DNY) Cantwell (DWA) Hollings (DSC) Rockefeller (DWV) Dodd (DCT) GRAHAM (DFL) Akaka (DHI) Byrd (DWV) Murray (DWA) Leahy (DVT) Daschle (DSD) Wyden (DOR) Johnson (DSD) Stabenow (DMI) Kennedy (DMA) EDWARDS (DNC) Corzine (DNJ) Durbin (DIL) Feingold (DWI) Mikulski (DMD) Lautenberg (DNJ) Dayton (DMN) Levin (DMI) Clinton (DNY) Boxer (DCA) Harkin (DIA) Reed (DRI) Sarbanes (DMD) KERRY (DMA)

McConnell (RKY) Allen (RVA) Thomas (RWY) Sessions (RAL) Santorum (RPA) Lugar (RIN) Hatch (RUT) Grassley (RIA) Cornyn (RTX) Cochran (RMS) Bunning (RKY) Burns (RMT) Bennett (RUT) Frist (RTN) Nickles (ROK) Alexander (RTN) Domenici (RNM) Crapo (RID) Craig (RID) Bond (RMO) Enzi (RWY) Inhofe (ROK) Kyl (RAZ) Lott (RMS) Chambliss (RGA) Miller (DGA) Dole (RNC) Allard (RCO) Voinovich (ROH) Roberts (RKS) Brownback (RKS) DeWine (ROH) Talent (RMO) Ensign (RNV) Hagel (RNE) Warner (RVA) Coleman (RMN) Hutchison (RTX) Graham (RSC) Fitzgerald (RIL) Gregg (RNH) Sununu (RNH) Smith (ROR) Shelby (RAL) Stevens (RAK) Murkowski (RAK) Campbell (RCO) Collins (RME) Specter (RPA) McCain (RAZ)

20

40

60

40

60
McConnell (RKY) Burns (RMT) Allen (RVA)

80

100

Snowe (RME) Chafee (RRI) Nelson (DNE) Breaux (DLA) Bayh (DIN) Baucus (DMT) Lincoln (DAR) Jeffords (IVT) Carper (DDE) Landrieu (DLA) Conrad (DND) Pryor (DAR) LIERBERMAN (DCT) Feinstein (DCA) Nelson (DFL) Inouye (DHI) Dorgan (DND) Reid (DNV) Schumer (DNY) Bingaman (DNM) Biden (DDE) Kohl (DWI) Cantwell (DWA) Hollings (DSC) Rockefeller (DWV) GRAHAM (DFL) Dodd (DCT) Akaka (DHI) Byrd (DWV) Wyden (DOR) Leahy (DVT) Daschle (DSD) Murray (DWA) Johnson (DSD) Stabenow (DMI) Kennedy (DMA) EDWARDS (DNC) Corzine (DNJ) Durbin (DIL) Feingold (DWI) Mikulski (DMD) Levin (DMI) Lautenberg (DNJ) Dayton (DMN) Clinton (DNY) Harkin (DIA) Boxer (DCA) Reed (DRI) Sarbanes (DMD) KERRY (DMA)

Bennett (RUT) Frist (RTN) Nickles (ROK) Bond (RMO) Domenici (RNM) Alexander (RTN) Enzi (RWY) Crapo (RID) Craig (RID) Kyl (RAZ) Inhofe (ROK) Lott (RMS) Chambliss (RGA) Talent (RMO) Dole (RNC) DeWine (ROH) Roberts (RKS) Ensign (RNV) Miller (DGA) Voinovich (ROH) Allard (RCO) Brownback (RKS) Hagel (RNE) Warner (RVA) Hutchison (RTX) Coleman (RMN) Graham (RSC) Fitzgerald (RIL) Sununu (RNH) Gregg (RNH) Smith (ROR) Shelby (RAL) Stevens (RAK) Murkowski (RAK) Campbell (RCO) Collins (RME) Specter (RPA) McCain (RAZ)
Thomas (RWY) Sessions (RAL) Santorum (RPA) Lugar (RIN) Hatch (RUT) Grassley (RIA) Cornyn (RTX) Cochran (RMS) Bunning (RKY)

20

40

60

40

60

80

100

20

40

60

80

100

20

40

60

80

100

1 20 40

Snowe (RME) Chafee (RRI) Nelson (DNE) Breaux (DLA) Baucus (DMT) Bayh (DIN) Lincoln (DAR) Jeffords (IVT) Carper (DDE) Pryor (DAR) Landrieu (DLA) Conrad (DND) LIERBERMAN (DCT) Feinstein (DCA) Nelson (DFL) Inouye (DHI) Dorgan (DND) Reid (DNV) Bingaman (DNM) Biden (DDE) Schumer (DNY) Hollings (DSC) Rockefeller (DWV) Kohl (DWI) Byrd (DWV) Cantwell (DWA) GRAHAM (DFL) Dodd (DCT) Akaka (DHI) Murray (DWA) Kennedy (DMA) Leahy (DVT) Johnson (DSD) Daschle (DSD) Stabenow (DMI) Wyden (DOR) EDWARDS (DNC) Corzine (DNJ) Feingold (DWI) Durbin (DIL) Mikulski (DMD) Levin (DMI) Clinton (DNY) Lautenberg (DNJ) Dayton (DMN) Boxer (DCA) Harkin (DIA) Sarbanes (DMD) Reed (DRI) KERRY (DMA)

BUSH (RUSA) Allen (RVA) McConnell (RKY) Burns (RMT) Thomas (RWY) Sessions (RAL) Santorum (RPA) Lugar (RIN) Hatch (RUT) Grassley (RIA) Cornyn (RTX) Cochran (RMS) Bunning (RKY) Bennett (RUT) Frist (RTN) Nickles (ROK) Alexander (RTN) Domenici (RNM) Enzi (RWY) Bond (RMO) Inhofe (ROK) Crapo (RID) Craig (RID) Chambliss (RGA) Lott (RMS) Kyl (RAZ) Voinovich (ROH) Brownback (RKS) Allard (RCO) DeWine (ROH) Dole (RNC) Ensign (RNV) Roberts (RKS) Miller (DGA) Hagel (RNE) Talent (RMO) Hutchison (RTX) Coleman (RMN) Warner (RVA) Gregg (RNH) Fitzgerald (RIL) Sununu (RNH) Graham (RSC) Shelby (RAL) Smith (ROR) Stevens (RAK) Murkowski (RAK) Campbell (RCO) Collins (RME) Specter (RPA) McCain (RAZ)

60

80

100

20

40

60

80

100

KERRY (DMA)
0.4 0.4

Reed (DRI)
0.4

Sarbanes (DMD)
0.4

Harkin (DIA)

0.3

0.3

0.3

Density

Density

Density

Density

0.2

0.2

0.2

0.1

0.1

0.1

0.0

0.0

0.0

10

20

30

40

10

20

30

40

10

20

30

40

0.0

0.1

0.2

0.3

10

20

30

40

Rank

Rank

Rank

Rank

Boxer (DCA)
0.4 0.4

Dayton (DMN)
0.4

Lautenberg (DNJ)
0.4 Density

Clinton (DNY)

0.3

0.3

0.3

Density

Density

Density

0.2

0.2

0.2

0.1

0.1

0.1

0.0

0.0

0.0

10

20

30

40

10

20

30

40

10

20

30

40

0.0

0.1

0.2

0.3

10

20

30

40

Rank

Rank

Rank

Rank

R A N K O R D E R I N G , 107 T H S E N AT E , P L U S B U S H A N N O U N C E D P O S I T I O N S
1 20 40 60 40 60
BUSH (RUSA) Helms (RNC) Kyl (RAZ) Gramm (RTX) Nickles (ROK) Thomas (RWY) Lott (RMS) Bunning (RKY) Smith (RNH) Enzi (RWY) Frist (RTN) Allard (RCO) Thurmond (RSC) Inhofe (ROK) Santorum (RPA) McConnell (RKY) Craig (RID) Gregg (RNH) Bennett (RUT) Bond (RMO) Roberts (RKS) Brownback (RKS) Hatch (RUT) Burns (RMT) Thompson (RTN) Murkowski (RAK) Hagel (RNE) Sessions (RAL) Lugar (RIN) Crapo (RID) Grassley (RIA) Voinovich (ROH) Allen (RVA) Hutchinson (RTX) Ensign (RNV) Stevens (RAK) Domenici (RNM) Shelby (RAL) Hutchinson (RAR) Campbell (RCO) Cochran (RMS) Dewine (ROH) Warner (RVA) Fitzgerald (RIL) Smith (ROR) McCain (RAZ) Collins (RME) Snowe (RME) Specter (RPA) Miller (DGA) Chafee (RRI) Jeffords (RVT)

80

100

1 20

Nelson (DNE) Breaux (DLA) Baucus (DMT) Lincoln (DAR) Cleland (DGA) Landrieu (DLA) Carper (DDE) Bayh (DIN) Torricelli (DNJ) Feinstein (DCA) Carnahan (DMO) Johnson (DSD) Byrd (DWV) Jeffords (IVT) Kohl (DWI) Bingaman (DNM) EDWARDS (DNC) GRAHAM (DFL) Wyden (DOR) Nelson (DFL) Daschle (DSD) Feingold (DWI) Conrad (DND) Dorgan (DND) LIEBERMAN (DCT) Hollings (DSC) Cantwell (DWA) Biden (DDE) Rockefeller (DWV) Murray (DWA) Reid (DNV) Schumer (DNY) Inouye (DHI) KERRY (DMA) Stabenow (DMI) Harkin (DIA) Akaka (DHI) Clinton (DNY) Mikulski (DMD) Dodd (DCT) Durbin (DIL) Levin (DMI) Kennedy (DMA) Leahy (DVT) Corzine (DNJ) Boxer (DCA) Reed (DRI) Sarbanes (DMD) Dayton (DMN) Wellstone (DMN)

Barkley (IMN)

Bush

Kerry

60

40

60

40

80

100

Elaborations
on close-votes, legislators are subject to party-specic inducements (Synder & Groseclose; CJR) we have extra information about ideal points (constituency characteristics; party afliations; legislative career) sequence matters: the status quo remains constant until something passes (Clinton & Meirowitz) estimating uncovered sets (Bianco and Sened, APSR, 2005)
simon jackman - aaas 2006