Lecture 3: Dierences-in-Dierences

Fabian Waldinger

Waldinger () 1 / 47
Topics Covered in Lecture

1 Dierences-in-Dierences Basics: Card & Krueger (1994).

2 Regression Dierences-in-Dierences.
3 Synthetic Controls: Abadie & Gardeazabal (2003).
4 Combining Dierences-in-Dierences with IV: Waldinger (2010).

Waldinger (Warwick) 2 / 47
Dierences-in-Dierences: Card & Krueger (1994)

Suppose you are interested in the eect of minimum wages on

employment (a classic and controversial question in labour
economics).
In a competitive labour market, increases in the minimum wage would
move us up a downward-sloping labour demand curve.
! employment would fall.

Waldinger (Warwick) 3 / 47
Dierences-in-Dierences: Card & Krueger (1994)

Card & Krueger (1994) analyse the eect of a minimum wage

increase in New Jersey using a dierences-in-dierences methodology.
In February 1992 NJ increased the state minimum wage from $4.25 to
$5.05. Pennsylvanias minimum wage stayed at $4.25.

They surveyed about 400 fast food stores both in NJ and in PA both
before and after the minimum wage increase in NJ.

Waldinger (Warwick) 4 / 47
Dierences-in-Dierences Strategy
DD is a version of xed eects estimation. To see this more formally:
Y1ist : employment at restaurant i, state s, time t with a high wmin .
Y0ist : employment at restaurant i, state s, time t with a low wmin .
In practice of course we only see one or the other.
We then assume that:

E [Y0ist js, t ] = s + t

In the absence of a minimum wage change, employment is determined

by the sum of a time-invariant state eect s and a year eect t
that is common across states.
Let Dst be a dummy for high-minimum wage states and periods.
Assuming E [Y1ist Y0ist js, t ] = is the treatment eect, observed
employment can be written:

Yist = s + t + Dst + ist

Waldinger (Warwick) 5 / 47
Dierences-in-Dierences Strategy II

In New Jersey:
Employment in February is:
E [Yist js = NJ, t = Feb ] = NJ + Feb
Employment in November is:
E [Yist js = NJ, t = Nov ] = NJ + Nov +
the dierence between February and November is:
E [Yist js = NJ, t = N ] E [Yist js = NJ, t = F ] = N F +
In Pennsylvania:
Employment in February is:
E [Yist js = PA, t = Feb ] = PA + Feb
Employment in November is:
E [Yist js = PA, t = Nov ] = PA + Nov
the dierence between February and November is:
E [Yist js = PA, t = Nov ] E [Yist js = PA, t = Feb ] = Nov Feb

Waldinger (Warwick) 6 / 47
Dierences-in-Dierences Strategy

The dierences-in-dierences strategy amounts to comparing the

change in employment in NJ to the change in employment in PA.
The population dierences-in-dierences are:

E [Yist js = NJ, t = N ] E [Yist js = NJ, t = F ]

E [Yist js = PA, t = Nov ] E [Yist js = PA, t = Feb ] =

This is estimated using the sample analog of the population means.

Waldinger (Warwick) 7 / 47
Dierences-in-Dierences Table

Surprisingly, employment rose in NJ relative to PA after the minimum

wage change.
Waldinger (Warwick) 8 / 47
Regression DD

We can estimate the dierences-in-dierences estimator in a

regression framework.
Advantages:
It is easy to calculate standard errors.
We can control for other variables which may reduce the residual
variance (lead to smaller standard errors).
It is easy to include multiple periods.
We can study treatments with dierent treatment intensity. (e.g.
varying increases in the minimum wage for dierent states).
The typical regression model that we estimate is:

Outcomeit = 1 + 2 Treati + 3 Postt + 4 (Treat * Post)it +

Treatment = a dummy if the observation is in the treatment group

Post = post treatment dummy

Waldinger (Warwick) 9 / 47
Regression DD - Card & Krueger

In the Card & Krueger case the equivalent regression model would be:
Yist = + NJs + dt + (NJs dt ) + ist
NJ is a dummy which is equal to 1 if the observation is from NJ.
d is a dummy which is equal to 1 if the observation is from November
(post).
This equation takes the following values.
PA Pre:
PA Post: +
NJ Pre: +
NJ Post: + + +
Dierences-in-Dierences estimate: (NJ Post - NJ Pre) - (PA Post -
PA Pre) =

Waldinger (Warwick) 10 / 47
Graph - Observed Data

Waldinger (Warwick) 11 / 47
Graph - DD

Yist = + NJs + dt + (NJs dt ) + ist

Waldinger (Warwick) 12 / 47
Graph - DD

Yist = + NJs + dt + (NJs dt ) + ist

Waldinger (Warwick) 13 / 47
Graph - DD

Yist = + NJs + dt + (NJs dt ) + ist

Waldinger (Warwick) 14 / 47
Graph - DD

Yist = + NJs + dt + (NJs dt ) + ist

Waldinger (Warwick) 15 / 47
Key Assumption of Any DD Strategy: Common Trends
The key assumption for any DD strategy is that the outcome in
treatment and control group would follow the same time trend in the
absence of the treatment.
This does not mean that they have to have the same mean of the
outcome!
Common trend assumption is di cult to verify but one often uses
pre-treatment data to show that the trends are the same.
Even if pre-trends are the same one still has to worry about other
policies changing at the same time.

Waldinger (Warwick) 16 / 47
Regression DD Including Leads and Lags

Including leads into the DD model is an easy way to analyze

pre-trends.
Lags can be included to analyze whether the treatment eect changes
over time after treatment.
The estimated regression would be:
1 m
Yist = s + t + Ds + Ds + Xist + ist
= q =0
treatment occurs in year 0.
includes q leads or anticipatory eects.
includes m leads or post treatment eects.

Waldinger (Warwick) 17 / 47
Study Including Leads and Lags - Author (2003)

Autor (2003) includes both leads and lags in a DD model analyzing

the eect of increased employment protection on the rms use of
temporary help workers.
In the US employers can usually hire and re workers at will.
Some states courts have made some exceptions to this employment at
will rule and have thus increased employment protection.
Dierent states have passed these exeptions at dierent points in
time.
The standard thing to do is to normalize the adoption year to 0.
Autor then analyzes the eect of these exeptions on the use of
temporary help workers.

Waldinger (Warwick) 18 / 47
Results

The leads are very close to 0. ! no evidence for anticipatory eects

(good news for the common trends assumption).
The lags show that the eect increases during the rst years of the
treatment and then remains relatively constant.

Waldinger (Warwick) 19 / 47
Standard Errors in DD Strategies

Many papers using a DD strategy use data from many years (not only
1 pre and 1 post period).
The variables of interest in many of these setups only vary at a group
level (say state) and outcome variables are often serially correlated.
In the Card and Krueger study for example, it is very likely that
employment in each state is not only correlated within the state but
also serially correlated.
As Bertrand, Duo, and Mullainathan (2004) point out, conventional
standard errors often severely understate the standard deviation of the
estimators.

Waldinger (Warwick) 20 / 47
Standard Errors in DD Strategies - Practical Solutions
Bertrand, Duo, and Mullainathan propose the following solutions:
1 Block bootstrapping standard errors (if you analyze states the block
should be the states and you would sample whole states with replacing
for the bootstrapping).
2 Clustering standard errors at the group level. (in STATA one would
simply add cl(state) to the regression equation if one analyzes state
level variation).
3 Aggregating the data into one pre and one post period.
Literally works only if there is only one treatment date. With staggered
treatment dates one should adopt the following procedure:
Regress Yst on state FE, year FE, and relevant covariates.
Obtain residuals from the treatment states only and divide them into 2
groups: pre and post treatment.
Then regress the two groups of residuals on a post dummy.
Correct treatment of standard errors sometimes makes the number of
groups very small: in the Card and Krueger study the number of
groups is only 2.
Waldinger (Warwick) 21 / 47
Synthetic Control Methods

In some cases, treatment and potential control groups do not follow

parallel trends.
! Standard DD method would lead to biased estimates.
Abadie & Gardeazabal (2003) pioneered a synthetic control method
when estimating the eects of the terrorist conict in the Basque
Country using other Spanish regions as a comparison group. (Card
(1990) implicitly used a very similar approach in his Mariel boatlift
paper investigating the eect of immigration on employment of
natives).
The basic idea behind synthetic controls is that a combination of
units often provides a better comparison for the unit exposed to the
intervention than any single unit alone.

Waldinger (Warwick) 22 / 47
Abadie & Gardeazabal (2003) - The Eect of Terrorism on
Growth

They want to evaluate whether Terrorism in the Basque Country had

a negative eect on growth.
They cannot use a standard DD method because none of the other
Spanish regions followed the same time trend as the Basque Country.
They therefore take a weighted average of other Spanish regions as a
synthetic control group.

Waldinger (Warwick) 23 / 47
The Basque Country is Dierent from the Rest of Spain

Waldinger (Warwick) 24 / 47
The Synthetic Control Method

They have J available control regions (the 16 Spanish regions other

than the Basque Country).
They want to assign weights W = (w1 , ..., wJ )0 a (J x 1) vector to
each region. (wj 0 & wj = 1; this ensures that there is no
extrapolation outside the support of the growth predictors for the
control regions).
The weights are chosen so that the synthetic Basque country most
closely resembles the actual one before terrorism.

Waldinger (Warwick) 25 / 47
The Synthetic Control Method - Details

Let X1 be a (K x 1) vector of K pre-terrorism economic growth

predictors (i.e. the values in the previous table: investment ratio,
population density, ...) in the Basque Country.
Let X0 be a (K x J) matrix which contains the values of the same
variables for the J possible control regions.
Let V be a diagonal matrix with nonnegative components reecting
the relative importance of the dierent growth predictors.
The vector of weights W* is then chosen to minimize:

(X1 X0 W)V(X1 X0 W)

They choose the matrix V such that the real per capita GDP path for
the Basque Country during the 1960s (pre terrorism) is best
reproduced by the resulting synthetic Basque Country.

Waldinger (Warwick) 26 / 47
The Synthetic Control Method - Details

The optimal weights they get are: Catalonia: 0.8508, Madrid: 0.1492,
and all other regions: 0.
Alternatively they could have just chosen the weights to reproduce
only the pre-terrorism growth path for the Basque country (and not
the growth predictors as well. In that case they would have minimized:

(Z1 Z0 W)(Z1 Z0 W)

Z1 is the (10 x 1) vector of pre-terrorism (1960-1969) GDP values for the

Basque Country.
Z0 is the (10 x J) vector of pre-terrorism (1960-1969) GDP values for the J
potential control regions.

Waldinger (Warwick) 27 / 47
The Synthetic Basque Country Looks Similar

Waldinger (Warwick) 28 / 47
Constructing the Counterfactual Using the Weights

Y1 is a (T x 1) vector whose elements are the values of real per

capital GDP values for T years in the Basque country.
Y0 is a (T x J) matrix whose elements are the values of real per
capital GDP values for T years in the control regions.
They then constructed the counterfactual GDP (in the absence of
terrorism) as: Y*1 =Y0 W*

Waldinger (Warwick) 29 / 47
Growth in the Basque Country with and without Terrorism

Waldinger (Warwick) 30 / 47
Terrorist Activity and Estimated GDP Gap

Waldinger (Warwick) 31 / 47
Combining DD and IV

Sometimes combining DD and IV methods can be quite useful.

In a recent paper (Waldinger, 2010), I have done that to estimate the
eect of faculty quality on the outcomes of PhD students.
Estimating the eect of faculty quality on PhD student outcomes is
challenging because of:
1 Selection of good students into good universities.
2 Omitted variables aecting both faculty quality and student outcomes.
3 Measurement error in faculty quality.
I address these issues by using the dismissal of scientists in Nazi
Germany as an exogenous shock to faculty quality.
The dismissal aected some departments very strongly, while other
departments were not aected.

Waldinger (Warwick) 32 / 47
Historical Background

Germany was the leading country for scientic research at the

beginning of the 20th century.
Immediately after gaining power in 1933 the new Nazi government
dismissed all Jewish and politically unreliablescholars from the
German universities.

Waldinger (Warwick) 33 / 47
Dismissed Professors Across German Universities

Waldinger (Warwick) 34 / 47
Dismissed Professors Across German Universities II

Waldinger (Warwick) 35 / 47
Eect of Dismissals on Department Size

Waldinger (Warwick) 36 / 47
Eect of Dismissals on Faculty Quality

Waldinger (Warwick) 37 / 47
Panel Data on PhD graduates from German Universities

I obtained a panel dataset of all mathematics PhD students

graduating from all German universities between 1923 and 1938 and
use the dismissal as exogenous variation in faculty quality.
The empirical strategy essentially compares changes in outcomes of
PhD students in aected department before and after 1933 to
changes in outcomes in unaected departments.
I investigate the following outcomes:
1 Whether former PhD student publishes dissertation in a top journal.
2 Whether former PhD student ever becomes full professor.
3 # of lifetime citations.
4 Positive lifetime citations.

Waldinger (Warwick) 38 / 47
Reduced Form Graphical Analysis - Publishing Dissertation

Waldinger (Warwick) 39 / 47
Reduced Form Graphical Analysis - Full Professor

Waldinger (Warwick) 40 / 47
Reduced Form Graphical Analysis - Lifetime Citations

Waldinger (Warwick) 41 / 47
Reduced Form Estimates

The reduced form of the dismissal eect is essentially a DD estimator.

Outcomeidt = 1 + 2 (Dismissal induced Reduction in Faculty Quality)dt

+ 3 (Dismissal induced increase in Student/Faculty Ratio)dt
+ 4 Femaleidt + 5 Foreignidt + 6 CohortFEt + 5 DepFEd + idt

Dismissal induced Reduction in Faculty Quality is 0 until 1933 and

equal to the dismissal induced fall in faculty quality after 1933 (and
remains 0 in departments without dismissals).
Dismissal induced increase in Student/Faculty Ratio is also 0 until
1933 but equal to the dismissal induced increase in student/faculty
ratio after 1933
! essentially a dierences-in-dierences estimator but with dierent
treatment intensities.

Waldinger (Warwick) 42 / 47
Reduced Form Estimates

Waldinger (Warwick) 43 / 47
Common Robustness Check for Parallel Trend Assumption
Only Look at Pre-Period Data and Move Placebo Treatment some Years Back

Here I move a placebo treatment to 1930.

Waldinger (Warwick) 44 / 47
Use Dismissal as IV

OLS model to the eect of university quality on PhD student

outcomes:
Outcomeidt = 1 + 2 (Avg. Faculty Quality)dt 1
+ 3 (Student/Faculty Ratio)dt 1
+ 4 Femaleidt + 5 Foreignidt + 6 CohortFEt + 7 DepFEd + idt
University quality and student/faculty ratio are endogenous ! use
dismissal as IV.
2 Endogenous Variables ! 2 First Stage Regressions:
1 Avg. Faculty Qualityidt = 1
+ 2 (Dismissal induced Reduction in Faculty Quality)dt
+ 3 (Dismissal induced increase in Student/Faculty Ratio)dt
+ 4 Femaleidt + 5 Foreignidt + 6 CohortFEt + 5 DepFEd + idt
2 Student/Faculty Ratioidt = 1
+ 2 (Dismissal induced Reduction in Faculty Quality)dt
+ 3 (Dismissal induced increase in Student/Faculty Ratio)dt
+ 4 Femaleidt + 5 Foreignidt + 6 CohortFEt + 5 DepFEd + idt
Waldinger (Warwick) 45 / 47
First Stages

To test for weak instruments one cannot simply look at the rst stage
F-statistics because here we have 2 endogenous regressors and 2 IVs.
! use Cragg-Donald EV statistic here critical value is 7.03.
Waldinger (Warwick) 46 / 47
OLS and IV

Waldinger (Warwick) 47 / 47