ECON-C4210 - Econometrics II: Capstone

Lecture 3: Difference in difference regression

Otto Toivanen

Toivanen ECON-C4210 Lecture 3 1 / 36

Learning outcomes

• At the end of lectures 3 & 4, you

1 understand what a difference-in-difference (DiD) estimator is

2 how it identifies the effect of the treatment

3 what the identifying assumptions are

4 what the economic content of the identifying assumptions are

5 how to implement DiD

6 what an Average Treatment Effect (ATE) is.

Toivanen ECON-C4210 Lecture 3 2 / 36

Difference estimator

• Let’s assume a discrete treatment (=all those who get the treatment get the same
treatment):

Ti = 1 if individual i gets the treatment.

Ti = 0 if individual i does not get the treatment.

• Notice how - unlike in perhaps medicine - giving the same treatment in a social science
context can be difficult to ensure (or even to define).

Toivanen ECON-C4210 Lecture 3 3 / 36

Randomized Control Trial (RCT)

• In economics:
1 The researcher decides on what the experiment is.
2 The researcher decides what the population of interest is.
3 The researcher draws a random sample.
4 Individuals in the random sample are randomly allocated into control and treatment groups.

Toivanen ECON-C4210 Lecture 3 4 / 36

Difference estimator

Yi = β0 + β1 Ti + i

Q1: what is the interpretation of β1 ?

Q2: is there an omitted variable problem?

Q3: what if individuals truly randomized and the researcher observes other characteristics
besides Yi , Ti ?

Toivanen ECON-C4210 Lecture 3 5 / 36

RTC estimator

Q4: is there any reason to include control variables (Wi )?

1 Efficiency - corr (Ti , Wi ) = 0 by design.

2 Control for randomization: if β1 without controls 6= β1 with controls (= in a statistically

(& economically) significant way), then RCT has failed.

Toivanen ECON-C4210 Lecture 3 6 / 36

RCT and the Difference estimator

• Treatment: E[Y |T = 1] = β0 + β1 × 1

• Control: E[Y |T = 0] = β0 + β1 × 0

• Difference: E[∆Y ] = β1

• This is why a t-test on the difference in Y between treatment and control groups often
sufficient.

Toivanen ECON-C4210 Lecture 3 7 / 36

So what is DiD?

Imagine the researcher has 2 consecutive observations / individual:

1 Before experiment (period t = 1).

2 After experiment (period t = 2).

Toivanen ECON-C4210 Lecture 3 8 / 36

So what is DiD?

Two possibilities:

1 Everybody gets the treatment → event study.

2 Some get the treatment (treatment group), some don’t (control group) →
Difference-in-difference (DiD) setup.

• Notice: the use of the term ”event study” fluctuates somewhat.

• We concentrate on DiD.

Toivanen ECON-C4210 Lecture 3 9 / 36

Examples of DiD setups

Bloom, N., Liang, J., Roberts, J. & Ying, Z. J. (2015). Does working from home work?
Evidence from a Chinese experiment. The Quarterly Journal of Economics, 130(1),
165–218

1 Effect of work from home (WFH) on productivity.

2 Difference in change of performance between those workers that shift to WFH and those
who stay in office.

Toivanen ECON-C4210 Lecture 3 10 / 36

Examples of DiD setups

Gil, R. (2015). Does vertical integration decrease prices? Evidence from the paramount
antitrust case of 1948. American Economic Journal: Economic Policy, 7(2), 162–91

1 Effect of vertical integration (VI) on downstream prices (=theatre tickets).

2 Difference in change of ticket prices between VI and non-VI theatres due to removing VI.

Toivanen ECON-C4210 Lecture 3 11 / 36

Examples of DiD setups

Aghion, P., Akcigit, U., Hyytinen, A. & Toivanen, O. (2022). A year older, a year wiser
(and farther from frontier): Invention rents and human capital depreciation. Review of
Economics and Statistics, forthcoming

1 Effect of invention on wages of (co-)workers.

2 Difference in change of wages before and after invention for individuals in inventing and
non-inventing firms.

3 Differential effects by co-worker type.

Toivanen ECON-C4210 Lecture 3 12 / 36

Regression framework
Write generally

Yit = α0 + βgroup Ti + βperiod Dafter ,t + β1 Ti Dafter ,t + it

• Ti = 1 if in treatment group, 0 if in control group. Notice missing t-index.

• Ti Dafter ,t = Ti × Dafter ,t .
• Notice t-index. It allows us to keep track of whether an observation is before or after
the introduction of the treatment. t ∈ {1, 2}.
• Dafter ,t = 1 if period after, 0 otherwise. Notice missing i-index.
• α0 = common constant. Notice that with individual level data, we can replace α0 with
αi , but then cannot identify βgroup any more / do not need it any more.
• it = error term / unobservables.
Toivanen ECON-C4210 Lecture 3 13 / 36
Regression framework - control group

For control group Ti = 0 rewrite:

Yit = α0 + βperiod Dafter ,t + β1 Ti Dafter ,t + it

= α0 + βperiod Dafter ,t + it

Toivanen ECON-C4210 Lecture 3 14 / 36

Regression framework - control group

Yit = α0 + βperiod Dafter ,t + it

1 After: Yi2 = α0 + βperiod + i2

2 Before: Yi1 = α0 + i1

3 Diff: ∆Yi = βperiod + i2 − i1

Toivanen ECON-C4210 Lecture 3 15 / 36

Regression framework - treatment group

For treatment group Ti = 1 rewrite:

Yit = α0 + βgroup Ti + βperiod Dafter ,t + β1 Ti Dafter ,t + it

= α0 + βgroup + βperiod Dafter ,t + β1 Dafter ,t + it

Toivanen ECON-C4210 Lecture 3 16 / 36

Regression framework - treatment group

Yit = α0 + βgroup + βperiod Dafter ,t + β1 Dafter ,t + it

1 After: Yi2 = α0 + βgroup + βperiod + β1 + i2

2 Before: Yi1 = α0 + βgroup + i1

3 Diff: ∆Yi = βperiod + β1 + i2 − i1

• Notice how we could replace α0 + βgroup with αi if we have individual level data. αi , too,
would vanish in the differencing over time in both the control and the treatment group.

Toivanen ECON-C4210 Lecture 3 17 / 36

DiD

1 Treatment group: E[∆Y ] = βperiod + β1

2 Control group: E[∆Y ] = βperiod

3 DiD: E[∆∆Y ] = β1

• β1 is the Average Treatment Effect, (ATE), as it measures the average change in Yit due
to the treatment.
• An RCT also delivers (an estimate of) ATE.

Toivanen ECON-C4210 Lecture 3 18 / 36

Diff vs. DiD

• DiD needs data over at least 2 periods.

• DiD allows for individual specific constants if you have data on the same individuals
before and after.
• → DiD doesn’t necessitate randomization.

1 Identifying assumption #1: common trends: The outcome variable would have
developed similarly in the treatment group as it did in the control group, had the
treatment group not received the treatment.

2 Identifying assumption #2: E[it |X it , Ti , Dafter ,t , αi ] = 0.

Toivanen ECON-C4210 Lecture 3 19 / 36

Diff vs. DiD - Identifying assumption #1

Common trends.

1 When would this be violated?

2 Technically, βperiod|control 6= βperiod|treatment .

3 Call βperiod|treatment − βperiod|control = ∆βperiod .

Toivanen ECON-C4210 Lecture 3 20 / 36

Diff vs. DiD - Identifying assumption #1

1 Treatment group: E[∆Y ] = Y2 − Y1 = β1 + βperiod|treatment

2 Control group: E[∆Y ] = Y2 − Y1 = βperiod|control

3 DiD: E[∆∆Y ] = β1 + ∆βperiod 6= β1 if and only if ∆βperiod 6= 0; in other words, if and

only if βperiod|treatment 6= βperiod|control .

Toivanen ECON-C4210 Lecture 3 21 / 36

Diff vs. DiD - Identifying assumption #1

• Substantively?
• Example #1: Bloom et al., 2015
• Those that know their productivity is (permanently) declining decide to work from home (or
office).
• Example #2: Gil, 2015
• Think of the effect of hiring a new CEO on firm performance. Firm observes performance is
(permanently) declining compared to peers, and therefore hires a new CEO.
• Example #3: Aghion et al., 2018
• The treatment firms are in growing markets where within-firm human capital important. The
trend growth of wages therefore different from that of control group firms.

Toivanen ECON-C4210 Lecture 3 22 / 36

Diff vs. DiD - Identifying assumption #2

• Identifying assumption #2: E[it |X it , Ti , Dafter ,it , αi ] = 0.

→ selection into treatment can depend on individual specific ”things” that are constant over
the periods.
• Even 2 period DiD allows control variables.

• Controls may be more important than in an RCT to reduce variation & to remove
omitted variable bias.

Toivanen ECON-C4210 Lecture 3 23 / 36

Diff vs. DiD - Identifying assumption #2

• Identifying assumption #2: E[it |X it , Ti , Dafter ,it , αi ] = 0.

• When would this be violated?

• Technically, the ”shock” in the 1st period leads somebody to (not) choose the treatment.

Toivanen ECON-C4210 Lecture 3 24 / 36

Diff vs. DiD - Identifying assumption #2
• Substantively?
• Example #1: Bloom et al., 2015
• Those that know their productivity was (temporarily) lower decide to work from home (or
office).
• Example #2: Gil, 2015
• Think again of the effect of hiring a new CEO on firm performance. Firm observes a shock
to performance compared to peers, and therefore hires a new CEO.
• Example #3: Aghion et al., 2018
• Inventions usually do not come as a surprise. The firm may change wages in anticipation of
the invention.
• Notice AAHT take this into account by dividing the treatment period into pre- and
post-invention. The problem may still remain.

Toivanen ECON-C4210 Lecture 3 25 / 36

What data for more than 2 periods?

• More data always a plus.

• Makes distinction between (differential) trends and temporary shocks clearer.

• Can allow for more flexible models (e.g. introduction of time/period dummies; testing of
common trends using treatment group - time period dummy interactions).
• BUT: notice that one stretches what αi captures.

• Remember: Even 2-period DiD allows control variables.

Toivanen ECON-C4210 Lecture 3 26 / 36

A complication on all causal estimators

• What have we assumed about the effect of treatment on the control group?

• That there is none.

= ”no general equilibrium effects”.

= ”Stable Unit Treatment Value Assumption (SUTVA)”.

Toivanen ECON-C4210 Lecture 3 27 / 36

A complication on all causal estimators

• When is this an issue?

• In a lab, think of infectious diseases.

• Regarding human behavior, think of interactions (markets).

• Important but difficult topic. We will neglect it, as is all too often done in the literature,
too.

Toivanen ECON-C4210 Lecture 3 28 / 36

SUTVA

• Example #1: a merger affects the prices of all firms (products) in the market, not just
those of the merging parties.
• Example #2: a wholesale education reform (think of the Finnish reform making secondary
education compulsory) affects the wages of not only those whose education changes
because of the reforms, but also of those who compete with them in the job market.
• Example #3: a regulatory reform affects the prices of all pharmaceuticals based on the
same molecule. Kortelainen et al. (2023) study reforms in Nordic pharmaceutical markets
and define prices both at the
1 market-level and the
2 package-level.

Toivanen ECON-C4210 Lecture 3 29 / 36

ATT: Main outcome variables

Part I Part II Part III

Finland 2003 Finland 2009 Denmark 2000 Denmark 2005 Norway 2005 Sweden 2009
VGS → GS GS → IRP IRP → ERP ERP → IRP GS → SP-IRP GS-IRP → Auction-IRP
Average Expenditure -0.03 -0.13* -0.05* 0.04 -0.21* -0.27*
[ -0.07, 0.01] [ -0.18, -0.08] [ -0.09, -0.01] [ -0.01, 0.09] [ -0.29, -0.12] [ -0.34, -0.20]
Number of Product Names 0.01 0.04 -0.02 -0.01 -0.01 0.04
[ -0.03, 0.05] [ -0.02, 0.10] [ -0.06, 0.02] [ -0.05, 0.03] [ -0.15, 0.15] [ -0.00, 0.09]
Average Price -0.04 -0.05 -0.07* 0.07* -0.10 -0.04
[ -0.12, 0.04] [ -0.09, -0.00] [ -0.12, -0.01] [ 0.02, 0.12] [ -0.18, -0.00] [ -0.11, 0.04]
Number of Doses 0.01 0.04* 0.00 0.07* 0.04 0.12*
[ -0.04, 0.07] [ 0.01, 0.07] [ -0.04, 0.04] [ 0.03, 0.12] [ -0.00, 0.09] [ 0.02, 0.22]
Wholesale Price -0.05 -0.10* -0.09* 0.05 -0.11* -0.06*
[ -0.11, 0.02] [ -0.14, -0.07] [ -0.13, -0.05] [ -0.02, 0.12] [ -0.20, -0.01] [ -0.11, -0.01]
1 Estimator: Two-way fixed effects and Callaway and Sant’Anna (2020). Outcome data source: DLI-MI (1999–2013), Farmastat (2004–
2013), Fimea (1999–2012), IQVIA MIDAS Quarterly Sales and IQVIA MIDAS (2007–2013).
2 * = statistically significant at the 95% confidence level. 10000 replications for ATC-5 wild bootstrapped standard errors.

Kortelainen, M., Markkanen, J., Siikanen, M. & Toivanen, O. (2023). The effects of price
regulation on pharmaceutical expenditure and availability [Unpublished manuscript].
Toivanen ECON-C4210 Lecture 3 29 / 36
Choosing the comparison group

• A large body of literature has demonstrated that key to success in using DiD (more
generally, in identifying causal effects) is the choice of the control group.
• Control group observation units should be “as similar” to treatment group observation
units.

→ conditional DiD.
• Conditional = first choose carefully which observation units to include in the control
group.
• When done correctly, this helps a great deal.

Toivanen ECON-C4210 Lecture 3 30 / 36

Choosing the comparison group

• Execution of conditional DiD:

1 Choose some key characteristics.

2 Choose treatment group observation unit #1.

3 Go through potential control group observation units and choose a unit / units that are
as similar as the treatment group observation unit #1. Many different technical solutions
to implement this.

4 Repeat for all treatment group observation units.

Toivanen ECON-C4210 Lecture 3 31 / 36

Choosing the comparison group

• Aghion, P., Akcigit, U., Hyytinen, A. & Toivanen, O. (2018). On the returns to invention
within firms: Evidence from finland. AEA Papers and Proceedings, 108, 208–12
• We study what happens to wages of individuals after invention.

• We split individuals in a firm into 4 groups:

1 Inventors

2 Entrepreneurs

3 White-collar workers

4 Blue-collar workers

Toivanen ECON-C4210 Lecture 3 32 / 36

Treatment group

• Those in the treatment group work in the same firm as the inventor in the year of the
patent application.

Toivanen ECON-C4210 Lecture 3 33 / 36

Control group

• Those in the control group:

1 Never work in a firm that invents.

2 Have the same socioeconomic status (excl. inventors)

3 Are similar to an inventor in terms of
i Education (MSc or not)
ii Age (< 30, 31 − 40, 41 − 50; > 50)
iii Quintiles of firm size
iv IQ (< 50th percentile, 51st − 80th , 81st − 90th , > 90).

Toivanen ECON-C4210 Lecture 3 34 / 36

Results

Toivanen ECON-C4210 Lecture 3 35 / 36

DiD-issues we ignore but which should be considered

• Different units get the treatment at different times (at the extreme, all units eventually
get the treatment, leading to an event study setting).
• Different units get a different treatment. Example: Finnish cost subsidies to firms during
the COVID-19 crises vary from 2 000€ to 500 000€.
• The effect of the treatment is different for different treatment units, possibly conditional
on observables.
• DID methods have developed rapidly in the last few years re all these issues. It is now
well understood that the base two-way FE DID may produce biased results in settings
that are even a little more complicated than the textbook setting.

Toivanen ECON-C4210 Lecture 3 36 / 36