Ungarded problem set 4

Difference in differences
Question 1
The 1993 Family Medical Leave Act (FMLA) mandates at least three months of maternity leave
for working women. Prior to 1993, thirteen states had their own version of this law in place, but
you do not know the exact dates. You are interested in the impact of maternity leave on birth
weight, which is a measure of infant health. You have individual level data on birth weight,
along with state and year identifiers, and demographic information, such as the mother’s race and
age. The data spans 1987 to 1997.

a. How would you use the above setting to measure the impact of maternity leave on birth
weight? Write down the equation you plan to estimate (don’t forget to include subscripts on
your variables). Include clear definitions of all variables and explain how you would
interpret your coefficient of interest. What are the assumptions underlying your estimation
strategy?

BWGHTist= b0+b1TREATs+ b2POSTt+ b3(TREATsXPOSTt)+ b4 (controls)ist + uist

where

TREATs=0 if one of the 13 states that already had maternity leave prior to FMLA
=1 if one of the other 37 states

POSTt=1 if after1993, 0 otherwise

b3 is the impact of FMLA on birth weight.

Assumptions: Parallel trends and assume nothing else changed at the time of FMLA (for
example, health care coverage didn’t improve at the same time)

b. Would you still be able to estimate the impact of the FMLA on birth weight if you only had
data from 1994 to 1997? Why or why not? If not, please explain what you can no longer
include as a control for with the more limited data, and why this is a problem.

Answer: If we only use data post-FMLA (1993), a naïve way to estimate the impact of the
policy is to run the regression of birth weight on a dummy for TREAT (defined above).
However, we can no longer control for permanent differences in birth weight between the
treated states and control states. Surely, the states which had a form of maternity leave prior
to 1993 may have had other provisions for mothers, such as increased access to health care,
which may have impacted birth weight.
Question 2
You are estimating the impact of clean water on child mortality in Thailand, a rapidly developing nation.
The World Bank has data on child mortality in two areas that are otherwise similar, but one (region A)
received clean piped water in 1994, before the other (region B) which received clean piped water in
1996.

Region Child Mortality 1993 per 1000 Child mortality 1995

A 10 6
B 4 2

1. Write down the dif in dif estimator of the impact of clean piped water on child mortality. Clearly
label all variables. (3)
Mortality = b0 + b1*1(region=A) + b2*post1994 + b3*1(region=A)*post1994 + e
2. Based on the table above, what is the value of the difference in difference estimator? (provide a
number). (3)
(6-10) – (2-4) = -2
3. Now tell me the value of each coefficient included in your regression in 1. You can figure this out
based on the table above. (6)

B0 = 4, b1 = 6, b2 = -2, b3 = -2

Question 3
Hypothetical situation: Congress has just increased the federal minimum wage of 7.25 by 20%. A staff
economist at the Dept of Labor has been asked to estimate the impact of this federal increase in the
minimum wage law on employment.

A. What does economic theory say the effect should be? (You should be able to answer this question
with tools/intuition learned in intermediate micro).

Minimum wage increases labor costs for employers. This may reduce employment. (3)

B. You offer to look at US employment before and after the law change to answer this question. Please
write down the econometric specification (eg, the equation) that would accomplish this. What might be
the problem with this approach? Would your estimate represent a true causal effect? What threats
would there be to identification?

Employment rate_t = b0 + b1*Fed minimum wage + b2*X + e (4)

No, because of secular trend in employment would bias the estimate (3)

C. As you prepare to run the above regression, your colleague informs you that the new federal
minimum wage is still below the state minimum wage in about 15 states (note, the highest minimum
wage, either state or federal, is what binds). Will this affect your analysis? Now what regression
should you run? Clearly label all variables and be clear about how they are defined. Write down your
equation and tell me whether your proposed specification is an improvement over part B) or not, and
why.

Yes, I would use DD specification. We can control the secular trend. There is an improvement because
control states would capture the counterfactual, what would have happened to employment in the
absence of the increase in the minimum wage. (4)

DD specification (6)

Employment_i,t = b0 + b1*Post_t + b2*T_i + b3*Post_t*T_i + e_i,t

Post = 1 if federal raise the minimum wage, and =0 otherwise, T = 1 if states except for those 15, =0 if
those 15 states.

Question 4

A district in India has a large number of missing girls – 900 girls for 1000 boys at age 5. There is evidence
to suggest that girls are missing because of both sex-selected abortion (girls are more likely to be
aborted and therefore never born) and neglect in early childhood that leads to premature death before
age 5. To combat this, the government decided that for every girl born, if that girl is immunized and
enrolled in school and survives to her 18 th birthday, the family will be guaranteed a generous cash
transfer equivalent to the annual salary for an average Indian worker. This subsidy is available for all
girls born after August 30, 2005. The program was announced on August 15, 2005. (This is an actual
existing policy).

A. You want to estimate whether this program increases parental investment in girls (it’s intended
effect). Your measure of investment is how much food girls are being fed (nutritional intake) at age
5. What identification concerns do you have, if any? You decide to implement a regression
discontinuity design with information on investments in girls (nutrition) for girls born between July
and September 2005. How does this address your concerns? Write down your equation.
 Concern: clearly there are underlying trends in the female/male sex ratio over time that might
complicate a pre-post design. The regression discontinuity should reduce the influence of any
underlying trends.
B. You try another design which is a dif in dif where you use another district that implemented no such
policy as a control district. How does the dif in dif strategy address the identification problem? How
will you select your control district? In other words, what kinds of things should be similar in
treatment and control districts? What is the dif in dif specification you would run? Label everything.

Control district should be similar in terms of the initial sex ratio, female education, employment,
importantly underlying trends in sex ratio (prior to the reform) should be the same.

Addresses the identification problem of underlying trends in the sex ratio being collinear with the
reform.

Dif in dif:

Nutrition at age 5. = a0 + a1Treatment District + a2Post reform a3Treatment*post reform + a5 year-

month FE + a6 X + e

A2 represents the dif in dif estimate

C. You run the dif in dif using 4 years of data – 2 pre and 2 post reform. You find that when you look at
girls born within 6 months of the reform there is a large increase in average investments in girls, but
when you go further out (2 years after the reform), average investments in girls seems to decline
somewhat over time from their initial high point right after the reform. Can you think of any reason
why that might be? (hint: think about the flu paper and concerns Almond raised in that paper).
Please be specific. This is likely the most difficult question on the exam.
In the first 6 months of the program, there should be no selection into fertility and birth. Therefore,
we are only getting the effect of the program on girls carried to term.
2 years after the reform, there may be selection into fertility. More/different girls may be born after
the reform because parents adjust to the reform. These girls may suffer worse outcomes/be subject
to fewer investments (this is the opposite of culling) and this might be bringing the average down.

Question 5
Assume that in 2010 all the high-tech firms in Rhode Island became unionized at the same time, while in
Connecticut there are no unions in any high-tech firm. Emily has the following variables for high-tech
firms in Rhode Island and Connecticut, between 2000 and 2018:

 pit – patents issued by firm i in year t.

 RI i – a dummy variable that equals to 1 if the firm is in Rhode Island and 0 if it is in Connecticut.
 sit – size of firm i in year t
 pos t t - a dummy variable that equals to 1 if the year is after 2010
Emily wants to estimate the effect of unions by comparing high-tech firms in Rhode Island and
Connecticut, before and after 2010.

Write an appropriate econometric model. What are the identification assumptions needed to interpret
her results as the causal effect of unions?

Model:

pit = β0 + β 1∗RI i+ β2∗pos t t + β3∗RI i∗pos t t + β 4∗sit + ϵ it

We can also include year fixed effects.

Identification assumptions: Connecticut looks like RI would have looked without the unions, no different
trends in the two states in any variable that might be related to patents (beside unions).

Question 6 (harder)
You work for Valentinian III, the Western Roman Emperor in the years 425 to 455. Attila the
Hun invaded Italy in 452 and conquered some parts of it. One of Attila’s goals was to capture
productive regions in order to supply food for his army. Luckily, Pope Leo I convinced Attila to
leave, so he left Italy in the same year. The Emperor suspects that the conquest had long-term
effects on agricultural output even after the Huns left, due to immigration and destruction of
infrastructure.

You want to estimate the impact of being conquered by the Huns on agricultural output. You
collect the following data from a large sample of villages in Northern Italy before the invasion
and after the Huns left, for the years 451 and 453:

 T i – Treatment group indicator, 1 if village i was under the Huns occupation in 452, 0
otherwise
 Y it – Agricultural output of village i at year t
 Rit – Rainfall in village i at year t

Assume that besides the Huns, the only thing that might affect agricultural output is rainfall.

a. Using a lot of research assistances (because the Romans didn’t have computers or Stata), you
run the following econometric model, only for the year 453:

Y i=β 0 + β 1∗T i + β 2∗Ri +ϵ i

Is β 1 a biased measure for the causal effect of being conquered?

Circle the correct answer:
1. Direction of the bias:
a. This will probably lead to a negative bias in the coefficient of interest.
 b. This will probably lead to a positive bias in the coefficient of interest.
2. Overestimation / underestimation:
a. Due to the bias we are overestimating the effect of the variable of interest.
b. Due to the bias we are underestimating the effect of the variable of interest.
Explain the threat for the identification and why this is the direction of the bias.

Because Attila wanted to capture more productive regions, we have a problem of reverse
causality. The output was initially higher in the conquered villages, and it could remain relatively
high in 453. We might get a positive estimator for β 1, or it will be less negative than the real
effect – positive bias, underestimating the negative effect.

b. Suggest an identification strategy for estimating the causal effect of being conquered, using all
the years in the data. Write your econometric model, and explain why it is better than the model
of section a. What are the key identification assumptions behind your strategy? Can you collect
more data that might help you support these assumptions?

Diff-in-diff:
Y it = β0 + β 1∗T i + β 2∗After+ β 3∗T i∗After + β 4∗R it +ϵ i
where After=1 if the year is 453 and 0 if the year is 451.

This model controls for unobserved differences between the control and the treatment group that
don’t change with time, or for unobserved time trends that are the same for all villages.

Identification assumption: no different time trends for the treatment and control group between
451 and 453 (besides the trends that are a result of the occupation). We assume that the change
over time in the control villages represents the change that we would have observed in the
treatment villages if the Huns did not invade. You can collect data on previous years, before 451,
to show that there were no different pre-treatment trends in both groups.

c. Northern Italy is divided to 3 different provinces: Northeast, Northwest and North. Each
province includes both control and treatment villages. Aelia, one of your research assistants,
argues that between 440 and 455 the Northeast experienced fast technological progress due to a
new watermill technology. This technology might have also partly diffused to North province.
You don’t have data about the watermills.
Aelia says that because of that you should run the model you designed in the previous section
separately for each province. Marius, another research assistance, argues that you can still use
one regression for all the provinces and control for the effect of this technological progress.
Write an econometric model that can do what Marius argues. What is the difference between the
two suggestions of Aelia and Marius?

Controlling for different intercepts and time trends in each province (omitting the Northwest):

Y it = β0 + β 1∗T i + β 2∗After+ β 3∗T i∗After + β 4∗R it +¿

+ β 5∗North∗After+ β6∗Northeast∗After + β 7∗North+ β 8∗Northeast + ϵ i

The difference between this model and running the previous model separately for each province
is that here we assume that the treatment effect and the rainfall effect are the same for all
provinces. If we run separate models, we assume different effects for the treatment and rainfall in
each province.

2017 Mid-term 2

1. The city of Los Angeles passed a law requiring restaurants to post their dept of health inspection
certificate that includes a large letter grade (A-F) on their front doors starting in the year 2000.
You hypothesize that this led to an increase in restaurant revenue because restaurants would
respond by cleaning up and customers would respond by going out more for food because they
would be more confident in the cleanliness of the restaurant. You have data on the revenues of
all business establishments in California that includes where they are located (the city), the type
of business (restaurant, accounting firm, movie theatre, etc.) and the amount of revenue for each
year 1998-2003.

A. Propose a dif in dif estimator to estimate the impact of this law change. Be clear about your
treatment and control groups as well as your treatment variable. Include your estimating
equation and label everything. 4

Y = α0 + α1 Treatment group + α2 Post Reform + α3 Treatment group*Post reform + ε

Y = restaurant revenue
Treatment group=restaurants in LA
Control group= restaurants in San Diego or another comparable city that displays similar
pre-trends
Post reform = indicator for years 2000-2003 (zero for years 1998-1999)
The dif in dif estimator is α3 and represents the difference in revenues, post reform – pre-
reform, in LA relative to the same difference in San Diego.
 B. What is the use of the control designed to address? Be specific by providing an example of
something the control is designed to address including what the bias would be in our
estimate if we didn’t employ a dif in dif strategy. 4

Tourism increasing generally in California would lead to an increase in restaurant revenue over
time that was unrelated to the reform. This would lead to an upward bias in our estimate of the
impact of the reform.

C. You find that restaurant revenues do in fact increase in your dif in dif specification, but your
colleague points out that in 2001 Harry Potter World opened in LA and that had a huge
impact on tourism in the city, increasing it considerably. Does this threaten your dif in dif
strategy? Why or why not? 8

Yes, it suggests that San Diego (or any other city in CA) is not a good control for LA.

D. If it threatens your DIf in Dif estimate, can you propose another one that will address this
concern? 8

Treatment group = restaurants in LA

Control group = hotels in LA
Now the control group represents a better counterfactual than restaurants in SD because it will
also see an increase in revenue due to the increase in tourism associated with Harry Potter
World.
Treatment group=restaurants
Control group= hotels (or other establishments that should also be affected by increased
tourism)

1. In class we discussed the example of estimating the impact of free child care in the form of universal
pre-kindergarten on women’s labor force participation as measured by the employment rate of all
women ages 18-64 using a difference-in-differences estimation strategy. The treatment state was
Maryland which implemented universal pre-k in 1993 and the control state was VA. We used CPS
data for 1990-1995 which is an annual survey of household that includes: year, state of birth,
gender, age, marital status, race, number and age of children, educational attainment, occupation,
and labor force participation.
a. Write down the estimating equation and label everything. What are the parameters of
interest? (3)
b. Someone points out to you that in 1993 Maryland passed a law outlawing gender-based
discrimination at work , generating significant financial penalties for instances of
discrimination. Will this affect your estimates? Why and how? (7)

c. Can you think of another way to test your original hypothesis with the data you have?
Please write down your estimating equation and explain why it is identified. (10)
2016 Mid-term 2

2. Three strikes and you’re out.

You hypothesize that harsher sentencing reduces future criminal activity by deterring people from
criminal activity. The state of California implemented a three strikes and you’re out law in November,
1994. The law had multiple stipulations, but for the purpose of this question let’s just focus on one and
assume there were no others. The law stipulated that if you already had 2 prior felony convictions and
were convicted of a third, you would receive a sentence 8 times longer than if they only had 1 prior
felony conviction. You have individual panel data on all criminal activity for individuals from 1990
through 2015 in every state in the US. These data include the date of the current arrest, defendant
zipcode of residence, defendant age, gender, race, ethnicity, criminal history, current criminal charge
and current sentence.

A. Briefly describe an identification strategy that will help you to recover a causal estimate. 10

B. The law was passed in March 1994 and was heavily covered in the media at the time, but the law did
not go into effect until November 1994. How do you think this might influence your estimates above?
2018 Midterm 2

c. Assume that in 2010 all the high-tech firms in Rhode Island became unionized at the same time, while
in Connecticut there are no unions in any high-tech firm. Emily has the following variables for high-tech
firms in Rhode Island and Connecticut, between 2000 and 2018:

 pit – patents issued by firm i in year t.

 RI i – a dummy variable that equals to 1 if the firm is in Rhode Island and 0 if it is in Connecticut.
 sit – size of firm i in year t
 pos t t - a dummy variable that equals to 1 if the year is after 2010

Emily wants to estimate the effect of unions by comparing high-tech firms in Rhode Island and
Connecticut, before and after 2010.

Write an appropriate econometric model. What are the identification assumptions needed to interpret
her results as the causal effect of unions?

Model:

pit = β0 + β 1∗RI i+ β2∗pos t t + β3∗RI i∗pos t t + β 4∗sit + ϵ it

We can also include year fixed effects.

Identification assumptions: Connecticut looks like RI would have looked without the unions, no different
trends in the two states in any variable that might be related to patents (beside unions).

Question 2 (42%)

a. Esther Duflo estimates the effect of a school construction program in Indonesia on schooling and
future wages. Suppose that during the time of the program many Indonesian families chose to
immigrate from areas in which the program was less intensive to the areas in which the program was
more intensive, in order to increase the schooling of their children. β 1 is the diff-in-diff estimator for the
effect of the program, assume it is positive. Circle the correct answer:

1. We have a positive bias to the estimator of β 1 and we are overestimating the effect
2. We have a negative bias to the estimator of β 1 and we are overestimating the effect
3. We have a positive bias to the estimator of β 1 and we are underestimating the effect
4. We have a negative bias to the estimator of β 1 and we are underestimating the effect
5. There is no bias in the estimator of β 1
6. We cannot tell the direction of the bias or if there is a bias

Explain (an answer without explanation will not count):

In the areas in which the program was more intensive schooling will increase both because of the school
construction and because of immigrants who are more interested in schooling, relatively to areas in
which the program was less intensive, who lost those more-ambitious families.

b. Hoyt Bleakley estimate the effect of a public health program in the US South on schooling and future
wages. Suppose that during the time of the program many families chose to immigrate from areas that
had high infection rates to the areas that had low infection, in order to prevent their children from being
infected and improve their children’s schooling outcomes. β 1 is the diff-in-diff estimator for the effect of
the program, assume it is positive. Circle the correct answer:

1. We have a positive bias to the estimator of β 1 and we are overestimating the effect
2. We have a negative bias to the estimator of β 1 and we are overestimating the effect
3. We have a positive bias to the estimator of β 1 and we are underestimating the effect
4. We have a negative bias to the estimator of β 1 and we are underestimating the effect
5. There is no bias in the estimator of β 1
6. We cannot tell the direction of the bias or if there is a bias

Explain (an answer without explanation will not count):

In the areas that were infected more, schooling might increase because of the program, but it will
increase less than what it could have been, relatively to the less infected areas, because migrants who
are relatively interested in schooling move to the less infected areas. Because of the migrants we will
see a larger increase in schooling in the control group, the less infected areas, so the relative increase of
schooling in the treatment group, the more infected areas, will be smaller.

2015 Mid-term 2
A. Go back to the full sample. Propose a difference in differences estimator that would allow you to
estimate the impact of financial aid on college completion. Write down the equation you would
estimate, clearly label everything, and indicate which coefficient represents the dif in dif
estimator. (10)

College_completion = b0 + b1*1(income<=$40k) + b2*1(year>=73) +

b3*1(income<=$40k)*1(year>=73)+e

B3 = DD estimator
 What is your counterfactual in this set up? What assumptions are you making for this dif in dif to
generate a causal estimate?

Trends in college completion among students with >=$40k.

Parallel pre trends of college completion between treatment and control group

B. This is the hardest question on the exam. Now you learn that the amount of the Pell grant varied
with not only income (with poorer children receiving greater grant amounts), but that even
conditional on parental income, the amount of the Pell grant increased with family size (see
graph below showing this).