HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES?

COPING WITH AMBIGUITY USING BOUNDED-VARIATION ASSUMPTIONS

Charles F. Manski and John V. Pepper*

Abstract—Despite dozens of studies, research on crime has struggled to researchers have found that RTC laws either have little
reach consensus about the impact of right-to-carry (RTC) gun laws. With
this in mind, we formalize and apply a class of bounded-variation impact or may increase violent crime rates.2 Consider, for
assumptions that flexibly restrict the degree to which outcomes may vary example, Aneja, Donohue, and Zhang (2011), who make
across time and space. Using these assumptions, we present empirical seemingly minor modifications to the basic model and data
analysis of the effect of RTC laws on violent and property crimes in Vir-
ginia, Maryland, and Illinois. Imposing specific assumptions that we (e.g., reducing the number of demographic covariates) that
believe worthy of consideration, we find that RTC laws increase some Lott (2010) used. Whereas Lott (2010) found that RTC laws
crimes, decrease other crimes, and have effects that vary over time decrease the different violent crime rates by between 5%
for others.
and 8%, Aneja et al. (2011) found that RTC laws have a
negligible impact on murder but increase other violent
I. Introduction crime rates by 20% to 30%. Aneja et al. (2011) also report
that many of the point estimates are statistically insignifi-

R ESEARCH on crime in the United States has com-

monly used data on county or state crime rates to eval-
uate the impact of laws allowing individuals to carry con-
cant by conventional criteria.
This ambiguity may seem surprising. How can research-
ers using similar data draw such different conclusions? In
cealed handguns—so-called right-to-carry (RTC) laws. fact, it has long been known that inferring the magnitude
Theory alone cannot predict even the direction of the and direction of treatment effects is an inherently difficult
impact. The knowledge or belief that potential victims may undertaking. Suppose that one wants to learn how crime
be carrying weapons may deter commission of some crimes rates (an outcome of interest) would differ with and without
but may escalate the severity of criminal encounters.1 Ulti- an RTC law (a treatment) in a given place and time. Data
mately, how allowing individuals to carry concealed weap- cannot reveal counterfactual outcomes. That is, data cannot
ons affects crime is an empirical question. reveal what the crime rate in an RTC state would have been
Lott (2010) describes this empirical research to a lay if the state had not enacted the law. Nor can data reveal
audience in a book with the provocative and unambiguous what the crime rate in a non-RTC state would have been if
title: More Guns, Less Crime. Yet despite dozens of studies, an RTC law had been in effect. To identify the law’s effect,
the research provides no clear insight on whether ‘‘more one must somehow fill in the missing counterfactual obser-
guns’’ leads to less crime. Some studies find that RTC laws vations. This requires making assumptions that cannot be
reduce crime, others find that the effects are negligible, and tested empirically. Different assumptions may yield differ-
still others find that such laws increase violent or property ent inferences.
crime or both. In a series of papers starting in 1997, Lott Yet the empirical research on RTC laws has struggled to
and coauthors have argued forcefully that RTC laws have find consensus on a set of credible assumptions. Reviewing
important deterrent effects that can play a role in reducing the literature, the National Research Council (NRC, 2005)
violent crime. Lott and Mustard (1997) and Lott (2010), for Committee to Improve Research Information and Data on
example, found that RTC laws reduce crime rates in every Firearms concluded that it is not possible to infer a credible
violent crime category by between 5% and 8%. Using dif- causal link between RTC laws and crime using the current
ferent models and revised or updated data, however, other evidence. Indeed, the committee concluded that: ‘‘addi-
tional analysis along the lines of the current literature is
Received for publication March 15, 2016. Revision accepted for publi- unlikely to yield results that will persuasively demonstrate’’
cation March 15, 2017. Editor: Bryan S. Graham.
* Manski: Northwestern University; Pepper: University of Virginia. (p. 150) this link. The committee found that findings are
We thank David Rodina for research assistance. We are grateful to Dan highly sensitive to model specification. Yet there is no solid
Black, Jennifer Doleac, Jeff Dominitz, John Donohue, Mark Duggan, Joel foundation for particular assumptions and, as a result, no
Horowitz, Brent Kreider, John Lott, Stephane Mechoulan, Daniel Nagin,
Franco Peracchi, David Rivers, and participants at the 2015 Southern obvious way to prefer specific results (also see Durlauf
Economic Association Conference and the 2016 NBER Economics of et al., 2016). Hence, drawing credible precise findings that
Crime Working Group Summer Institute session for helpful comments. lead to consensus about the impact of RTC laws has thus
We are also grateful to three anonymous reviewers for their comments.
1
Donohue and Levitt (1998) formalize a simple stylized model show- far proven to be impossible.
ing that in illegal markets with scarce resources, firearms (or lethality How then should research proceed? We think that analy-
more generally) do not have a clear impact on violence. This model sis of treatment response is most useful if researchers per-
implies that the impact of firearms might vary across markets and over
time, a finding inconsistent with standard models used in empirical analy- form inference under a spectrum of assumptions of varying
sis. Duggan (2001) and Durlauf, Navarro, and Rivers (2016) make a simi-
lar point when noting that the impact of RTC laws may be sensitive to
2
rates of gun ownership and the underlying environments and propensities See, for example, Black and Nagin (1998), Duggan (2001), Aneja
for criminality. We elaborate in section III. et al. (2011), and Durlauf et al. (2016).

The Review of Economics and Statistics, May 2018, 100(2): 232–244

Ó 2018 by the President and Fellows of Harvard College and the Massachusetts Institute of Technology
doi:10.1162/rest_a_00689
 HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES? 233

identifying power, recognizing the tension between the Virginia. In an online appendix, we further illustrate the
strength of assumptions and their credibility. Research on approach by examining what can be inferred about the
RTC laws has commonly made invariance assumptions impact of RTC laws in Maryland and Illinois, two states
asserting that specified features of treatment response are that did not adopt RTC statutes during the period we study.
constant across space or time. These assumptions may be In section III, we illustrate the sensitivity of inferences to
too strong, but weaker assumptions asserting bounded var- different identifying restrictions but do not argue in a favor
iation may be credible. of any particular assumption. Here, as in earlier research,
For example, an invariance assumption may assert that we find it valuable to entertain a set of assumptions of vary-
neighboring states such as Virginia and Maryland have ing strength and determine how the strength of the assump-
identical environments and propensities for criminality. tions imposed affects the strength of the conclusions drawn.
This assumption may seem attractive because it enables (See Manski & Pepper, 2000, 2013, for other applications
contemporaneous comparison of the two states, but it may of this research philosophy).
not be credible. Yet it may be credible to assume that these In section IV, we motivate particular bounded-variation
states are similar to one another. Likewise, an invariance assumptions and present results on the impact of RTC laws
assumption may be that in the absence of RTC statutes, Vir- under these assumptions. Here, we find that RTC laws
ginia and Maryland would have experienced the same appear to reduce some crimes, increase others, and for
change in murder rates between two periods, say, 1988 and some, the results vary over time. In Virginia for example,
1990. This assumption enables difference-in-differences the data combined with these assumptions imply that the
analysis of treatment response, but, again, it may not be RTC law in the state reduced murder and larceny rates in
credible. Yet it may be credible to assume that the two nearly every year after 1989 but increased assaults after
states would have experienced similar changes. 1997. For other crimes, the sign of the bounds is generally
With this in mind, we formalize and apply a class of negative in the 1990s but positive or indeterminate in the
bounded-variation assumptions that flexibly restrict the var- 2000s. Section V draws conclusions.
iation of treatment response across states and years. These
assumptions generally do not point-identify the effects of
II. Data
RTC laws on crime rates. Instead, they partially identify
them, yielding bounds rather than precise conclusions.3 To evaluate the impact of RTC laws on crime, we use
Some readers may believe that partial identification ana- state-level data on annual crime rates (per 100,000 resi-
lysis cannot lead to informative conclusions about the dents) from 1970 to 2007. The focus on annual crime rates
effects of RTC laws on crime. Under very weak assump- within states has been common in the literature.4 However,
tions, it is correct that the data cannot reveal whether RTC we do not normalize crime rates to lie in the unit interval or
laws increase or decrease crime. However, under a specific take the natural log of the crime rate, as is done in much of
set of bounded-variation assumptions that we believe the literature. Rather we are interested in directly studying
worthy of consideration, we draw substantive conclusions state-year crime rates: 100,000 times the ratio of the num-
about the qualitative and quantitative impact of RTC ber of reported crimes to the population of the state. We
laws (see section IV). Moreover, our analysis makes trans- mostly focus on crime rates in Virginia. To further illustrate
parent how these conclusions depend on the maintained the bounded-variation assumptions, we examine crime rates
assumptions. in Maryland and Illinois in an online appendix.
The paper is organized as follows: After providing a brief The crime data, obtained from the FBI’s Uniform Crime
overview of the data in section II, section III formally Reports (UCR), were originally assembled by Lott and
defines the empirical question and the selection problem Mustard and have subsequently been modified, corrected,
and then introduces invariance and bounded-variation and updated several times. Our analysis uses the iteration
assumptions. Throughout, we allow for the possibility that assembled and evaluated by Aneja et al. (2011).5 For each
the effects of RTC laws vary across years, states, and state and year, we observe crime rates separately for mur-
crimes. To keep our task manageable, we focus on drawing der, rape, assault, robbery, auto theft, burglary, and larceny.
inferences on the impact of RTC laws in a single state,
4
Some studies take the county rather than the state as the geographic
unit of analysis, and most studies condition the analysis on characteristics
3
Partial identification analysis of treatment effects from observational of the state or county (e.g., demographics, number of police per capita, or
data was initiated in Manski (1990) and has developed subsequently. median county income). It may be of interest to perform more disaggre-
Manski (2007) provides a textbook exposition. The closest methodologi- gated analyses that examine crime commission by particular categories of
cal precedent to our study is the Manski and Pepper (2000) study of person or even by individuals. However, the requisite data to support such
monotone instrumental variable assumptions. The closest applied prece- analyses are not available. Likewise, rather than evaluating the effect of
dent to our study is the Manski and Pepper (2013) analysis of the deter- an RTC statute, it may be of interest to study how the number of RTC per-
rent effect of the death penalty, which uses some simple bounded-varia- mits affect crime. However, in general, data on the number of permits or
tion assumptions to examine data from two repeated cross-sections. In the number of persons carrying concealed handguns are not available
this paper, we introduce a much broader class of bounded-variation (Lott, 2010).
5
assumptions to evaluate annual state crime rates from multiple repeated The data were downloaded from http://works.bepress.com/john_dono
cross-sections. hue/89/ in June 2012.
234 THE REVIEW OF ECONOMICS AND STATISTICS

FIGURE 1.—MURDER AND ROBBERY RATES BY YEAR AND STATE and dropping faster in the 1990s. Virginia enacted an RTC
A: Murder Rate by Year and State statute in 1989.
14

12 III. Basic Issues in Inference on Treatment Effects,

with an Illustrative Application
Rate (per 100,000)

10

8 In this section, we formally define the empirical question

Maryland
6 and the selection problem and introduce invariance and
Virginia
4
bounded-variation assumptions. We begin by defining the
average treatment effect and then assess the effect of an
2
RTC law on the 1990 murder rate in Virginia. We focus on
0 this somewhat narrow question to clearly illustrate the uti-
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
lity of bounded-variation restrictions. In particular, we
show how bounded-variation restrictions provide an intui-
B: Robbery Rate by Year and State tive and simple way to improve the credibility of empirical
500
450
research. We also illustrate the sensitivity of inferences to
400
different identifying restrictions, without arguing in favor
of any particular set of assumptions.7 In section IV, we use
Rate (per 100,000)

350
300 the observed data on crime rates to motivate a particular set
250 Maryland of bounded-variation assumptions and explore, in some
200 Virginia detail, the inferences that arise from this set of assumptions.
150
Departing from conventional practice in applied econo-
100
metric analysis, we do not refer to our empirical findings as
50
0
‘‘estimates’’ and do not provide measures of statistical pre-
cision when studying the impact of RTC laws. Instead, we
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006

perform a finite-population analysis that views states as the

units of interest rather than as realizations from some sam-
For each state-year, we observe whether an RTC statute is pling process. One reason is expositional: we want to focus
in place.6 attention on the identification problem arising from the
Figures 1A and 1B display the annual time series of mur- unobservability of counterfactual outcomes. The bounded-
der and robbery rates in Virginia and Maryland over the variation assumptions that we consider place deterministic
period 1970 to 2007. The figures reveal several interesting constraints on the values of counterfactual outcomes.
characteristics of the crime rates. First, except for murder Hence, imprecision in our findings results from the selec-
rates in a few years during the 1970s, crime rates in Mary- tion problem and is expressed through the width of the
land exceed the analogous rate in Virginia, in many cases bounds that we report rather than through any measure of
by a substantial margin. Second, the figures show well- sampling uncertainty.
known temporal patterns in crime: crime rates rose in the Considering the available data, a fundamental reason for
1980s and then declined sharply beginning in the mid- not performing statistical inference is that measurement of
1990s. Crime rates in Maryland, which did not adopt an statistical precision requires specification of a sampling pro-
RTC statute over this period, have more pronounced cess that generates the data. Yet we are unsure what type of
changes than those in Virginia, rising faster in the 1980s sampling process would be reasonable to assume in this
application. One would have to view the existing United
States as the sampling realization of a random process
defined on a superpopulation of alternative nations.8 That
6
As of 2014, all fifty states have passed laws allowing citizens to carry is, one would have to pose a random process generating
concealed firearms. Not all states, however, have RTC laws as defined in
the empirical research. The research has classified a state as having an actual American history, with its division of the country
RTC law when legal gun owners are allowed to carry concealed firearms, into states with their populations of persons, as one among
perhaps after certification and training. These are often referred to as a set of possible histories that could have generated alterna-
‘‘shall-issue’’ provisions, in that the state shall issue a permit subject to
the applicant who meets basic determinate criteria. Other states require
7
applicants to demonstrate ‘‘good cause’’ before granting a concealed- Manski (2007, chaps. 7–10) provides a textbook exposition of the
carry permit. These so-called may-issue states are not classified as having identifying power of alternative assumptions, ranging from weak assump-
RTC laws. Ten states and the District of Columbia have may-issue provi- tions yielding partial identification of average treatment effects to strong
sions, and 41 states have shall-issue provisions. A recent wave of high- ones yielding point identification. Richardson et al. (2014) also review
profile state legislation allows citizens to openly carry firearms in public, some of this literature and the related body of research on sensitivity ana-
enacting so-called open-carry laws. While not considered in this analysis, lysis, which examines the implications of varying certain unknown para-
the same identification problems creating uncertainty in the research on meters within some class of models.
8
RTC laws apply to any evaluation of the impact of open-carry laws on See Cochran (1977) and Deaton (1997), among others, for expositions
crime. of the distinction between finite-population and superpopulation analysis.
 HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES? 235

tive state-year crime rates. But what random process should from a superpopulation. Given that state populations are so
be assumed to have generated the existing United States, large, containing millions of persons, the sampling variabil-
with its realized state-year crime rates? ity of counts would be a negligible consideration except
There is no obvious answer, and the literature analyzing possibly when considering relatively rare crimes such as
RTC laws has not engaged the question. We do not argue murder.
that it would be impossible to specify a credible random In the setting of a randomized experimental design, Aba-
process generating the United States and base statistical die et al. (2014) raise similar concerns about drawing infer-
inference on this process, but doing so goes well beyond ences in state-level analyses and propose an alternative
the scope of this paper. We leave this an open question that approach for measuring uncertainty. In particular, they
may perhaps draw attention in our own future research or argue that if one observes the population, there is no sam-
that of others. pling variability, but there is statistical uncertainty if treat-
A reviewer has suggested that we compute confidence ments are assigned at random. The fact that one cannot
intervals using methods studied in the econometric litera- observe counterfactual outcomes creates randomization uncer-
ture on inference in settings with partial identification. tainty. Abadie et al. (2014) propose using randomization-
Existing methods for computing such confidence intervals based standard errors to express such uncertainty. They find
assume that the data are a random sample drawn from an that these standard errors are generally smaller than the
infinite population (see, e.g., Imbens & Manski, 2004; conventional sampling-based standard errors made under a
Chernozhukov, Hong, & Tamer, 2007; Chernozhukov, Lee, random sampling assumption.
& Rosen, 2013; and Kaido, Molinari, & Stoye, 2016). Like- This approach provides an alternative way to conceptua-
wise, random sampling assumptions are implicit in the lize uncertainty in cases where one observes a population
inferential methods used to estimate the standard errors of rather than a sample. The proposed randomized-based stan-
parameter estimates in the linear panel data models preva- dard errors, however, apply only when treatments are ran-
lent in the existing literature on RTC laws.9 Random sam- domly assigned, in which case the average treatment effect
pling assumptions, however, are not natural when consider- is point identified. Their approach is not applicable in
ing states or counties as units of observation. observational settings where the treatment, such as RTC
Similar concerns about the sampling process arise if one laws, may be endogenous.
takes the unit of analysis to be an individual person and
considers state crime rates to aggregate individual crime A. Treatment Effects and the Selection Problem
counts across the population of a state. To draw inferences
in this setting, one must have prior information on the sam- Consider the problem of inferring the average treatment
pling process generating the individuals who populate a effect (ATE) of an RTC statute on the rate of commission
state. Random sampling from an infinite population, the of a specified crime in a specified year in a group of states
underlying assumption used in the partial identification lit- that share specified observed covariates:
erature, applies if one is willing to assume that the realized ATEdx ¼ E½Yd ð1ÞjX  E½Yd ð0ÞjX: (1)
population of a state in a given year is a random sample of
persons drawn from a superpopulation of potential state Outcome Yd(1) denotes the crime rate if a state were to have
residents and that crimes are committed independently by an RTC statute in year d, Yd(0) denotes the analogous out-
members of the population. However, this random sampling come if the state were not to have an RTC statute, X denotes
assumption may not be credible because commission of the specified covariates, and d indicates the specified year.
crimes may involve complex social interactions that are not To define the ATE, we consider the fifty states plus
independent across individuals or over time. Moreover, it Washington, DC, to be the population and use the expecta-
may not be credible to suppose that the observed population tion operator E[|X] to denote an average across states with
of a state in a given year is a random sample from some covariates X. The unit of observation being a state, the aver-
superpopulation. age treatment effect measures how the crime rate would dif-
Suppose that despite these concerns, one is willing to fer if all states with covariates X were to have an RTC sta-
assume that a state population is a random sample drawn tute in year d versus the rate if all such states did not have
an RTC law. The ATE can vary with d and X. We follow
9
the literature by assuming that treatment response is indivi-
There is an ongoing debate about the appropriate method of estimating
standard errors for the linear models used in the RTC literature (see Aneja dualistic; that is, an RTC law in state j may affect crime in
et al., 2011, and National Research Council, 2005). Many researchers state j but not elsewhere. Thus, there are no spillover effects
report standard errors that allow for arbitrary correlation within a state or to other states.
county—so-called state/county clustered standard errors—while others do
not allow for such correlations. The NRC report explains that these clus- For each state j and year d, there are two potential out-
tered sampling standard errors are inappropriate in the standard linear comes, Yjd(1) and Yjd(0). The former outcome is counterfac-
panel data models (with fixed county effects) used in the literature. Our tual if state j did not have an RTC statute in year d, while
concerns are distinct from these issues. We find no basis for reporting
standard errors using any of the conventional methods, allowing for clus- the latter is counterfactual if the state did have an RTC sta-
tering or not. tute. The fact that the data reveal only one of the two
236 THE REVIEW OF ECONOMICS AND STATISTICS

mutually exclusive outcomes constitutes the selection prob- TABLE 1.—MURDER RATES PER 100,000 RESIDENTS BY YEAR AND STATE
lem. The observed crime rate is Yjd(Zjd) ¼ Yjd(1) Zjd þ Year Maryland Virginia
Yjd(0) (1  Zjd), where Zjd ¼ 1 if state j has an RTC statute 1988 9.63 7.75
in year d and Zjd ¼ 0 otherwise. We henceforth write 1990 11.55 8.81
Yjd ¼ Yjd(Zjd) for short.

Given certain invariance assumptions, each finding

B. Invariance Assumptions
appropriately measures the effect of the RTC law on the
The conventional practice used to address the selection 1990 murder rate in Virginia. The one assuming time invar-
problem has been to invoke assumptions that are strong iance is correct under the assumption that no determinant of
enough to point-identity counterfactual mean outcomes and, criminal behavior changed in Virginia between 1988 and
hence, the average treatment effect. These assumptions typi- 1990 except for enactment of the RTC statute. Interstate
cally assert invariance of some kind. One might, for exam- invariance of the 1990 murder rates is correct under the
ple, assume that the ATE is constant across X or d, or both. assumption that the populations of Virginia and Maryland
To illustrate in a simple setting, consider inference on the had the same propensities for criminal behavior and faced
impact of adoption of an RTC law on the 1990 murder rate the same environments except for the presence of the RTC
in Virginia (VA). Virginia enacted an RTC statute in 1989. statute in Virginia. The difference-in-difference (DID) find-
In this case, d ¼ 1990, X ¼ VA, and ZVA,1990 ¼ 1. There ing is correct under the assumption that in the absence of
exists only one state with X ¼ VA, so P(Z1990 ¼ 1|X ¼ RTC statutes, Virginia and Maryland would have experi-
VA) ¼ 1 and E[Y1990(t) | X ¼ VA] ¼ Y1990(t). Hence, enced the same change in murder rates between 1988 and
equation (1) reduces to 1990. Thus, each of the three findings can be justified by
specific invariance assumptions. However, the variation in
ATE1990;VA ¼ E½Y1990 ð1ÞjX ¼ VA  E½Y1990 ð0ÞjX ¼ VA empirical findings shows that these invariance assumptions
¼ YVA;1990 ð1Þ  YVA;1990 ð0Þ: (2) cannot hold jointly. Indeed, it may be that none of the
assumptions holds. Thus, the TE may equal none of the
The available data reveal that YVA,1990(1) ¼ YVA,1990, but values 1.06, 2.74, and 0.86.
they do not reveal the counterfactual quantity YVA,1990(0). The literature evaluating RTC laws has analyzed crime
Throughout this paper, we focus attention on treatment data across many states and years rather than two states and
response in a single state, such as Virginia, rather than on a two years. Having more data, however, does not reduce the
group of states. This being so, we henceforth omit the dependence of empirical findings on the assumptions that
expectation notation. We also omit the adjective average researchers maintain. It has been common, for example, to
when discussing treatment effects and use the abbreviation assume a linear model with a homogeneous treatment effect
TEjd to denote the treatment effect in state j in year d. and state-year fixed effects.10 This model has the form
Table 1 displays the murder rates per 100,000 residents
in Virginia and Maryland in 1988 and 1990. Neither state Yjd ðtÞ ¼ h  t þ Xjd b þ aj þ cd þ ejd : (6)
had an RTC statute in 1988. Virginia enacted one in 1989,
Here treatment t ¼ 1 denotes the presence of an RTC law
but Maryland did not. Thus, YVA,1990(1) ¼ YVA,1990 ¼ 8.81.
and t ¼ 0 otherwise. The parameter y is the treatment
These data may be used to compute three simple but
effect, which does not vary with j and d. Thus, the model
different findings on the counterfactual 1990 murder rate
assumes that right-to-carry laws have the same effect on
in Virginia, YVA,1990(0), under alternative invariance
crime rates, y, in all states and years.11 Model (6) permits
assumptions:
variation in crime rates across states and years only through
Time invairiance : YVA;1990 ð0Þ ¼ YVA;1988 ð0Þ ¼ YVA;1988 the composite additive intercept Xjdb þ aj þ gd þ ejd.
Here b is a parameter vector, while aj and gd are state and
¼ 7:75 (3)
year fixed effects. The unobserved variable ejd is a random
state-year interaction assumed to have mean 0 conditional
Interstate invariance : Y VA;1990 ð0Þ ¼ Y MD;1990 ð0Þ
on each realized value of t and X.
¼ YMD;1990 ¼ 11:55 (4)
10
Difference-in-difference invariance : Y VA;1990 ð0Þ The existing research on RTC laws does not formally model the
selection process by which states adopt RTC statutes or, in general, apply
¼ ½YMD;1990 ð0Þ  YMD;1988 ð0Þ þ YVA;1988 ð0Þ an invariance assumption that the response functions are mean indepen-

dent of observed instrumental variables. Notable exceptions are Lott and
¼ YMD;1990  YMD;1988 þ YVA;1988 ¼ 9:67 (5) Mustard (1997) and Lott (2010), who present results under a variety of
instrumental variable assumptions.
11
These, in turn, imply treatment effects of 1.06, 2.74, and One may allow the treatment effect y to vary across states and years
if one assumes that passage of RTC laws is exogenous—that is, statisti-
0.86 respectively. Thus, the three empirical findings differ cally unrelated to the state-date specific treatment effect. However,
in direction and magnitude. researchers typically do not want to make this assumption.
 HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES? 237

This model relies on the invariance assumption that the not formalize what this means. Instead, they perform ana-
effect of RTC laws, y, is the same for all states and all lyses that use some invariance assumptions as if they are
years. While this homogeneity assumption is convenient truth and that entirely dismiss other assumptions. For exam-
and has substantial identifying power when combined with ple, DID estimation maintains assumption (5) but places no
certain other assumptions, there is little support for the restrictions on response levels.
notion that the effects of RTC laws are identical across A simple way to improve the credibility of empirical
states and time. In fact, the empirical literature provides research is to weaken invariance assumptions to assump-
some evidence to the contrary. Some researchers have con- tions of bounded variation. In this paper, we report empiri-
cluded that the effects of RTC laws vary over time and cal findings under bounded-variation assumptions that
across states (see Black & Nagin, 1988; National Research weaken assumptions (3), (4), and (5) for a specified treat-
Council, 2005; Aneja et al., 2011; Durlauf et al., 2016). ment t as follows:
The Virginia and Maryland crime rates displayed in fig-
ure 1 provide direct evidence that the invariance assump- Bounded time variation : Yjd ðtÞ  Yje ðtÞ  djðdeÞ (7)
tions in equations (3), (4), and (5) do not hold across all
years and states. Figure 1 displays the annual murder rates Bounded interstate variation : Yjd ðtÞ  Ykd ðtÞ  dð jkÞd
in Maryland and Virginia from 1970 to 2007. While neither (8)
state had an RTC statute prior to 1989, crime rates vary
over time and across states from 1970 to 1988. For exam-
ple, in 1988, the murder rate in Virginia was 1.89 less than Bounded DID variation :
 
the rate in Maryland in that year and 0.39 greater than the Yjd ðtÞ  Yje ðtÞ  ½Ykd ðtÞ  Yke ðtÞ  dð jkÞðdeÞ ; (9)
Virginia rate in 1987.12 Thus, the invariance assumptions
are violated in the years before Virginia adopted an RTC where ( j, k) are specified states, (d, e) are specified years,
statute, and the signs and magnitudes of these violations and (dj(de), d( jk)d, d( jk)(de)) are specified positive constants.
differ over time and across assumptions. Manski and Pepper (2013) observed that the bounds on
While all three of these invariance assumptions are counterfactual quantities implied by such bounded-variation
rejected in the pre-1989 period, the existing literature on assumptions are solutions to linear programming problems
the effects of RTC laws on crime consistently applies these and, hence, are relatively easy to compute. (See also Laf-
types of restrictions, especially variations of the DID model férs, 2015.)
in equation (5) or the related model in equation (6). In this To simplify the notation, we suppress the dependence of
setting, researchers implicitly assume that the invariance the d values on the treatment t. We also suppress for simpli-
restrictions apply when outcomes are counterfactual even city the logical requirement that crime rates must take non-
though they are rejected in periods where outcomes are negative values. This requirement does not bind in the
observed. This is hard to motivate, and the literature using empirical analyses that we report.
these invariance assumptions fails to do so. Likewise, the The bounded-time variation assumption restricts the
variety of alternative but related difference-in-difference absolute difference in treatment response in state j between
type approaches that point-identify the average treatment two years, say, 1988 and 1990, to be less than dj(de). Letting
effect in data from repeated cross sections—for example, dj(de)  0 weakens the traditional time invariance assump-
the propensity score and the synthetic control (see Abadie, tion by supposing that response may differ over time by no
Diamond, & Hainmueller, 2010) approaches—rely on more than dj(de). The larger the selected value of dj(de), the
invariance assumptions that are difficult to motivate in this weaker the assumption. Similarly, the bounded interstate
application. variation assumption restricts the contemporaneous abso-
lute difference at year d in response between two states,
say, Virginia and Maryland, to be less than d( jk)d. Letting
C. Bounded-Variation Assumptions d( jk)d  0 weakens the traditional invariance assumption by
supposing that response across the two states may differ by
The Assumptions. Invariance assumptions have a sharp- no more than d( jk)d. The larger the selected value of d( jk)d,
ness that often makes them suspect. Consider each of the the weaker the assumption.
three equalities (3), (4), and (5) that point-identify the These bounded-variation assumptions have identifying
impact of the RCT law in Virginia in 1990. Why should power because they imply that counterfactual state-year
any of them hold exactly? Why should the treatment effect crime rates are similar to observed rates in other states
be exactly constant across states and years as assumed in and years. The degree of similarity is determined by the
the linear model, equation (6)? Empirical researchers often bound parameters (dj(de), d( jk)d, d( jk)(de)). These assumptions
say that such assumptions are approximations, but they do resemble the smoothing assumptions made in kernel non-
12
parametric regression analysis, with d acting as the band-
Also notice that the time invariance assumption does not hold in
Maryland, where the murder rate increased from 9.63 in 1988 to 11.55 in width (aka tuning parameter). A fundamental difference is
1990. that the asymptotic theory for nonparametric regression
238 THE REVIEW OF ECONOMICS AND STATISTICS

FIGURE 2.—BOUNDS ON THE TREATMENT EFFECTS AS A FUNCTION OF d Bounded-variation assumptions can be easily adapted to
8 fit particular features of the application. For example, figure
6
1 reveals that crime rates in Maryland generally exceed the
4
2
contemporaneous crime rates in Virginia, even in the years
0 UB Time Invariance before 1989 when neither state had adopted an RTC statute.
TE

-2 LB Time Invariance One might therefore think it reasonable to make the

-4 UB Interstate Invariance bounded-variation assumption one sided, that is,
-6
LB Interstate Invariance
-8 0  YMD;1990 ð0Þ  YVA;1990 ð0Þ  dðMD;VAÞ1990 : (10)
-10
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
δ
Assumption (10) makes the lower bound on the TE equal
2.74, the finding under invariance assumption (4). The
makes smoothing go to 0 with sample size, so smoothing upper bound is identical to the one displayed in figure 2.
is performed to achieve desired asymptotic statistical prop- Finally, letting d( jk)(de)  0 weakens the traditional
erties rather than identification. In contrast, we impose
DID invariance assumption. The traditional assumption
assumptions that relate counterfactual quantities to observed (d( jk)(de) ¼ 0) point-identifies the TE, revealing that the
ones and thus provide identifying power. RTC statute decreases the mean murder rate by 0.86.
It is of interest to compare the bounded-variation
Ambiguity about the TE increases with d( jk)(de), and for
assumptions to those made in the linear homogeneous values in excess of 1, it is impossible to sign the TE.
model. These assumptions are not nested. The bounded-
variation assumptions are weaker in two respects. First,
they permit the treatment effect to vary across states and D. Joint Bounded Geographic and
years, whereas model (6) restricts the treatment effect to Time Variation Assumptions
have the constant value y across states and years. Second, Rather than use a single invariance assumption in isola-
assumptions (7) to (9) do not impose a condition on mean tion, one may want to combine assumptions. For example,
variation in treatment response akin to the restriction of one might simultaneously assume that Virginia and Mary-
model (6) that ejd has mean 0 conditional on each realized land are similar to one another and that Virginia had similar
value of t and X. On the other hand, assumptions (7) to characteristics in 1988 and 1990. Here we evaluate bounds
(9) restrict the potential magnitudes of time, interstate, on TE1990,VA under joint interstate and time-bounded varia-
and DID variation in outcomes under a given treatment, tion assumptions.
whereas model (6) does not. Under a joint interstate and time variation assumption,
the counterfactual murder rate YVA,1990(0) is bounded as
Findings for Virginia in 1990. Figure 2 shows bounds follows:
on TE1990,VA for different values of dj(de), where d ¼ 1990,

e ¼ 1988, and j ¼ VA. Our objective is to illustrate the max YMD;1990  dðMD;VAÞ1990 ; YVA;1988  dVAð1990;1988Þ
sensitivity of inferences to different identifying restrictions  YVA;1990 ð0Þ 
without arguing in favor of any particular set of assump-

tions. The traditional time-invariance assumption (dj(de) ¼ min YMD;1990 þ dðMD;VAÞ1990; YVA:1988 þ dVA ð1990;1988Þ :
0) point-identifies the TE, revealing that enactment of the (11)
RTC statute increases the murder rate by 1.06. Ambiguity
about the TE increases with dj(de). The figure shows that the A necessary condition for this assumption to be valid is
bound on the TE is entirely positive when dj(de) is less than that dVA(1990,1988) þ d(MD,VA)1990  |YMD,1990  YVA,1988|.
1, but any value of dj(de) larger than 1 renders it impossible Otherwise, the lower bound exceeds the upper bound.
to sign the TE. For example, when dj(de) ¼ 2, we know that Table 2 displays the upper and lower bound on the TE
YVA,1990(1) ¼ 8.81 and YVA,1990(0) [ [5.75, 9.75]. Hence, under joint bounded interstate and time variation assump-
TE1990,VA [ [0.94, 3.06]. tions. Findings highlighted in dark gray are not valid; the
Figure 2 also shows bounds on TE1990,VA for different lower bound exceeds the upper bounds. Findings in light
values of d( jk)d, where k ¼ Maryland. The traditional inter- gray identify the sign of the treatment effect, with those in
state invariance assumption (d( jk)d ¼ 0) point-identifies the bold as negative and those in italicized text as positive.
TE, revealing that the RTC statute decreases the murder Findings that are not highlighted are informative but do not
rate by 2.74. Ambiguity about the TE increases with reveal whether adoption of the RTC law in 1989 increased
d( jk)d. The figure shows that the bound is entirely negative or decreased the 1990 murder rate in Virginia.
when d( jk)d < 2.74, but values in excess of 2.74 make it There are several interesting findings. First, the necessary
impossible to sign the TE. For example, when d( jk)d ¼ 3, condition for validity of assumptions rules out a range of
we know that YVA,1990(1) ¼ 8.81 and YVA,1990(0) [ [8.55, small values of dVA(1990,1988) and d(MD,VA)1990 as infeasible.
14.55]. Hence, TE1990,VA [ [5.74, 0.26]. Thus, it cannot jointly be true that (Virginia, Maryland) are
 HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES? 239

To simplify the presentation, the murder rates reported in this table are rounded to the nearest whole number (except for the 1990 murder rate in Virginia). Bounds highlighted in dark gray are infeasible. Bounds highlighted in light gray and bold identify the sign of the TE to
highly similar in 1990 and that (1988, 1990) are highly

UB
0.8
1.3
1.8
2.3
2.8
2.8
2.8
2.8
2.8
2.8
2.8
2.8
2.8
similar years in Virginia.

0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
Second, the assumptions point-identify the TE for a variety

LB
6 of feasible values (i.e., dVA(1990,1988) þ d(MD,VA)1990 ¼ 4),
UB
0.8
1.3
1.8
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
and they identify the sign of the TE for others. For example,
0.8 when dVA(1990,1988) ¼ d(MD,VA)1990 ¼ 2, the TE is identified
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
LB
5.5

to equal 1.2. When dva(1990,1988) ¼ 0.5 and d(MD,VA)1988 ¼

3.5, the TE is identified to equal 0.3. The sign of the TE is
UB
0.8
1.3
1.8
1.8
1.8
1.8
1.8
1.8
1.8
1.8
1.8
1.8
1.8
identified to be negative for all feasible d(MD,VA)1990  3,
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
and greater than 0 for all feasible dVA(1990,1988)  0.5. For
LB
5

larger values of dVA(1990,1988) and d(MD,VA)1990, the assump-

UB
0.8
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
tions do not identify the sign of the TE.
TABLE 2.—BOUNDS ON THE TREATMENT EFFECT GIVEN BOUNDED INTERSTATE AND TIME VARIATION ASSUMPTIONS, VIRGINIA, 1990

To summarize, in this section, we have examined the sen-

0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
LB
4.5

0.8
0.3

sitivity of inferences to different bounded-variation assump-

tions, focusing on the effect of the 1989 adoption of an
UB
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
RTC law on the 1990 murder rate in Virginia. We find that
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2

the sign of the TE is identified for some assumptions but

LB
4

not others. Especially interesting findings emerge when dif-

UB
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3

ferent bounded-variation assumptions are combined. In par-

ticular, the strongest assumptions are ruled out, including
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
LB
3.5

the traditional strict invariance assumptions. Less strong

assumptions either point-identify the TE or identify the sign
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
UB

of the TE, while weaker ones do not identify whether adop-

tion of an RTC law increased or decreased murder in Virgi-
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
LB
d(VA,MD)1990
3

nia in 1990.
The type of sensitivity analysis performed here,
0.7
0.7
0.7
0.7
0.7
0.7
0.7
0.7
0.7
0.7
0.7
0.7
0.7
UB

which fully maps out the ambiguity resulting from the

selection problem, may prompt one to seek a purely data-
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
LB
2.5

driven approach to determine the validity of different

assumptions—a ‘‘stopping rule.’’ In general, however, no
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
UB

such rule exists: the data alone cannot identify the counter-
factual outcomes.
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
5.2
LB
2

Alternatively, one might rely on an understanding of the

institutional setting, the historical record, theory, or other
1.7
1.7
1.7
1.7
1.7
1.7
1.7
1.7
1.7
1.7
1.7
1.7
1.7
UB

sources of information to develop a set of credible assump-

tions on treatment response. For example, in most markets,
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.7
4.7
LB
1.5

economists are comfortable assuming demand is downward

sloping even though the market data cannot reveal the
2.2
2.2
2.2
2.2
2.2
2.2
2.2
2.2
2.2
2.2
2.2
2.2
2.2
UB

demand function (see Manski, 1997). When evaluating the

impact of RTC laws, finding common ground based on the-
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
4.2
4.2
4.2
LB
1

ory is difficult. Instead, in section IV, we use data on crime

be negative, and those in light gray and italics are identified to be positive.

rates prior to Virginia’s adoption of an RTC law to provide

2.7
2.7
2.7
2.7
2.7
2.7
2.7
2.7
2.7
2.7
2.7
2.7
2.7
UB

guidance on what we believe to be a set of bounded-

variation assumptions worthy of consideration.
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.7
3.7
3.7
3.7
LB
0.5

3.2
3.2
3.2
3.2
3.2
3.2
3.2
3.2
3.2
3.2
3.2
3.2
3.2
UB

IV. Evaluating the Impact of RTC Laws Using

Bounded-Variation Assumptions
0.8
0.3
0.2
0.7
1.2
1.7
2.2
2.7
3.2
3.2
3.2
3.2
3.2
LB
0

Whereas section III assessed the sensitivity of inference

to different assumptions, in this section we evaluate the
0.5

1.5

2.5

3.5

4.5

5.5
0

impact of RTC laws under particular bounded-variation

assumptions. We begin in section IVA by developing an
dVA(1990,1988)

approach for selecting the parameters d based on observed

data. Then, using these parameters, we present findings for
Virginia in section IVB and for Maryland and Illinois in an
online appendix. The latter states did not adopt RTC sta-
240 THE REVIEW OF ECONOMICS AND STATISTICS

TABLE 3.—BOUNDED-VARIATION PARAMETERS 1972, and so forth), and then select the maximum
Interstate Time DID DID_0.75 of these values. Table 3 displays the minimum parameters
Murder 2.7 2.0 2.3 1.2
that ensure the bounded-variation assumptions are consis-
Rape 17 4 6 3 tent with the observed pre-1989 data on various crime rates.
Aggravated assault 324 39 60 34 For murder, the interstate variation parameter is at least 2.7,
Robbery 292 30 55 27
Auto theft 371 46 61 43 the time variation parameter needs to be at least 2.0, and
Burglary 496 206 136 67 the DID variation parameter at least 2.3. For robbery, the
Larceny 775 410 175 93 analogous parameters are 292, 30, and 55.13
These parameters are found by taking the maximum of the minimum parameter value for the models
to be consistent with the data in each year prior to 1989. The DID_0.75 parameter is the 0.75 quantile.
We use the parameter values displayed in table 3 as a
starting point for implementing the bounded-variation
assumptions in equations (7), (8), and (9). Of course,
tutes prior to 2007. In section IVD, we discuss and summar- whether these particular assumptions are valid is unknown,
ize the results. and one must invariably make judgments about the trade-
off between the strength and credibility of assumptions and
A. Selecting the Bound Parameters findings. Using these parameters as an anchor, one might
assess the sensitivity of findings by using some multiple of
In section III, we assessed how inferences vary across the maximum value or some quantile of the distribution.
different values of the bound parameters d. In this section, Table 3, for example, shows the 0.75 quantile of the mini-
we use observed data in Virginia and Maryland to develop mum valid parameters for the DID assumption (DID_0.75).
sensible data-based parameter values. In particular, we use In this case, the DID parameter for murder falls from 2.3 to
data prior to 1989, when Virginia did not have an RTC sta- 1.2, and for robbery it falls from 55 to 27.
tute, to determine the minimum parameter values that
would be required to make bounded-variation assumptions
consistent with the observed data. While this approach does
B. The Impact of a RTC Statute in Virginia
not ensure the validity of the bounded-variation assump-
tions, it provides what we believe to be a reasonable starting In this section, we use bounded-variation assumptions
point for our analysis. with the parameter values displayed in table 3 as a starting
To illustrate the idea, consider the murder rates for Mary- point for drawing inferences on the impact of an RTC law
land and Virginia in 1987 and 1988. The observed murder on crime in VA. Tables 4 and 5 display findings on the TE
rates vary over time and across states: the 1988 murder rate of an RTC statute in Virginia by year from 1990 to 2006
in Virginia is 7.75, and the rate in Maryland is 9.63, while and for seven different crimes. Each set of results applies a
the analogous rates for 1987 are 7.36 and 9.55, respectively. different assumption. Recall, as discussed in section III, that
Neither state had an RTC statute in these years, so these we do not report measures of sampling variability. Valid
data can be used to test whether invariance and bounded- standard errors reflecting uncertainty due to the sampling
variation assumptions are valid. In fact, the data are process would likely have an impact on some of the qualita-
inconsistent with the time invariance assumption (3), the tive conclusions about the effect of RTC laws on crime,
interstate invariance assumption (4), and the DID assump-
tion (5). 13
There are many other ways to generate plausible restrictions on the
While the invariance assumptions are inconsistent with bounded parameters using the pre-1989 data. For example, rather than
finding parameters based on the observed absolute differences, one could
the observed pre-1989 data, the bound parameters in compute separate bounds on the maximum positive difference and the
assumptions (7), (8), and (9) can be chosen to ensure inter- maximum negative difference. Assumption (10), where the difference in
nal consistency. The bounded interstate variation assump- the crime rates between Maryland and Virginia is assumed to be nonnega-
tive, is a variant on this idea. This approach weakly narrows one side of
tion in equation (8), for example, is valid in 1988 if the bounded parameters and thus will generally give tighter bounds on the
d(VA,MD)1988  1.88 (¼ 9.63  7.75). Likewise, the bounded average treatment effect. An alternative approach would be to compare
time variation assumption is consistent with the Virginia observed rates in both adjacent and nonadjacent years. This will weakly
increase the bound parameters, and thus generally give wider bounds on
1988 murder rate if dVA(1988,1987)  0.39 (¼ 7.75  7.36), the average treatment effect. For example, if we compare crime rates
and the DID assumption is valid if d(VA,MD)(1988,1987) separated by two years, the difference-in-difference parameter for murder
 0.31. These restrictions on the bound parameters make the remains at 2.3, but the parameter on robbery increases from 55 to 75. If
we compare crime rates separated by ten years, the parameter for murder
bounded-variation assumptions consistent with the 1987 and increases to 3, and the parameter on robbery increases to 79.
1988 data. Moreover, one can refine or expand the analysis to introduce additional
Extending this approach to perform comparisons bounded-variation restrictions and refined measures of d. For example,
one might include other states or evaluate county crime rates. Likewise,
from 1970 to 1988, we find the minimum values of the one might consider the crime rate in Maryland after excluding Baltimore.
bound parameters required for the assumptions to be consis- These and other variations would allow one to apply additional moment
tent with the data prior to 1989. To do this, we compute the restrictions on the counterfactual outcomes and develop refined measures
of closeness, d. For this paper, we focus our analysis on two states, Virgi-
minimum valid parameter value for each pair of nia and Maryland, over multiple years. This focus keeps our task manage-
adjacent years from 1970 to 1988 (1970 to 1971, 1971 to able and results in transparent findings.
 HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES? 241

TABLE 4.—THE TE OF AN RTC LAW IN VIRGINIA GIVEN THE DID INVARIANCE MODEL, BY YEAR AND CRIME TYPE
Murder Rape Assault Robbery Auto Theft Burglary Larceny
1990 0.9 4.7 3.9 52.0 10.9 26.9 246.3
1991 0.6 5.9 11.1 81.1 22.1 12.9 45.4
1992 1.4 4.9 10.0 103.1 51.4 61.8 165.3
1993 2.6 1.9 11.5 105.4 29.3 105.4 206.9
1994 1.0 2.2 0.7 82.3 115.8 45.2 304.4
1995 2.4 5.1 9.5 104.0 56.3 102.4 499.7
1996 2.2 0.8 1.1 83.0 63.8 44.0 373.8
1997 0.7 1.5 28.6 23.7 49.0 7.1 285.8
1998 1.9 3.3 38.9 4.7 91.0 0.5 296.4
1999 1.4 5.1 48.6 26.1 139.7 2.0 222.5
2000 0.6 3.8 22.5 21.6 82.7 47.9 381.7
2001 1.3 7.7 23.5 32.4 36.8 28.7 231.6
2002 2.2 10.1 19.4 37.3 1.8 66.6 174.8
2003 2.0 9.9 33.9 37.6 48.2 53.9 65.8
2004 2.3 10.6 20.9 50.4 43.6 84.8 18.9
2005 2.0 10.7 45.8 29.8 29.2 112.4 39.1
2006 2.6 12.5 65.8 33.5 19.6 111.3 109.3
Treatment effects in bold identify the sign to be negative and in italics text are identified to be positive.

TABLE 5.—BOUNDS ON THE TE OF AN RTC LAW IN VIRGINIA GIVEN THE JOINT DID, INTERSTATE, AND
TIME-BOUNDED VARIATION MODELS (ASSUMPTIONS A, B, AND C), BY YEAR AND CRIME TYPE
Murder Rape Assault Robbery Auto Theft Burglary Larceny
LB UB LB UB LB UB LB UB LB UB LB UB LB UB
1990 2.0 0.0 7.9 1.4 38.0 13.1 79.2 24.7 53.6 10.9 94.3 81.9 153.2 291.6
1991 1.7 0.3 9.2 2.7 23.0 28.1 108.3 53.8 64.9 22.1 80.3 29.5 47.8 138.6
1992 2.6 0.6 8.2 1.7 44.1 7.0 130.4 75.9 94.1 51.4 129.2 104.5 258.5 72.1
1993 3.7 1.7 5.2 1.4 45.6 5.5 132.7 78.2 72.0 29.3 172.8 142.4 300.1 113.7
1994 2.1 0.2 5.4 1.1 34.8 16.3 109.6 55.1 158.6 115.8 112.6 172.5 397.6 211.3
1995 3.5 1.5 8.3 1.8 43.6 7.5 131.2 76.7 99.0 56.3 169.8 216.2 592.9 406.5
1996 3.4 1.4 4.1 0.1 33.0 18.1 110.2 55.7 106.5 63.8 111.4 223.9 467.0 280.7
1997 1.9 0.1 1.8 4.7 5.4 45.6 51.0 3.6 6.2 49.0 74.5 60.3 379.0 192.6
1998 3.0 1.1 0.1 6.6 4.8 55.9 32.0 22.5 48.2 91.0 67.9 67.0 389.6 203.3
1999 2.0 0.6 1.9 8.4 32.1 65.7 1.2 53.3 97.0 139.7 69.5 65.4 315.7 129.4
2000 1.7 0.3 0.5 7.0 13.5 5.5 5.7 48.8 40.0 82.7 19.6 115.3 474.8 288.5
2001 2.5 0.5 4.5 11.0 15.4 6.5 5.2 59.7 6.0 36.8 38.7 96.1 324.8 138.5
2002 2.4 1.4 6.9 13.4 15.0 2.3 10.1 64.6 44.5 1.8 0.8 134.0 267.9 81.6
2003 2.1 1.2 6.6 13.1 6.0 50.9 10.4 64.9 55.7 48.2 13.6 121.3 158.9 27.4
2004 2.5 1.5 7.4 13.9 3.3 38.0 23.2 77.7 65.7 43.6 17.4 152.2 74.2 112.1
2005 1.7 1.1 7.4 14.0 11.7 62.8 2.6 57.1 71.9 29.2 45.0 179.8 54.1 132.2
2006 2.5 1.8 9.2 15.8 31.8 82.9 6.2 60.7 23.1 19.6 43.9 178.7 202.4 16.1
Bounds highlighted in dark gray are infeasible. Bounds highlighted in light gray and bold identify the sign of the TE to be negative, and those in light gray and italics are identified to be positive.

especially for highlighted findings that are close to 0 and TE. To see this, add and subtract the DID parameters in
for rare crimes such as murder. table 3 to the findings in table 4. For example, the impact of
Table 4 presents results for all seven crimes and all years RTC laws on the 2000 murder rate is bounded between
under the DID invariance assumption. We use 1988 as the [0.6  2.3, 0.6 þ 2.3] ¼ [2.9, 1.7]. The bound on the
anchor year for the DID invariance assumption. Findings in impact on the 2005 burglary rate is [24, 248].
bold identify the TE to be negative while those in italics are The results from these two DID assumptions provide a
positive. For murder, the DID assumption findings are stark illustration of the trade-off between the strength of
strictly negative. For the other crimes, the findings are nega- assumptions and conclusions. The strong invariance form
tive in earlier years but tend to be positive in later years. of the DID assumption point-identifies the TE, but the
For example, the RTC statute is found to have increased the weaker bounded-variation form of this assumption gener-
2005 burglary rate by 112.4. ally does not identify even the sign of the TE.
These results, however, rely on a strict invariance Partial identification analysis jointly using variations of
assumption. If we use the parameter values in table 3 to assumptions (7), (8), and (9) allows us to bridge the gap
relax this invariance restriction to a bounded-variation DID between these two extremes by considering middle-ground
assumption, nearly all of the bounds except those for lar- assumptions that yield information on the sign of the TE.
ceny include 0. That is, under the bounded DID variation To do this, we continue to apply DID bounded-variation
assumption, we cannot generally identify the sign of the assumptions and also use bounded time and interstate varia-
242 THE REVIEW OF ECONOMICS AND STATISTICS

tion assumptions that seem well suited for this application. upper bound. Findings that are not highlighted do not iden-
In particular, we impose the following three assumptions: tify the sign of the TE. The results vary by crime and year.
We find that the RTC law in Virginia reduced murder and
A. j½Y VA,d (0)  Y VA,e (0)  ½Y MD,d (0)  Y MD,e (0)j larceny rates in nearly every year but increased assaults
 d(VA,MD)(de), where d(VA,MD)(de) ¼ DID_0.75 in after 1997. For other crimes, the sign of the bounds is gen-
table 3 for d [ [1990, 2006] and e ¼ 1988 erally negative in the 1990s but positive or indeterminate in
B. 0  Y M D , d (0)  Y V A , d (0)  d ( M D , V A ) d , where the 2000s.
d(MD,VA)d ¼ interstate in table 3 for d [ [1990, 2006] Interestingly, these results are generally inconsistent with
C. Y V A , d (0)  Y V A , 8 8 (0) for d [ [1990, 1996]; the standard assumption that the TE does not vary over time
YVA,d(0)  YVA,88(0) for d [ [1999, 2006] (see model [6]). To see this, note that under the assumption
that the TE is the same in every year, one can take the inter-
Assumption A is the DID bounded-variation assumption— section of the bounds from each year to derive a refined
assumption (9)—where the bound parameter equals the tighter bound. This is an instrumental variable bound, as
0.75 quantile of the distribution of the minimum bound defined in Manski (1990). Yet the intersection bounds are
parameters required for the assumptions to be consistent empty for all seven crimes; the lower bound exceeds the
with the data in each year prior to 1989 (see the DID_0.75 upper bound. For example, the lower and upper intersection
column in table 3). This assumption relaxes the strict bounds for robbery are 23.2 and 78.2. Thus, this bounded-
invariance assumption where d ¼ 0 but strengthens the variation assumption is inconsistent with the homogeneous
bounded-variation assumption using the DID parameters in treatment effect assumption in model (6).
table 3. Assumption B is the one-sided bounded interstate
variation assumption—assumption (10)—which restricts D. Discussion of Results
the crime rate in Virginia to be no greater than the crime
rate in Maryland. Finally, assumption C is a bounded time Our findings about the impact of RTC laws on crime are
variation assumption that operationalizes the idea that crime nuanced and not amenable to a simple punch-line conclu-
rates rose in the early 1990s and fell in the 2000s.14 sion. Inferences are sensitive to assumptions. Under the
We do not apply the bounded time variation assumption weakest assumptions, we cannot identify whether RTC laws
(7). This assumption is not flexible enough to account for increase or decrease crime. Under stronger assumptions, we
the large temporal changes in crime rates over 1970 to find that adoption of the RTC statute in Virginia has
2006. To identify the counterfactual crime rate that would reduced some crimes (e.g., murder), increased others, and
have occurred had Virginia not adopted a RTC law, the had ambiguous sign effects on others. For some crimes, the
time variation assumption in equation (7) adds and subtracts findings vary across years, indicating that the RTC law
dVA(d,88) to the observed crime rate in 1988. This bound reduced the crime rate in the 1990s but increased it in the
on the counterfactual crime rate does not depend on the 2000s.
year under consideration, d—the bound is the same for When the signs of the TE are identified for Virginia, the
1989 and 2006—even though there were significant magnitudes are generally large. For example, using the
changes in observed crime rates over this period. Thus, middle-ground DID assumption (see table 5), RTC laws are
while the bounded-variation parameter dVA(d,88) in table 3 found to reduce the Virginia murder rate in 1995 by at least
may be sensible when evaluating the counterfactual crime 16% (from 9.1 to 7.6), the rape rate by at least 6%, the rob-
rate in 1989, it is hard to defend when evaluating the rate in bery rate by at least 37%, and the larceny rate by at least
2006. In fact, if we add in the standard bounded time varia- 13%. In contrast, in 2005, RTC laws are found to reduce
tion assumption in equation (7), just over two out of every the murder rate by at least 15% but increase the rape rate by
five of the bounds are infeasible, with the lower bound at least 47%, the assault rate by at least 8%, the robbery rate
being above the upper bound. by at least 3%, and the burglary rate by at least 13%.
Table 5 displays the findings using these bounded-varia- These findings of heterogeneous effects across years may
tion assumptions. Findings highlighted in light gray identify partially explain why the findings obtained with homoge-
the sign of the treatment effect, with those in bold as nega- nous response assumptions like equation (6) are sensitive to
tive and those in italics as positive. Findings highlighted in the years of data included in the sample. Analyses per-
dark gray are infeasible, the lower bound being above the formed using more recent data tend to find that RTC laws
have negligible or even positive effects on violent crime,
whereas the same analyses performed using data through
14
In general, the bounds are not sensitive to the cutoff dates used for the early 1990s tend to find that RTC laws decrease violent
the bounded time variation assumption C. For example, changing the cut-
off years from 1996 to 1995 and 1999 to 2000 does not change any of the crime rates. (See Aneja et al., 2011, and National Research
qualitative conclusions. If we drop assumption C altogether, 10% of the Council 2005, tables 6-5 and 6-6.)
bounds are affected, but only a handful of the qualitative results change. Given that the effects vary over time and across crimes
In particular, without assumption C, the bounds for assault in 2003 and
2004 are not identified to be positive, and the bounds for burglary from and in many cases do not reveal the sign of the TE, it is dif-
1990 to 1993 are not identified to be negative. ficult to provide a simple assessment of the overall efficacy
 HOW DO RIGHT-TO-CARRY LAWS AFFECT CRIME RATES? 243

of the RTC law in Virginia. One’s perspective will necessa- In this paper, we have not considered prospective analysis.
rily depend both on the assumption that seems most plausi- The literature on RTC laws, including this paper, may pro-
ble, if any, as well as how to weigh results that vary over vide policymakers useful retrospective information. It is not,
time and across crimes. Under the weaker assumptions, the per se, informative about prospective questions and, in parti-
bounds generally do not identify the sign of the TE and thus cular, about what would happen if an RTC statute were to be
provide little guidance about whether RTC laws increase or adopted in a state that does not currently have such a law.
decrease crime. Under the stronger DID invariance assump-
tion (table 4), RTC laws are found to have reduced crime in V. Conclusion: Incredible Certitude
the early 1990s and have mixed effects a decade later,
in the 2000s. Similar but less definitive results are found Given that research on RTC laws is often inconclusive,
under the middle-ground assumption used to generate the contradictory, and confusing, the research community has
results displayed in table 5. been largely marginalized in this important policy debate.
To assess the overall benefit of an RTC statute, one must Why? Researchers are rewarded for producing simple find-
somehow aggregate the effects across the different crimes. ings leading to definitive policy prescriptions (e.g., more
Obviously the decision of how to weight the effects for dif- guns leads to less crime), yet generating such findings
ferent crimes is a complex and subjective undertaking. A requires strong assumptions that cannot be persuasively
simple metric might be to count the aggregate change in defended. In this setting, researchers report findings with
crimes. In this case, using the results from table 5, one ‘‘incredible certitude’’ rather than expressing due caution
would conclude that RTC laws tended to decrease the num- (Manski, 2013). Drawing inferences about the effects of
ber of crimes in the 1990s but increase the total number of RTC laws, or gun policy more generally, is an inherently
crimes in the 2000s. difficult undertaking: conclusions are highly sensitive to the
Instead, one might adjust the raw counts for the fact that data and assumptions, the available data are limited, and a
some crimes are more costly to society than others. A num- wide range of assumptions, and thus conclusions are consis-
ber of researchers have sought to estimate the average costs tent with the data. Researchers combining data with differ-
of different crimes.15 By combining the effects of RTC laws ent maintained assumptions reach different logically valid
on crime rates with a set of cost estimates, one can compute conclusions yet fail to adequately express sensitivity of the
how the average costs of crime would change if an RTC findings to untestable assumptions.
statute were to be adopted in Virginia. While admittedly a For empirical research on complex policies such as gun
tenuous exercise, this might provide a rough snapshot of the laws, and RTC laws in particular, to be informative to the
aggregate impact of RTC laws on crime. policy debate, we believe it is critical that the discussion
A final complication in evaluating the impact of RTC move away from this paradigm of incredible certitude
laws arises because the results reported are retrospective, toward an honest portrayal of partial knowledge (Manski,
not prospective. The two forms of analysis differ in their 2013). Adequate expression of caution goes beyond using
objectives. Researchers performing retrospective analysis temperate language, replicating results under marginally
aim to learn past treatment effects in a study population, different assumptions, or reporting confidence intervals.
asking questions such as: What do we know about the Although helpful, these means of expressing caution do not
effects of RTC laws in Virginia from 1990 to 2006? A cen- go nearly far enough. Drawing inferences under a variety
tral objective of prospective analysis is to inform treatment of assumptions that are not credible does not resolve the
choice in a future population, asking questions of the form: problem. Adequate expression of caution requires formal
What would happen if a state were to adopt an RTC statute methods to perform empirical research under assumptions
in 2018? that are weak enough to be credible.
Prospective analysis is more difficult than retrospective In this paper, we develop and apply one such set of
analysis. Empirical evidence on treatment response is assumptions: bounded-variation assumptions. These assump-
entirely absent prospectively, but it is partially present ret- tions, which relax the traditional invariance assumptions
rospectively after the outcomes of realized treatments are applied in the literature, provide an intuitive, simple, and flex-
observed. Under a time-invariance assumption, as applied ible way to improve the credibility of empirical research,
in assumption (6), the answer is straightforward: the impact assess the sensitivity of inferences to different identifying
of RTC laws does not vary over time. Yet this assumption restrictions, and apply middle-ground assumptions that
is not tenable. One must address the forecasting problem sometimes identify the sign of the TE.
inherent in prospective inference. The results reveal the inherent trade-off between the
strength of assumptions and findings. Even under the stron-
15
As might be expected, there is a great deal of variation in these cost- gest invariance models, the TE of an RTC statute varies
of-crime estimates, but they all imply that murder is many times more across crimes, states, and time. Findings under the more
costly than any other crime. For example, McCollister, French, and Fang credible bounded-variation models suggest even greater
(2010) estimate that the average cost (in 2008 dollars) of a murder is
nearly $9 million, whereas the average cost of a rape, the next most costly degrees of ambiguity, with many results not identifying the
offense, is $241,000. sign of the TE and others varying over time and across
244 THE REVIEW OF ECONOMICS AND STATISTICS

states. Still, under these assumptions, we find that the RTC Duggan, Mark, ‘‘More Guns, More Crime,’’ Journal of Political Economy
109 (2001), 1086–1114.
law in Virginia reduced murder and larceny rates in nearly Durlauf, Steven. N., Salvador Navarro, and David A. Rivers, ‘‘Model
every year but increased assaults after 1997. For other Uncertainty and the Effect of Shall-Issue Right-to-Carry Laws on
crimes, the sign of the bounds is generally negative in the Crime,’’ European Economic Review 81 (2016), 32–67.
Imbens, Guido, and Charles F. Manski, ‘‘Confidence Intervals for Par-
1990s but positive or indeterminate in the 2000s. tially Identified Parameters,’’ Econometrica 72 (2004), 1845–
These results are informative but they are inconsistent 1857.
with the notion that RTC laws uniformly increase or Kaido, Hiroaki, Francesca Molinari, and Jorg Stoye, ‘‘Confidence Inter-
vals for Projections of Identified Parameters,’’ Department of Eco-
decrease crime. In this light, we do not report findings with nomics, Cornell University (2016).
incredible certitude: there are no simple conclusions. Still, Lafférs, Lukas, ‘‘Bounding Average Treatment Effects Using Linear Pro-
our findings may inform the policy debate by providing gramming,’’ cemmap working paper CWP70/15 (2015).
Lott, John R, More Guns, Less Crime: Understanding Crime and Gun-
credible information that constrains the resulting discussion Control Laws (Chicago: University of Chicago Press, 2010).
to lie within a set of bounds. Lott, John R., and David Mustard, ‘‘Crime, Deterrence and Right-to-
Carry Concealed Handguns,’’ Journal of Legal Studies 26 (1997),
REFERENCES 1–68.
Manski, Charles F., ‘‘Nonparametric Bounds on Treatment Effects,’’
Abadie, Alberto, Susan Athey, Guido Imbens, and Jeffrey Wooldridge, American Economic Review Papers and Proceedings 80 (1990),
‘‘Finite Population Causal Standard Errors,’’ NBER working paper 319–323.
20325 (2014). ——— ‘‘Monotone Treatment Response,’’ Econometrica 65 (1997),
Abadie, Alberto, Alexis Diamond, and Jens Hainmueller, ‘‘Synthetic Con- 1311–1334.
trol Methods for Comparative Case Studies: Estimating the Effect ——— Identification for Prediction and Decision (Cambridge, MA: Har-
of California’s Tobacco Control Program,’’ Journal of the Ameri- vard University Press, 2007).
can Statistical Association 105 (2010), 493–505. ——— Public Policy in an Uncertain World: Analysis and Decisions
Aneja, Abhay, John J. Donohue, and Alexandria Zhang, ‘‘The Impact of (Cambridge, MA: Harvard University Press, 2013).
Right-to-Carry Laws and the NRC Report: Lessons for the Empiri- Manski, Charles F., and John V. Pepper, ‘‘Monotone Instrumental Vari-
cal Evaluation of Law and Policy,’’ American Law and Economic ables: With an Application to the Returns to Schooling,’’ Econo-
Review 13 (2011), 565–632. metrica 68 (2000), 997–1010.
Black, Dan A., and Daniel S. Nagin, ‘‘Do Right-to-Carry Laws Deter Vio- ——— ‘‘Deterrence and the Death Penalty: Partial Identification Analysis
lent Crime?’’ Journal of Legal Studies 27 (1998), 209–219. Using Repeated Cross Sections,’’ Journal of Quantitative Crimin-
Chernozhukov, Victor, Han Hong, and Elie Tamer, ‘‘Estimation and Con- ology 29 (2013), 123–141.
fidence Regions for Parameter Sets in Econometric Models,’’ McCollister, Kathryn E., Michael T. French, and Hai Fang, ‘‘The Cost of
Econometrica 75 (2007), 1243–1284. Crime to Society: New Crime-Specific Estimates for Policy and
Chernozhukov, Victor, Simon Lee, and Adam Rosen, ‘‘Intersection Bounds: Program Evaluation,’’ Drug and Alcohol Dependence 102 (2010),
Estimation and Inference,’’ Econometrica 81 (2013), 668–737. 98–109.
Cochran, William G., Sampling Techniques (New York: Wiley, 1977). National Research Council, Firearms and Violence: A Critical Review,
Deaton, Angus, The Analysis of Household Surveys (Baltimore: Johns edited by C. Wellford, J. Pepper, and C. Petrie (Washington, DC:
Hopkins University Press, 1997). National Academy Press, 2005).
Donohue, John J., and Steven Levitt, ‘‘Guns, Violence, and the Efficiency Richardson, Amy, Michael G. Hudgens, Peter B. Gilbert, and Jason P.
of Illegal Markets,’’ American Economic Review 88 (1998), 463– Fine, ‘‘Nonparametric Bounds and Sensitivity Analysis of Treat-
467. ment Effects,’’ Statistical Science 29 (2014), 596–618.