0% found this document useful (0 votes)
38 views38 pages

Can Financial Aid Help To Address The Growing Need For STEM Education? The Effects of Need-Based Grants On The Completion of Science, Technology, Engineering, and Math Courses and Degrees

Download as pdf or txt
Download as pdf or txt
Download as pdf or txt
You are on page 1/ 38

Can Financial Aid Help to Benjamin L.

Castleman
Bridget Terry Long
Address the Growing Need Zachary Mabel
for STEM Education? The
Effects of Need-Based
Grants on the Completion of
Science, Technology,
Engineering, and Math
Courses and Degrees

Abstract
Although workers in science, technology, engineering, and math (STEM) fields earn
above-average wages, the number of college graduates prepared for STEM jobs lags
behind employer demand. A key question is how to recruit and retain college students
in STEM majors. We offer new evidence on the role of financial aid in supporting STEM
attainment. Exploiting a regression discontinuity that allows for causal inference, we
find that eligibility for need-based financial aid increased STEM credit completion by
20 to 35 percent among academically-ready students in a large, public higher education
system. These results appear to be driven by shifting students into STEM-heavy course
loads, suggesting aid availability impacts the academic choices students make after
deciding to enroll. We also find suggestive evidence that aid offers increase degree
attainment in STEM fields, although we cannot rule out null impacts on STEM degree
production.  C 2017 by the Association for Public Policy Analysis and Management.

INTRODUCTION
Employers in the United States are seeking to hire more workers in the science,
technology, engineering, and mathematics (STEM) sector. Since 2008, the number
of STEM jobs has increased by 14 percent compared to 2 percent in non-STEM
fields, and above-average growth in STEM employment is projected to continue
over the next decade (Fayer, Lacey, & Watson, 2017; Noonan, 2017). Workers in
STEM fields also earn a wage premium, even after controlling for ability sorting
in school and the workplace (Arcidiacono, 2004; Hastings, Neilson, & Zimmerman,
2013; Kirkeboen, Leuven, & Mogstad, 2016). The average starting salary for entry-
level STEM jobs requiring associate or bachelor’s degrees is $50,000 and $69,000
(in 2017 dollars), respectively, which represent 30 percent premiums over other
entry-level jobs with the same degree requirements (Burning Glass Technologies,
2014).
Given the private returns to STEM attainment and the labor market demand for
more college-educated workers in those fields, a key policy question is how to in-
crease educational investments and degree completion in STEM. STEM attainment
is especially low for low-income and minority students. Only 40 percent of college

Journal of Policy Analysis and Management, Vol. 37, No. 1, 136–166 (2018)

C 2017 by the Association for Public Policy Analysis and Management
Published by Wiley Periodicals, Inc. View this article online at wileyonlinelibrary.com/journal/pam
DOI:10.1002/pam.22039
The Effects of Need-Based Grants on STEM attainment / 137

students overall and one-quarter of black and Latino students initially interested
in pursuing STEM majors persist to earn a degree in the field (ACT, 2014; Higher
Education Research Institute, 2010). These rates concern policymakers because the
country is projected to require one million additional STEM professionals over the
next decade to retain global competiveness in science and technology (Lacey &
Wright, 2009; Langdon et al., 2011). With foreign-born workers currently account-
ing for less than 20 percent of STEM workers with a bachelor’s degree (Hanson
& Slaughter, 2016), meeting current and future STEM employment needs will re-
quire substantially increasing the number of domestic college graduates qualified
for STEM jobs unless employment-based immigration or international student re-
cruitment is expanded, neither of which appears to be the focus of current policy.
In this paper, we examine a potentially overlooked barrier to STEM attainment
in college: the financial cost to pursuing study in STEM courses and majors. While
other work has investigated the role of information, identity, preferences, & peer
influence on STEM attainment (Steele & Aaronson, 1995; Griffith, 2010; Price,
2010; Espinosa, 2011; Wang, 2013), there are several reasons why policies targeting
affordability could impact STEM attainment, including by altering the decision of
whether to enroll in college for students on the margin of attendance, as well as
where to enroll and what field to study among inframarginal enrollees. Our paper
offers new insight into the role that affordability plays in determining whether
students pursue and persist in STEM fields.
We examine the effects of need-based grant eligibility on STEM attainment us-
ing a quasi-experimental research design. Specifically, we focus on the impact of
eligibility for the need-based Florida Student Assistance Grant (FSAG) on whether
students complete courses and degrees in STEM fields. In the early 2000s, colleges
and universities in Florida determined eligibility for the FSAG using the federal
need analysis calculation.1 During the 2000/2001 school year, students whose Ex-
pected Family Contribution (EFC) was less than or equal to $1,590 were eligible
for a $1,300 FSAG award (in 2000 dollars); this translates to families with incomes
below approximately $30,000 being eligible for a FSAG. The state grant was suffi-
cient to cover 57 percent of the average cost of tuition and fees at an average public,
four-year university in Florida.2 These students also qualified for at least a $1,750
Federal Pell Grant; as a result, FSAG awards increased the amount of need-based
financial aid eligible students could receive by nearly 75 percent. In contrast, stu-
dents whose EFCs were just above $1,590 were not eligible for the FSAG and only
received the Federal Pell Grant.
Capitalizing on this EFC threshold for aid eligibility, we utilize a regression-
discontinuity (RD) approach to estimate the causal effect of FSAG eligibility on
STEM credit and degree attainment. To focus on students who could plausibly
enroll in STEM courses, we limit the sample to Florida high school graduates who
demonstrate academic readiness for postsecondary study in STEM fields. We proxy
for STEM readiness in two ways. First, we condition on students who surpass
college-ready math standards on the Florida College Placement Test in math (CPT-
M) or the SAT Math examinations. Second, we condition on students who completed

1 Applying for federal financial aid, and often for state and institutional aid, requires a student to complete
the Free Application for Federal Student Aid (FAFSA). The FAFSA collects information on family income
and assets to determine the EFC, the amount that a family is estimated to be able to pay for college. To
calculate need, the government subtracts the EFC from the total cost of attendance. A student’s financial
need, in combination with his or her EFC, determines whether the student is eligible for certain grants
and loans.
2 Authors’ calculations using data from the Integrated Postsecondary Education Data System (IPEDS).

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
138 / The Effects of Need-Based Grants on STEM attainment

trigonometry or a more advanced mathematics class in high school, given that high
school math achievement is predictive of entrance into STEM majors (Wang, 2013).
We find meaningful effects of need-based grant aid eligibility on STEM attain-
ment among students ready for college-level STEM coursework. FSAG award
offers increased cumulative STEM course completion seven years following high
school graduation by 3.7 credits for students who placed into college-level math and
7.3 credits for students who completed trigonometry or higher in high school, which
represent respective gains of 20 and 34.5 percent relative to students just above the
aid eligibility cutoff. Award offers also had a concentrated effect on STEM-related
outcomes, rather than just improving academic outcomes across all subjects. FSAG
eligibility shifted students’ course loads towards more STEM-heavy choices and this
effect does not appear to be driven by the impact of aid offers on institutional choice.
The pattern of results therefore suggests that students make cost-conscious decisions
when choosing not only where, but what to study in college. We find weaker evi-
dence that award offers increased bachelor’s degree attainment in STEM fields. The
magnitude of the estimates suggests that FSAG eligibility increased STEM degree
completion by 3 percentage points (representing gains of 50 to 60 percent relative
to students just above the EFC threshold), but the effect estimate is only significant
at the 10 percent level in one sample and non-significant in the other. We therefore
cannot rule out that FSAG award offers had no impact on STEM degree production.
We structure the remainder of the paper into four sections. In the next section,
we review the existing literature on college access and success pertinent to our
examination of need-based grants and STEM achievement. In the third section, we
describe our data and research design. We present our results in the fourth section.
The fifth section concludes and discusses the implications of the results for policy
and research.

BACKGROUND

Possible Impacts of Financial Aid on STEM Attainment


Offering students more generous need-based aid could theoretically increase STEM
attainment in college on either the extensive (e.g., enrollment) or intensive (e.g.,
major choice) margins. On the extensive margin, need-based aid has been shown to
increase investments in schooling by reducing credit constraints for some students
(i.e., by producing an income effect) and by lowering the cost of attendance relative
to non-schooling options (i.e., by producing a substitution effect) (see Dynarski
& Scott-Clayton, 2013, for an extensive review of this literature). Offering more
generous need-based aid may therefore induce some students with STEM aptitude
into college who would otherwise not enroll.
Financial aid may also impact STEM attainment on the intensive margin, al-
though it is not immediately obvious if the effect would be positive or negative.
On the one hand, if need-based grants increase the likelihood of attending flag-
ship universities that offer more robust STEM programs and make those majors
more attractive than institutions with fewer resources, aid might increase STEM
attainment.3 However, there are at least two reasons why the effects, conditional on
enrollment, are ambiguous. First, most grant aid, including the FSAG we study, is

3 Between the 2000/2001 and 2006/2007 school years, the three public flagship universities in Florida—
the University of Florida, Florida State University, and University of South Florida—offered 17 or more
STEM programs of study. This exceeded the number of STEM programs available at each of the other
seven public universities in the State.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 139

not targeted. As a result, on the intensive margin financial aid may continue to relax
credit constraints via an income effect but will generally not produce a substitution
effect by altering the relative tuition cost of pursuing different majors.
Second, it is unclear whether the income effect on the intensive margin will
encourage or discourage students from studying STEM. For example, financial aid
could have a negative effect by relieving pressure students feel to pursue a major
with high expected earnings (Andrews & Stange, 2016; Rothstein & Rouse, 2011).
Additional grant aid may therefore encourage some students who would experience
greater non-pecuniary benefits from alternative majors to switch away from STEM
fields. Alternatively, aid could have a positive effect by lowering investment costs,
which may be real costs students incur, such as laboratory or materials fees that
make STEM fields more expensive than alternative options (Stange, 2015), effort
costs required to succeed in rigorous coursework, or perceived costs of foregone
work and leisure time. Additional financial aid may therefore free up time and
lower costs to pursuing more demanding coursework by reducing the hours needed
to work and allow credit-constrained students to pay extra costs associated with
STEM programs that would otherwise be unaffordable.4
The combined effect of lowering net cost on the intensive margin is therefore
ambiguous and depends on whether the effect on earnings expectations or invest-
ment costs dominates. Furthermore, it is reasonable to assume that those effects
will vary in size across students and produce heterogeneous impacts of financial aid
on STEM attainment. Indeed, because experimental evidence indicates that student
preferences are a more important factor than expected earnings in choosing a major
(Wiswall and Zafar, 2015), it is plausible that additional need-based aid would have
null effects among students with weak academic backgrounds and preferences for
STEM, as well as students with very strong backgrounds and preferences who do
not face financial barriers to pursuing STEM coursework. Yet for students prepared
for and interested who cannot afford to pursue STEM coursework in college, the net
impact of additional grant aid on STEM attainment may be positive and increasing
in both interest and financial need.

Prior Literature
Previous work investigating the impact of financial aid on STEM college outcomes
is very limited, has found mixed results, and may not generalize to non-targeted,
need-based aid programs like the FSAG. Two studies of which we are aware have
examined the impacts of the federal SMART grant, a targeted aid program that
awarded grants to low- and moderate-income college juniors and seniors who ma-
jored in STEM during the 2007 to 2011 academic years. Whereas Denning & Turley
(2017) find that income-eligible students in Texas were approximately 3 percentage
points more likely to major in STEM fields in their junior or senior year, Evans
(2017) finds no evidence of SMART grant impacts on whether students in Ohio
persisted in STEM majors or earned STEM degrees.
Other studies have examined the effects of non-targeted aid programs on STEM
attainment, but this literature is limited to investigations of merit-based scholarship
programs and the evidence to date is also inconclusive. Zhang (2011) finds little evi-
dence that merit aid programs in Florida and Georgia influenced the share of degrees
conferred in STEM fields, although the possibility of merit-induced selection effects

4 We refer the reader to Appendix A for a model of major choice that would generate these ambiguous
predictions. All appendices are available at the end of this article as it appears in JPAM online. Go to the
publisher’s website and use the search engine to locate the article at http://onlinelibrary.wiley.com.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
140 / The Effects of Need-Based Grants on STEM attainment

are not accounted for in this study. By comparison, Sjoquist and Winters (2015a,
2015b) account for the effects of merit aid programs on student quality and find
evidence that state merit aid programs decrease the number of STEM graduates,
perhaps because academic renewal requirements have the unintended consequence
of inducing students to avoid rigorous coursework to maintain their awards (Corn-
well, Lee, & Mustard, 2005, 2006). Yet because need-based aid programs typically
have less stringent renewal requirements than merit aid programs and target more
credit-constrained students, whether need-based aid creates positive or negative
incentives for students to pursue STEM coursework remains an open question.
Ours is the first paper of which we are aware that examines the impact of eligibility
for a non-targeted, need-based grant at the end of high school on whether students
accumulate STEM credits and earn STEM degrees. Because the FSAG grant pro-
gram was not targeted specifically to students pursuing STEM fields, and neither
is most need-based aid, its impact is likely more generalizable to financial aid pol-
icymaking than previous research. For instance, compared to federal government
expenditures of $195 million annually through the SMART grant program in the
2006/2007 and 2007/2008 school years, federal and state governments spend over
$40 billion annually on need-based grant programs that are similar in structure and
design to the FSAG award (Baum, Elliot, & Ma, 2014; Newman, 2014). Programs
like FSAG therefore represent the principle sources of need-based aid allocated to
support college access and attainment.

Background on Florida and the FSAG Program


This paper examines the following research question: Does eligibility for additional
need-based grant funding (above the federal Pell Grant) increase the number of
STEM credits that students accumulate in college and increase their probability of
earning a bachelor’s degree in a STEM field? To investigate our research question,
we focus on Florida high school seniors who graduated in the 1999/2000 school year.
The impact of need-based financial aid in Florida is potentially informative about
the efficacy of financial aid programs in a broad range of settings across the country.
Similar shares of students attend Florida public institutions and enroll at in-state
institutions as in the country overall (88 percent vs. 91 percent and 86 percent vs.
92 percent, respectively). The average in-state tuition, room, and board levels at
Florida public four-year institutions are also fairly similar in Florida compared to
public four years across the country ($14,677 vs. $18,632). Florida also represents
the increasing racial and ethnic diversity of the country as a whole: 16 percent of its
residents are black and 23 percent of its residents are of Hispanic or Latino origin
(United States Census Bureau, 2011).
Specific to the context of financial aid, Florida students could qualify for both
need- and merit-based state grants. Each year, families must complete the FAFSA,
which asks for information on income, assets, and family size. Using this informa-
tion, the U.S. Department of Education (USDOE) estimates the families’ EFC that
it uses, along with the cost of attendance at students’ intended institutions, to de-
termine each student’s eligibility for financial aid. States also use the EFC to award
need-based grants, which totaled $8 billion in the 2012/2013 academic year (Baum,
Elliot, & Ma, 2014).
To apply for the need-based FSAG, students needed to complete the FAFSA by
March 1st of their senior year in high school. The Florida Department of Educa-
tion sets an EFC cutoff each year to determine FSAG eligibility, which during the
2000/2001 academic year was $1,590 (Florida Postsecondary Education Planning
Commission, 2001). Institutions were prohibited from awarding grants to students
whose families exceeded this amount, thus making this a sharp eligibility cutoff.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 141

Students could use the FSAG at any public two- or four-year college or university in
Florida. During the 2000/2001 academic year, the FSAG award for which students
were eligible ($1,300) was sufficient to pay 57 percent of the average cost of tuition
and fees at a public university in the state or about 28 percent of the average cost
of tuition/fees, room, and board. Added on top of the federal Pell Grant, which all
students around the FSAG cutoff were eligible to receive, students could receive
up to $3,050 in need-based grants. The FSAG was also renewable from one year to
the next, conditional on students remaining financially eligible and maintaining a
cumulative college GPA of 2.0 or higher.5
In addition to FSAG, Florida students were also eligible for the merit-based Florida
Bright Futures Scholarship (BFS). There are two tiers of BFS awards. The lower tier
amounted to approximately $1,700 for students who completed 15 core academic
credits, had a cumulative high school GPA of 3.0 or higher, and had a composite SAT
score of 970 or higher. Seventy percent of students who received a BFS award in the
2000/2001 academic year received the lower-tier award. The higher tier offered a
$2,500 award plus a small living stipend and was offered to students who completed
15 core academic credits, had a cumulative high school GPA of 3.5 or higher, and
had a composite SAT score of 1,270 or higher. Approximately 54 percent of students
were eligible for a BFS award in our analytic samples. We control for BFS eligibility
in our analysis to account for this potential confounding source of state grant aid;
however, because students immediately on either side of the cutoff were equally
likely to qualify for the BFS, in practice our results are robust to excluding this
control. We also find no evidence that effects on STEM attainment are concentrated
among BFS-eligible students. It therefore does not appear that this source of aid is
responsible for any differences in academic outcomes at the FSAG eligibility cutoff.
In Figure 1, we summarize the variation in total grant aid eligibility in Florida
according to small differences in family resources. Focusing on the area around the
FSAG eligibility cutoff, students ineligible for the BFS could receive $3,050 in total
FSAG and Pell Grant funding if their EFC did not exceed $1,590, or only $1,750
in Pell Grants if their EFC was above $1,590. For students who met the criteria for
BFS eligibility, being above or below the FSAG cutoff resulted in the same difference
in aid, but the levels of grant aid were higher ($4,750 versus $3,450 for lower-tier
BFS-eligible students on either side of the cutoff, and $5,650 versus $4,350 for the
higher-tier BFS-eligible students).

EMPIRICAL ANALYSIS

Data
The data in this article are from the Florida Department of Education K-20 Data
Warehouse (KDW), which maintains longitudinal student-level records at Florida
public colleges and universities. We also have data from KDW secondary-school
records, including demographics, high school transcripts, and college entrance
examination scores. These data are linked to KDW postsecondary data that provide
the financial information that families supplied when completing the FAFSA and
all financial-aid disbursements students received while enrolled in college. The
postsecondary data also track students’ enrollment and course-taking histories and
their degrees received. We therefore observe students’ semester-by-semester STEM

5There was no limit on the number of years for which students in our data could renew their FSAG
award.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
142 / The Effects of Need-Based Grants on STEM attainment

Notes: Total aid eligibility is the sum of the federal Pell Grant, Florida Student Access Grant (FSAG), and
Bright Futures Scholarship (BF) awards for which students qualify. During the study period, students
with EFCs below $1,590 were eligible for FSAG awards that offered $1,300 in additional need-based aid
per year. The two tiers of merit-based BF scholarships vary in generosity based on students’ incoming
high school GPA and SAT scores. See text for eligibility details.

Figure 1. Federal and Florida Grant Aid Eligibility by EFC.

credits attempted and completed and can examine credit accumulation over short-,
medium-, and long-term intervals.
The data also include students’ field of study each semester, although we are
unable to distinguish between intended and declared majors. We instead focus on
the number of STEM credits students attempted each semester as a better measure
of the extent to which they advanced towards a STEM degree over time. We also
observe whether students earned degrees and their field of study at the time of
degree receipt, which enable us to report on bachelor’s degree attainment in STEM
fields. These three measures—STEM credits attempted, STEM credits completed,
and bachelor’s degree receipt in STEM disciplines—are our primary outcomes of
interest.

Defining STEM Courses and Degrees


We employed two methods to define STEM courses and degrees in the data, both
of which avoided counting math and science courses as STEM that non-STEM
majors are often required to complete to graduate. At the course level, we identified
STEM using course prefix codes included in the KDW postsecondary data file. This
was our best method of identifying STEM courses by department at both two- and
four-year institutions, and it eliminated reliance on indiscriminate course names
across subjects and institutions to flag courses as STEM. To identify STEM degrees,
we matched Classification of Instructional Program (CIP) codes in the KDW data
to the list of STEM-designated CIP codes maintained by the National Center for Ed-
ucation Statistics and U.S. Department of Homeland Security (National Center for
Education Statistics, 2011; United States Department of Homeland Security, 2012).
Unfortunately, we could not identify programs of study for most students attending
two-year colleges in the data; we therefore identified STEM degrees exclusively

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 143

for bachelor’s degree recipients.6 We also separately identified the types of STEM
courses and degrees (e.g., Computer Science, Physical Sciences, Health Sciences,
etc.) to examine the sensitivity of our results to more and less restrictive definitions
of STEM. The results we present throughout the paper are based on our most
restrictive definition of STEM, comprised of Computer Science, Engineering,
Mathematics & Statistics, Physical Sciences, and Biological Sciences, although our
results are robust to alternative definitions that include Agricultural, Health, and
Environmental Sciences.

Samples
The KDW data set captures college enrollment and completion records for the
majority of college-bound, low-income Florida high school seniors. During the
2000/2001 academic year, 90 percent of Florida residents who enrolled in college
for the first time did so at in-state institutions and 74 percent of first-time freshmen
enrolled in public institutions (National Center for Education Statistics, 2002). The
coverage of the data is likely even higher for low-income Florida residents because
the average cost of attendance at private and out-of-state colleges was considerably
higher than the price of Florida public colleges and universities for in-state students.
To investigate this, we examined the college enrollment patterns of high-achieving,
low-income high school seniors in the Educational Longitudinal Study of 2002
(ELS:2002). Because this is a nationally representative sample of students, there are
only 43 to 63 high-achieving, low-income Florida students in the sample, depending
on the definition of high-achievement that is used. However, among those students,
only 6 to 9 percent enrolled at out-of-state institutions. Furthermore, we also
observe a proxy for enrollment at private, four-year colleges and universities in
Florida in the KDS data based on receipt of the Florida Resident Assistance Grant
(FRAG). The FRAG was a tuition-assistance grant of $2,800 automatically awarded
to students to offset the cost of tuition at private institutions. In fall 2000, only
7 percent of all public high school graduates in the 1999/2000 school year who filed
a FAFSA received a FRAG award. Taken together, the evidence above lends support
for the case that the KDW data capture college outcomes for the vast majority of
STEM-ready, low-income students in Florida.
In our empirical work, we focus on a subset of Florida high school graduates
in the 1999/2000 academic year who demonstrate academic readiness for STEM
coursework in college. We present results for two analytic samples.7 Of the 101,094
graduates in Florida that year, we first restricted our samples to include only stu-
dents who submitted a FAFSA application since this is a necessary step for receiving
government and most institutional aid. This restriction resulted in the exclusion of
55,309 students from our sample.
We proxied for STEM readiness by further restricting the data in two ways: First,
we conditioned on students who surpassed college-ready math standards on the
CPT-M or the SAT Math examinations. Florida’s CPT is designed to provide place-
ment, advisement, and guidance information for students entering two- or four-year

6 Approximately three-quarters of students attending two-year colleges had an uninformative major code
of “General Degree Transfer.”
7 As a falsification test, we also estimated FSAG eligibility impacts for a third sample comprising Florida
high school graduates near the FSAG eligibility cutoff who were not academically prepared for STEM
coursework in college. Results for this sample are presented in Appendix Table B4. We find negligible
impacts on STEM outcomes for this group of students. All appendices are available at the end of this
article as it appears in JPAM online. Go to the publisher’s website and use the search engine to locate the
article at http://onlinelibrary.wiley.com.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
144 / The Effects of Need-Based Grants on STEM attainment

colleges and universities. For the incoming class of first-year college students in
the 2000/2001 academic year, Florida established a mandatory cutoff score of 72
out of 120 points on the CPT-M examination for placement into college-level math
(Florida Department of Education, 2006). Because students with SAT Math scores of
440 points or higher were exempted from taking the CPT placement examination, we
also include in this sample students with missing CPT-M scores who had SAT Math
scores above the exemption threshold (Florida Department of Education, 2006).8
This yields a sample of 20,738 students with college entrance examination scores
above the state-mandated cutoff for placement into college-level math courses, 2,834
of whom had EFCs near the FSAG eligibility cutoff (i.e., plus or minus $1,000 around
the cutoff).9
We constructed a second sample because the mandatory CPT-M and SAT Math
scores that determined placement into college-level coursework established rela-
tively low thresholds for STEM readiness. Our second analytic sample is conditioned
on students who completed trigonometry or a more advanced math class (e.g., cal-
culus, differential equations, linear algebra, etc.) in high school. We established
trigonometry as the cut-point by examining the math course enrollment patterns
of Florida high school seniors; after the typical three-course sequence comprised of
algebra 1, geometry, and algebra 2, trigonometry was the next most popular course
students took in twelfth grade. This restriction generated a sample of 8,907 stu-
dents, 1,283 of whom had EFCs within $1,000 of the FSAG eligibility cutoff, and
likely captures students more prepared to pursue STEM at the start of college.
In Table 1, we present empirical evidence that the restrictions we employ do,
in fact, identify students who are more likely to study STEM in college. Students
who met the test score conditions and enrolled in college in fall 2000 completed
21 credits in STEM on average throughout their time in school, and 6 percent
earned a bachelor’s degree in a STEM field. By comparison, students with lower
achievement scores completed 10 fewer credits in STEM and only 1 percent earned
a STEM degree. We observe similar differences in STEM attainment between stu-
dents who completed trigonometry or a more advanced math course in high school
and those who did not. The results in Table 1 also indicate that our restriction con-
ditions establish relatively conservative proxies for STEM readiness. For example,
whereas 16 percent of students who completed calculus or another advanced math
course in high school earned a BA in STEM, only 4 percent of students who took
math up to trigonometry graduated with a degree in STEM. In spite of this evidence,
we adhere to conservative proxies because we do not observe enough students with
higher preparation around the FSAG eligibility cutoff to detect effects using more
stringent thresholds. Our results likely provide lower-bound effect estimates as a re-
sult, given that some students in our samples may have had insufficient backgrounds
for studying STEM at selective four-year universities in Florida.10

8 In practice, nearly all students with missing placement examination scores scored considerably higher
than 440 on the SAT Math. The mean score for this subset of students is 556 points with a standard
deviation of 79.8 points.
9 Although this sample is conditioned on a potentially endogenous regressor—completion of a college
placement test—students only learn if they have been offered an FSAG award after they have received
admissions offers. Furthermore, because our second sample is free of this endogeneity, we believe this
sample offers a useful robustness check for whether our results hold across different definitions of STEM-
readiness in college. Our results are substantively similar across both samples and the point estimates
tend to be larger in the second sample.
10 In Appendix Table B1, we present additional evidence of math achievement differences across the
two samples. Students who completed pre-calculus scored, on average, 89 and 526 points on the CPT-M
and SAT Math examinations, respectively, which lie well above the college math placement cutoffs on
both examinations. Our second sample therefore includes higher math achievers, while our first sample

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 145

Table 1. STEM attainment by college math test scores and highest math course completed
in high school, among college enrollees in fall 2000.

(1) (2) (3)


STEM credits
completed BA/BS degree
through Year 7 in STEM N

(A) College math test scores


CPT < 40 6.45 0.00 4,398
40 ࣘ CPT < 60 10.05 0.01 4,652
60 ࣘ CPT < 80 13.30 0.01 4,068
80 ࣘ CPT < 100 17.96 0.03 4,026
100 ࣘ CPT 25.85 0.08 2,488
Missing CPT 19.39 0.06 27,743
SAT < 400 14.12 0.01 1,708
400 ࣘ SAT < 500 15.66 0.02 8,177
500 ࣘ SAT < 600 19.88 0.05 10,626
600 ࣘ SAT < 700 27.92 0.13 5,216
700 ࣘ SAT 42.44 0.28 1,062
Missing SAT 19.39 0.06 27,743
Excluded from college math sample 11.01 0.01 18,072
In college math sample (CPT ࣙ 72 | SAT ࣙ 440) 20.64 0.06 29,303
(B) Highest math course in high school
Algebra 1 7.08 0.01 1,796
Geometry 8.42 0.01 5,255
Algebra 2 13.87 0.02 19,162
Trigonometry 19.04 0.04 5,413
Pre-calculus 22.57 0.06 5,028
Calculus or higher 32.20 0.16 5,922
Missing HS math course 15.35 0.03 4,799
Excluded from HS Trig+ sample 13.96 0.02 36,019
In HS Trig+ sample 26.51 0.10 11,356

Notes: Means are reported in columns 1 and 2. Sample sizes are reported in column 3. The sample
is restricted to Florida high school graduates in 1999/2000 who attended in-state public colleges and
universities in fall 2000.

Taken together, these two samples allow us to explore the impact of FSAG eligi-
bility for different groups of students. In Table 2, we present descriptive statistics
for the full sample of students (column 1) and compare them to all STEM-prepared
graduates (columns 2 and 4) and the subset of STEM-prepared graduates included
in our analytic samples (columns 3 and 5).11 There are clear differences between the
full census of public high school students and the subset in our estimation samples.
For instance, our analytic samples are more heavily female than the full sample
(59 and 61 percent versus 53 percent) and have greater minority representation (46
and 49 percent in the analytic samples versus 39 percent in the full sample). Stu-
dents in our conditioned samples also have considerably higher senior year GPAs

captures all students near the FSAG eligibility cutoff who, according to Florida Department of Education
guidelines, were prepared to undertake college-level math at the start of college. All appendices are
available at the end of this article as it appears in JPAM online. Go to the publisher’s website and use the
search engine to locate the article at http://onlinelibrary.wiley.com.
11 As discussed below, we restricted the samples in our analysis to students within $1,000 of the EFC
cutoff.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
146 / The Effects of Need-Based Grants on STEM attainment

Table 2. Summary statistics of the data.

(1) (2) (3) (4) (5)


College math sample HS Trig+ Sample
All public Florida
high school EFC ± EFC ±
seniors All students $1,000 All students $1,000

Female 0.53 0.58 0.59 0.57 0.61


White 0.62 0.63 0.54 0.64 0.52
Black 0.20 0.15 0.20 0.15 0.22
Hispanic 0.15 0.15 0.20 0.13 0.18
Other race 0.04 0.06 0.06 0.08 0.09
Age 17.92 17.82 17.83 17.81 17.81
(0.57) (0.49) (0.50) (0.48) (0.49)
[99,067] [20,400] [8,775]
EFC $6,889 $7,750 $1,570 $9,060 $1,560
($12,128) ($13,950) ($570) ($15,290) ($570)
[45,785]
AGI $43,662 $51,780 $28,380 $55,110 $28,630
($41,607) ($44,310) ($10,280) ($46,740) ($10,040)
[43,784] [20,028] [8,668]
12th Grade 2.84 3.10 3.04 3.23 3.18
GPA (0.75) (0.63) (0.64) (0.56) (0.55)
[57,021] [18,599] [8,879]
CPT math 59.43 92.81 91.62 89.27 87.18
score (27.02) (13.55) (13.28) (22.49) (23.31)
[36,328] [6,005] [886] [2,121] [345]
SAT math 528.73 554.04 536.98 579.55 557.1
score (97.27) (81.76) (75.63) (88.80) (85.94)
[35,080] [16,188] [2,102] [6,487] [877]
FSAG in $195 $221 $412 $242 $482
2000–2001 ($432) ($462) ($462) ($462) ($462)
[45,727]
Total grants $1,795 $2,834 $3,355 $3,602 $4,389
and loans in ($2,787) ($3,087) ($3,397) ($3,297) ($3,680)
2000–2001 [45,727]
Observations 101,094 20,738 2,834 8,907 1,283

Notes: Means are shown with standard deviations in parentheses and the number of observations in
brackets if less than the full sample. Columns 2 through 5 are restricted to students with non-missing
FAFSA information. EFC, expected family contribution toward college; AGI, parents’ adjusted gross
income; CPT, college placement test. Grade 12 GPA is weighted on a 4.5 scale.

(3.04 among students ready for college-level math and 3.18 for advanced high school
math-takers versus 2.84 in the full sample). By construction, we also observe higher
achievement on college entrance examinations among the conditioned samples. To
the extent that these observed differences correlate with interest in and proclivity
for STEM, our effect estimates likely demonstrate the impact of aid eligibility on
STEM attainment for low-income students who are most likely to consider pursuing
STEM majors in college.12

12 Because they have the highest math scores, and are therefore best positioned to pursue STEM fields
in college, a priori we might expect that the students in our second analytic sample would be most
responsive to the offer of need-based grant assistance.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 147

Empirical Strategy
We use an RD approach to estimate the causal effect of FSAG eligibility on whether
students pursued and completed courses and degrees in STEM fields. In earlier
work, Castleman and Long (2016) use a similar research design and demonstrate
that FSAG award offers led to substantial increases in overall credit completion and
degree attainment at Florida public institutions. In their analysis, the authors focus
on all high school seniors who filed a FAFSA and were eligible to receive FSAG
awards. In this paper, we focus on a subset of Florida high school graduates who
demonstrate academic readiness for postsecondary study in STEM fields and we
compare the STEM outcomes for students just below the EFC cutoff to students
who are just above the cutoff. The RD design therefore allows us to infer the effects
of FSAG award offers for students on the margin of grant eligibility (Murnane &
Willett, 2010; Shadish, Cook, & Campbell, 2002). We focus on intent-to-treat (ITT)
estimates and employ a “sharp” RD design (Imbens & Lemieux, 2008). This means
that we can directly interpret a jump in STEM outcomes at the FSAG cutoff as the
causal effect of FSAG eligibility for marginal students around the cutoff.
To estimate the causal effects of FSAG eligibility on STEM college outcomes we
fit the following OLS/LPM regression model:13

ST EM OU TC OMEi j = β0 + β1 EFCi j + β2 F SAGi j + β3 F SAGi j xEFCi j + ϒ AC ADi j


+ ρ DEMOGi j + SC HOOLi j + ∈ i j , (1)

where STEM_OUTCOME is one of several outcomes of interest corresponding to


STEM attainment in college for student i attending high school j in twelfth grade.
EFC measures students’ estimated family contribution to college and is centered at
the FSAG cutoff. FSAG is an indicator variable that takes on the value of “1” if stu-
dents are below the FSAG cutoff, and zero otherwise. The F SAGxEFC interaction
term allows the slope of the relationship between EFC and each outcome to vary
on either side of the eligibility cutoff. To increase the precision of our estimates,
we also include several controls in the model.14 ACAD is a vector of academic co-
variates, and DEMOG is a vector of demographic covariates. SCHOOL is a vector
of high school fixed effects that control for time-invariant school-specific (and, by
proxy, neighborhood-specific) effects on educational attainment. i j is a residual
error term, which we cluster at the high-school-level to adjust for the potential cor-
relation of residuals within school. In this model, parameter β2 is our coefficient
of interest and describes the causal effect of being just below the FSAG cutoff on
STEM attainment in college.
As indicated in equation (1), we incorporate a broad range of academic and demo-
graphic covariates into our analyses. We include measures of students’ senior year
high school GPA, their math and verbal SAT scores, whether students participated
in a gifted and talented program during high school, parents’ adjusted gross income
as reported on the FAFSA, and students’ gender, race/ethnicity, and age at expected

13 We present results from OLS/LPM models throughout the paper as a conservative estimate of the
impact of FSAG eligibility on STEM attainment. Tobit models return estimates that are approximately
one credit larger across all credit outcomes. We also modeled STEM degree receipt using a logistic
regression specification, which also returned slightly larger but substantively similar estimates to those
that we report.
14 Results from specifications without covariates are presented in Appendix Table B5 and are simi-
lar in magnitude to our main results. All appendices are available at the end of this article as it ap-
pears in JPAM online. Go to the publisher’s website and use the search engine to locate the article at
http://onlinelibrary.wiley.com.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
148 / The Effects of Need-Based Grants on STEM attainment

high school graduation.15 We also include a dummy variable that indicates whether
students were eligible for a Bright Futures scholarship award to account for the
potential effect of other financial aid eligibility on STEM attainment.16
The selection of bandwidth is a critical decision in RD analyses: the wider the
bandwidth, the greater the statistical power to detect an effect. However, a wider
bandwidth also makes it more difficult to model the functional form of the rela-
tionship between the forcing variable (EFC) and the outcome of interest (Imbens
& Lemieux, 2008). In our analysis, we employed the Calonico, Cattaneo, & Titiunik
(2014) method for bandwidth selection on the full analytic samples, which returned
optimal widths ranging from 0.8 to 1.2 across outcomes and samples. For sample
consistency, we estimate our main results on a subset of students with EFCs be-
tween $590 and $2,590, equivalent to the modal CCT bandwidth selection of plus or
minus $1,000 around the FSAG eligibility cutoff. To examine the sensitivity of our
results to the choice of bandwidth, we refit our models using varying window widths
and separately test polynomial specifications of the relationship between EFC and
each outcome. We describe these sensitivity analyses in more detail below.
There are two limitations to the external validity of our analyses. First, our in-
ferences are limited to the effect of FSAG eligibility on whether students pursue
STEM courses and degrees at Florida public or private universities. As we mention
above, students who enrolled in out-of-state institutions do not appear in our data.
This missing data issue is unlikely to alter our substantive findings since, in their
previous work, Castleman and Long (2016) find no evidence that FSAG award offers
impacted enrollment at in-state private institutions or induced students attending
out-of-state schools to enroll at public, in-state colleges and universities. Because
we focus on low-income students and the main effects of the grant program were
concentrated at the public institutions we observe in the data set, we are likely
capturing the impacts on STEM attainment for the vast majority of target students.
Second, given our sample restrictions, the students in our analytic samples rep-
resent only a fraction of college-bound low-income students. However, they likely
comprise those most interested in and prepared to pursue STEM majors in col-
lege, and for whom affordability may determine whether they are able to enroll and
persist in STEM programs of study. Furthermore, because Florida is demographi-
cally and socioeconomically representative of other large states in the United States,
our results should also inform how need-based financial aid impacts STEM credit
and degree attainment among academically qualified low-income students at other
public institutions nationwide.

Testing for Statistical Equivalence around the Cutoff


The key assumption underlying our research design is that students immediately on
either side of the FSAG eligibility cutoff are equivalent, on average, on all observed

15 The students in our samples have complete information for all academic and demographic covariates
with the exception of SAT scores; approximately 30 percent of students did not take the college entrance
examination. We include those students in our samples to increase the precision of our estimates and we
predict missing scores using the full set of other baseline characteristics. In all our results, we present
estimates from multiple imputation regressions that account for uncertainty in the imputed test scores
of students who did not take the examination.
16 BFS students were required to meet annual academic benchmarks to renew their scholarships. To
explore whether these requirements created a perverse incentive among BFS awardees to enroll in fewer
STEM courses, we examined the number of STEM credits BFS students attempted in their second
semester among those on either side of the GPA renewal threshold after first semester. The results,
presented in Table B2, suggest that BFS scholars on the margin of renewing their awards did not avoid
enrolling in STEM courses to maintain merit aid.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 149

and unobserved dimensions. That is, we expect that students on either side of the
cutoff differ only in terms of whether they are eligible for the FSAG grant. This
assumption implies that we should observe a smooth density of students across the
EFC cutoff. Spikes in the fraction of students just below the cutoff could indicate
sorting bias and violate the equality assumption upon which our identification strat-
egy relies (Urquiola & Verhoogen, 2009). In the case of the FSAG award, sorting does
not appear to be a major concern. The EFC cutoff values used to determine FSAG
eligibility were not publicly reported by the Florida Department of Education, and
institutional financial aid websites also did not make this information available to
students and their families. While Florida statutes from the time period reference
an EFC threshold beyond which students would not be eligible for the FSAG, an
exhaustive search found only one document from the Florida Postsecondary Plan-
ning Commission (2001) that reports the actual cutoff value. Given the difficulty
low-income students often experience in completing financial aid applications and
the effort required to calculate the EFC, it is unlikely that the students in our study
strategically positioned themselves below the eligibility cutoff in order to receive
award offers.
To empirically test whether sorting is a concern in our study, we employ
McCrary’s (2008) density test around the EFC cutoff. In Figure 2, we present graphi-
cal results of these tests. A statistically significant spike in the density of students on
either side of the cutoff would suggest that students were strategically manipulating
their EFC values to be just above or below the cutoff. While there are small visual
discontinuities at the cutoff for the two samples, neither is statistically significant,
as evidenced by the overlapping 95 percent confidence intervals on either side of
the vertical line positioned at the threshold. We therefore fail to reject that stu-
dents in our analytic samples did not strategically position themselves around the
EFC threshold, reinforcing that endogenous sorting does not appear to be a major
concern in our analysis.
To further test the assumption of baseline equivalence around the EFC cutoff, we
fit a version of equation (1) in which the dependent variable is one of several base-
line student characteristics and all other student-level academic and demographic
covariates are excluded. If students are equal in expectation on either side of the
cutoff, then we should not observe statistically significant differences on student
background measures at the cutoff. We performed this analysis within two win-
dows around the FSAG cutoff—a narrow plus or minus $250 around the cutoff, as
well as plus or minus $1,000, which corresponds to the bandwidth we use to estimate
our main results—since we expect students to differ on observed and unobserved
dimensions the further we move away from the cutoff. We present the results of
these baseline equivalence tests in Table 3. It appears that the students just below
the cutoff in our sample come from families with slightly higher family income than
students just above the cutoff ($30,000 to $31,500 vs. $28,500), but we otherwise
find no systematic evidence that students differ on other observable dimensions.17
We also examined whether the full set of student covariates jointly predict whether
students are eligible for the FSAG. The p-value associated with the F-test for joint
significance is presented in the last row of Table 3. Across both samples, we fail
to reject the null hypothesis that students on either side of the FSAG eligibility

17 Column 4 of Table 3 also shows a significant difference on high school GPA in the Trig+ sample within
the plus or minus $1,000 window, although this difference disappears within the plus or minus $250
window and is not distinguishable from zero in the college math sample. We also find no evidence of
differences at the cutoff with respect to: (1) college entrance examination scores, (2) enrollment in gifted
programs during high school, and (3) eligibility for state merit aid. We therefore find little evidence of
differences in academic performance at entry on either side of the cutoff.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
150 / The Effects of Need-Based Grants on STEM attainment

Notes: EFC is divided by $1,000 and centered at the FSAG cutoff. The density function of EFC was
estimated using McCrary’s (2008) test for manipulation of the forcing variable in regression discontinuity
analyses.

Figure 2. Density of Observations within Plus or Minus $1,000 of the FSAG Eligi-
bility Cutoff.

threshold are statistically equivalent within the narrow window of plus or minus
$250. These findings substantiate our use of an RD design to estimate causal effects
for students immediately on either side of the eligibility cutoff. Still, the p-value in
column 4 is significant at the 10 percent level (p equals 0.061). Observed differences
at the cutoff in the high school math sample might bias our results upwards if the

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 151

Table 3. Test for baseline equivalence around the FSAG eligibility cutoff.

(1) (2) (3) (4)


College math sample HS Trig+ Sample
EFCs window around
FSAG eligibility cutoff ± $250 ± $1,000 ± $250 ± $1,000

Female −0.172 −0.007 −0.150 −0.015


[0.104] [0.042] [0.225] [0.072]
Black 0.017 −0.031 0.198 −0.054
[0.090] [0.033] [0.160] [0.057]
Hispanic −0.024 −0.001 −0.061 −0.003
[0.077] [0.029] [0.157] [0.051]
Other race 0.004 0.021 0.018 0.008
[0.054] [0.020] [0.118] [0.044]
Age 0.099 0.005 0.009 −0.025
[0.141] [0.043] [0.322] [0.076]
Parents’ adjusted gross 1.845 1.572* 2.247 2.923**
income (AGI) [2.107] [0.848] [4.794] [1.361]
Eligible for bright futures −0.090 −0.006 −0.237 −0.103
scholarship [0.115] [0.042] [0.212] [0.068]
Enrolled in gifted/talented −0.007 0.019 0.025 −0.002
program [0.045] [0.018] [0.088] [0.030]
High school senior year 0.126 0.078 0.004 0.149**
GPA [0.141] [0.056] [0.269] [0.069]
SAT math −6.654 −4.639 −32.622 0.816
[18.361] [7.105] [39.872] [12.980]
SAT verbal −12.505 −6.893 −36.264 −5.982
[20.528] [7.725] [43.228] [13.360]

Observations 739 2,834 346 1,283


P-value on F-test for joint 0.710 0.353 0.986 0.061
significance
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Each cell reports the coefficient estimate from an OLS/LPM regression of the student characteristic
on the FSAG eligibility indicator. All models control for EFC (centered at the cutoff), the interaction of
the FSAG eligibility indicator and EFC, and high school fixed effects. Standard errors, clustered by high
school, are reported in brackets. SAT math and verbal scores have been imputed where missing using
multiple imputations. The F-test for joint significance tests whether the full set of baseline characteristics
jointly explain variation in whether students were just above or below the FSAG eligibility cutoff.

full set of covariates we include fails to account for unobserved differences. In all
of our results, we therefore view the high school math sample as providing upper
bound estimates of the impact of FSAG eligibility on students’ decisions to pursue
and complete postsecondary study in STEM disciplines.

RESULTS

First-Stage Effects on Financial Aid Receipt


We begin our presentation of results by examining the effect on financial aid receipt
at the FSAG eligibility cutoff. Importantly, the FSAG award does not appear to
have crowded out other forms of federal, state, or institutional grant aid. In Table
4, we present estimates of the aid packages that students received at the FSAG
eligibility cutoff in the 2000/2001 academic year. The results in columns 1 through 3
show that students just below the FSAG cutoff received approximately $700 more in

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
152 / The Effects of Need-Based Grants on STEM attainment

Table 4. The effect of FSAG eligibility on financial aid receipt in 2000/2001.

(1) (2) (3) (4) (5)


Total aid
Total grant Grant aid from all
FSAG aid minus FSAG Loan aid sources

Panel A: College math sample


Eligible for FSAG 673.97*** 694.92*** 20.95 −145.04* 549.88**
[38.45] [198.15] [186.31] [84.60] [234.09]
EFC (centered at FSAG −12.14 −457.94* −445.80* −144.99 −602.93**
cutoff) [31.38] [236.41] [230.25] [126.36] [292.84]
FSAG × EFC 33.37 398.88 365.50 147.67 546.54
[71.12] [404.25] [375.44] [163.41] [467.36]

R2 0.449 0.403 0.401 0.161 0.371


Outcome mean above 0 2580 2580 358 2938
FSAG cutoff
N 2,834
Panel B: HS Trig+ sample
Eligible for FSAG 673.05*** 735.40* 62.35 −105.65 629.75
[67.84] [423.58] [408.00] [207.08] [501.40]
EFC (centered at FSAG −92.67 −520.41 −427.74 −176.61 −697.02
cutoff) [63.54] [569.89] [559.77] [255.58] [673.47]
FSAG × EFC −15.25 422.23 437.49 311.07 733.30
[125.69] [814.13] [780.06] [297.04] [918.23]
R2 0.568 0.452 0.453 0.252 0.416
Outcome mean above 0 3430 3430 496 3925
FSAG cutoff
N 1,283
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results
are from multiple imputation OLS specifications estimated with an EFC window plus or minus $1,000
around the FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; student age, and whether the student was eligible for the
Florida Bright Futures Scholarship. All models also include school fixed effects and a constant.

FSAG and total grant aid than students just above the cutoff.18,19 There is suggestive
evidence in column 4 that the FSAG offer modestly displaced student borrowing.
In the college math sample, students just below the cutoff borrowed $145 less to pay

18 There are two reasons why the estimates in column 1 are less than the statutory amount of $1,300.
First, some students included in our samples did not enroll in college and therefore did not receive
FSAG awards. Second, students who enrolled in college were only eligible to receive awards if they filed
the FAFSA by March 1. Although we do not observe students’ FAFSA filing date in our data, analyses
of ELS:2002 survey data suggest that fewer than 40 percent of low-income students nationally file by
March 1.
19 Analogous estimates of effects on cumulative aid received through six years are reported in Appendix
Table B3. Students just below the FSAG cutoff in year one received $786 to $921 more in FSAG through
six years. Most, if not all, of the effect on cumulative FSAG receipt is operating through the effect in year
one, which is consistent with the assumption of the RD strategy that students just below the eligibility
cutoff in year one would be equally likely to fall just above the threshold in later years. All appendices
are available at the end of this article as it appears in JPAM online. Go to the publisher’s website and use
the search engine to locate the article at http://onlinelibrary.wiley.com.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 153

Notes: EFC is divided by $1,000 and centered at the FSAG cutoff. Each point represents the mean of
the dependent variable within a $100 bin of EFC. The trend lines present uncontrolled, locally-linear
regressions within $1,000 of the aid eligibility cutoff.

Figure 3. Relationship between EFC and Selected Outcomes, with Locally Linear
Regressions Fit on Either Side of the FSAG Cutoff.

for college in 2000/2001, and this difference is significant at the 10 percent level. As
a result, the net increase in total financial aid that FSAG-eligible students received
was slightly below the FSAG award amounts. However, the relevant counterfactual
is that FSAG eligibility increased grant aid to students without affecting Pell Grant
awards or other sources of need- or merit-based funding.

Graphical Results
In Figure 3, we present graphical descriptions of the bivariate relationship between
the forcing variable (EFC) and our STEM credits completed and degree receipt
outcomes. To capture differences over the full time span of our data set, we focus
on the bivariate relationship between EFC and the outcome of interest after seven
years. Each point on the graph reports the mean value of the dependent variable
within a $100 EFC bin, where EFC has been centered at the FSAG cutoff. We have
also superimposed linear regression lines onto the scatter plots that capture secular
trends in the bivariate relationship between EFC and the STEM outcome on either
side of the eligibility cutoff.
By visual inspection, it appears that FSAG eligibility has a positive impact on
STEM credit completion. As shown in panel A of Figure 3, students who are prepared
for college-level math, based on their CPT or SAT scores, and who fall just below

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
154 / The Effects of Need-Based Grants on STEM attainment

the cutoff, appear to have accumulated 21 STEM credits through seven years of
college, which is three more credits than their peers above the FSAG threshold. The
credit attainment gap is more pronounced in panel B, where FSAG-eligible students
who completed trigonometry or higher during high school earned approximately
seven additional STEM credits compared to their peers just above the cutoff. Given
that FSAG-ineligible students completed 20 STEM credits through seven years of
college, this jump at the cutoff represents a 35 percent increase over the control
group mean and 9 percent of the average number of STEM credits completed by
students who earned bachelor’s degrees in STEM fields.
It is less obvious whether FSAG eligibility had an impact on STEM degree attain-
ment from the graphical results in Figure 3. Although the fitted lines indicate a small
gap at the cutoff that is suggestive of FSAG award offers increasing STEM degree
attainment by 2 to 3 percentage points, many of the data points lie far from the line
on both sides of the cutoff. In summary, these visual illustrations suggest that FSAG
eligibility had a positive effect on STEM credit attainment and an ambiguous effect
on bachelor’s degree attainment in STEM fields for academically prepared students
at the FSAG eligibility cutoff.

RD Analysis: The Effects on STEM Credit and Degree Attainment


We now turn to the results of fitting our statistical models to the data, which largely
confirm the conclusions from the graphical analyses above. In Table 5, we present
estimates of the effect of FSAG eligibility on cumulative STEM credits attempted
and completed. The first two columns present short-term impacts through one
year following high school graduation. Columns 3 and 4 present outcomes through
three years of postsecondary study for students who seamlessly transitioned from
high school to college. Columns 5 and 6 present long-term outcomes through seven
years of college and are analogous to the outcomes presented graphically above.
Across all time horizons, our estimates of the impact of FSAG award offers on the
cumulative number of STEM credits attempted are positive, but lack precision and
are mostly indistinguishable from zero. The results therefore provide weak evidence
that additional need-based aid encouraged FSAG-eligible students to enroll in more
STEM courses, although we cannot rule out that award offers had no impact on
students’ course enrollment decisions.
We find larger and more conclusive impacts on STEM credit completion. For
students who were ready for college-level math, we estimate that FSAG award of-
fers increased STEM course completion by 0.7, 2.3, and 3.7 credits through years
one, three, and seven, respectively. These effects, though small in absolute terms,
represent large relative gains of 16 to 20 percent because the average number of
STEM credits completed among FSAG-ineligible students is small. We estimate
the largest credit completion effects, which increase to 1.9, 4.7, and 7.3 credits in
the short-, medium-, and long-term, respectively, among students who completed
trigonometry or higher in high school. For this subgroup, FSAG-eligible students
earned 34.5 percent more STEM credits through seven years compared to their
peers just above the aid eligibility cutoff. In other words, students just below the
cutoff were approximately one to two courses ahead of students just above the cutoff
after three years, and they maintained this margin seven years following high school
graduation.
In the final column of Table 5, we present impact estimates of an FSAG award of-
fer on whether students earned bachelor’s degrees in STEM fields within seven
years of high school graduation. For students with math scores above the col-
lege placement threshold, the coefficient on FSAG implies a 2.7 percentage point
(63 percent) increase in STEM degree attainment among FSAG-eligible students.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 155

Table 5. The effect of FSAG eligibility on cumulative STEM credits and BA/BS degrees earned
in STEM fields.

(1) (2) (3) (4) (5) (6) (7)


Through Year 1 Through Year 3 Through Year 7

STEM
STEM STEM STEM credits STEM STEM BA/BS
credits credits credits com- credits credits degree in
attempted completed attempted pleted attempted completed STEM

Panel A: College math sample (N = 2,834)


Eligible for 0.514 0.722* 1.600 2.254** 2.701 3.705** 0.027*
FSAG [0.471] [0.413] [1.311] [1.113] [2.076] [1.800] [0.015]
EFC (centered 0.553 0.439 1.756 1.577 1.165 1.219 0.022
at FSAG [0.558] [0.491] [1.553] [1.312] [2.462] [2.054] [0.021]
cutoff)
FSAG × EFC −0.574 −0.106 −1.360 −0.137 1.061 2.642 0.008
[0.719] [0.677] [1.996] [1.847] [3.185] [2.793] [0.026]
R2 0.191 0.208 0.202 0.216 0.195 0.204 0.191
Outcome 5.63 4.31 15.99 12.41 23.55 18.27 0.043
mean above
FSAG
cutoff
Panel B: HS Trig+ Sample (N = 1,283)
Eligible for 1.805* 1.947** 3.615 4.721** 5.456 7.259** 0.028
FSAG [0.940] [0.837] [2.514] [2.164] [4.115] [3.533] [0.032]
EFC (centered 2.105* 1.723 4.555 4.785 3.016 4.389 0.040
at FSAG [1.219] [1.080] [3.371] [2.953] [5.043] [4.379] [0.044]
cutoff)
FSAG × EFC −1.019 −0.300 −4.015 −3.430 0.340 0.153 −0.055
[1.531] [1.431] [4.163] [3.801] [6.588] [5.708] [0.051]
R2 0.319 0.326 0.322 0.330 0.300 0.310 0.290
Outcome 6.51 5.28 18.06 14.44 26.47 20.98 0.059
mean above
FSAG
cutoff
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results are
from multiple imputation OLS/LPM specifications estimated with an EFC window plus or minus $1,000
around the FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; student age, and whether the student was eligible for the
Florida Bright Futures Scholarship. All models also include high school fixed effects and a constant.

The estimate of 2.8 percentage points (47 percent) is similar for students in the
high school trigonometry sample. The estimate in panel A is significant at the
10 percent level and the estimate in panel B is not significant. The results there-
fore provide weak evidence that award offers increased STEM degree attainment,
although once again we cannot rule out that the effects are due to sampling
error.
It is possible that the results in Table 5 merely reflect that FSAG offers drew new
college-ready students in our samples into college. In Table 6, we examine whether
this explains the effects on STEM attainment. The results provide no evidence
that FSAG award offers increased overall enrollment at public colleges in Florida

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
156 / The Effects of Need-Based Grants on STEM attainment

Table 6. The effect of FSAG eligibility on enrollment and the share of credits completed in
STEM fields through seven years following high school graduation.

(1) (2) (3) (4) (5) (6)


Students enrolled in fall
All students 2000

Enrolled at
in-state Share of Share of Share of
private Enrolled at credits credits credits
Enrolled at college (FRAG UF, FSU, completed completed completed
any college proxy) or USF in STEM in STEM in STEM

Panel A: College math sample


Eligible for −0.006 0.003 0.086*** 0.046** 0.047** 0.048***
FSAG [0.025] [0.025] [0.032] [0.018] [0.019] [0.018]
EFC (centered 0.003 −0.015 0.064 0.034 0.021 0.018
at FSAG [0.028] [0.030] [0.041] [0.022] [0.021] [0.021]
cutoff)
FSAG × EFC −0.011 −0.001 0.009 0.003 0.016 0.021
[0.036] [0.042] [0.061] [0.029] [0.028] [0.028]
R2 0.182 0.149 0.258 0.188 0.243 0.265
Outcome 0.926 0.0848 0.242 0.223 0.217 0.217
mean above
FSAG
cutoff
Observations 2,834 2,834 2,834 2,834 2,151 2,151
Panel B: HS Trig+ Sample
Eligible for −0.020 0.007 0.098 0.078** 0.071** 0.081**
FSAG [0.045] [0.038] [0.061] [0.033] [0.035] [0.036]
EFC (centered −0.004 −0.019 0.100 0.055 0.059 0.072
at FSAG [0.054] [0.044] [0.076] [0.041] [0.044] [0.046]
cutoff)
FSAG × EFC 0.036 −0.006 −0.051 −0.020 −0.058 −0.065
[0.077] [0.073] [0.103] [0.057] [0.059] [0.062]
R2 0.315 0.300 0.362 0.321 0.384 0.426
Outcome 0.893 0.0911 0.324 0.235 0.232 0.232
mean above
FSAG
cutoff
Observations 1,283 1,283 1,283 1,283 971 971
*** p < 0.01; ** p < 0.05; * p < 0.10.
Note: Robust standard errors, clustered at the high school level, are shown in brackets. All results are
from multiple imputation OLS/LPM specifications estimated with an EFC window plus or minus $1,000
around the FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; student age, and whether the student was eligible for the
Florida Bright Futures Scholarship. All models also include high school fixed effects and a constant.
Results in column (6) also include college fixed effects.

(column 1), nor do they suggest that award offers induced students to attend public
instead of private institutions in Florida (column 2). Although estimated imprecisely,
the point estimates on both outcomes are negative or near zero in both samples.
In column 3, we examine whether FSAG award offers influenced the quality of
institution where students chose to attend. The results indicate that FSAG award
offers increased attendance at Florida State University, the University of Florida, or

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 157

the University of South Florida, the three public flagship universities in Florida in
2000/2001, by 9 to 10 percentage points (30 to 35 percent).
If the effects on STEM attainment are not driven by overall attendance gains,
we should observe a shift towards more intensive STEM course-taking at the
FSAG cutoff. We examine evidence for this in column 4 of Table 6, which shows
the proportion of total credits students completed in STEM fields through seven
years following high school completion. The overall STEM credit gains of 3.7 and
7.3 credits reported in column 6 of Table 5 represent increases in the share of
credits completed in STEM fields of 4.6 (24 percent) and 7.8 (33 percent) per-
centage points among students in the college math and high school trigonome-
try samples, respectively. To further examine whether these effects are mechani-
cally produced by selection into college, in column 5 we condition the samples on
students who entered college in fall 2000. The coefficients are similar in magni-
tude and significance in the conditioned samples, reinforcing that FSAG award
offers had a large influence on course selection among inframarginal college-
goers.20
As suggested by the literature on academic mismatch and our results on flagship
attendance, one avenue through which FSAG eligibility might lead to improved
STEM outcomes is by inducing students to attend higher-quality institutions where
there are more STEM course offerings and more academic supports for students pur-
suing STEM programs of study (Arcidiacono, 2004; Roderick et al., 2008; Bowen,
Chingos, & McPherson, 2009).21 However, another possibility is that additional
grant aid lowers non-institutional obstacles to STEM attainment, perhaps by reduc-
ing the financial barriers to pursuing more costly courses or by enabling students to
substitute working for pay with more demanding coursework. To investigate which
of these mechanisms appears to be driving our results, we add college fixed effects
to the regression model in column 6 of Table 6.22 If FSAG eligibility raised STEM
attainment by inducing students to attend higher-quality institutions, then the mag-
nitude of the effect should attenuate when we limit our comparisons to students who
attended the same institutions. The estimates in column 6 (4.8 and 8.1 percentage
points in the college math and Trig+ samples, respectively, which represent cumu-
lative STEM credit increases of 4.7 and 11.5 credits in the enrollment-conditioned
samples), do not diminish with the addition of college fixed effects. The results there-
fore suggest that the FSAG effects on STEM attainment are primarily explained by
within-institution factors rather than which schools students chose to attend, de-
spite the fact that FSAG eligibility increased attendance at flagship institutions and

20 In Appendix Table B6, we present estimates of the effect of FSAG eligibility on non-STEM outcomes.
The estimates on credit attainment are negative or near zero in both samples. The estimates on degree
completion are suggestive of a positive effect in the College Math sample and no effect in the Trig+
sample. Although the STEM and non-STEM estimates are not statistically distinguishable, the results
in Table B6, alongside those in Tables 6 and B4, provide additional evidence that STEM attainment
gains are not due to broad enrollment effects. All appendices are available at the end of this article as it
appears in JPAM online. Go to the publisher’s website and use the search engine to locate the article at
http://onlinelibrary.wiley.com.
21 Our analysis of STEM degree rates by institution also confirms this reasoning. Among high school
graduates in 1997/1998 who attended a Florida public university, the University of Florida, a flagship
campus and the only public university with a “highly competitive” Barron’s ranking, reported the top
STEM degree rate of 12.2 percent, more than 3 percentage points above the second highest-performing
institution (Florida Agricultural & Mechanical University).
22 We also examined whether FSAG eligibility increased the probability of summer enrollment and the
number of STEM courses attempted and completed in summer terms, on the hypothesis that award
offers may have allowed students to distribute more rigorous coursework throughout the academic year
or re-take courses they had previously failed. We find no evidence that the STEM attainment gains are
driven by changes in summer course-taking behavior.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
158 / The Effects of Need-Based Grants on STEM attainment

exposed students to more STEM options. This result is consistent with recent work
by Arcidiacono, Aucejo, & Hotz (2016), who find that students at the margin of at-
tending more selective public universities in California would have been more likely
to graduate in the sciences had they attended less selective campuses. It is therefore
possible that attending higher-quality institutions is not a channel through which
FSAG eligibility increased STEM attainment because the exposure effect was offset
by the rigor of coursework at flagship institutions, which presented an additional
barrier to STEM attainment for marginal enrollees.

Robustness Checks
We perform a number of robustness tests to validate our results. We first address
the possibility that our results are sensitive to the EFC window around which we
conducted our analysis by re-fitting our regression models using a variety of window
widths. To the extent that our results capture the true causal effect of FSAG eligibil-
ity, the parameter estimates associated with FSAG eligibility should be robust to the
choice of bandwidth selected. For illustrative purposes, in Table 7 we present the
effect of FSAG eligibility on STEM credit accumulation through seven years using
various window widths. Moving from left to right, each of the first five columns
presents the results of fitting equation (1) using progressively wider window widths.
We observe very little fluctuation in the coefficients on FSAG for both estimation
samples, which suggests that our main results are robust to the choice of window
width.
In addition to bandwidth selection, another possible concern is that our results
may be sensitive to the functional form of the relationship between EFC and our
STEM outcomes of interest. Misspecifying the functional form of the forcing vari-
able in RD models can yield particularly problematic findings, as the relatively
small number of observations within the analytic windows can cause outlying ob-
servations just above or below the cutoff to have disproportionate influence on
the estimated slope coefficients. This, in turn, could lead to biased effect esti-
mates in our models (Murnane & Willett, 2010). In all of our analyses we mod-
eled a locally linear relationship between the forcing variable and the outcome
of interest within $1,000 of the FSAG eligibility cutoff. To test whether we cor-
rectly specified the model, in columns 6 through 8 of Table 7 we add polynomial
EFC terms to equation (1) and include two-way interactions between each of these
terms and the FSAG eligibility indicator variable. None of the polynomial specifi-
cations are statistically significant. In column 9, we follow the suggestion of Cat-
taneo, Titiunik, & Vazquez-Bare (2017) and estimate the effect on STEM credit
attainment using a kernel-weighted nonparametric local-linear specification. The
point estimates remain significant at the 10 and 5 percent levels in panels A and
B, respectively, and are of similar magnitude to our main estimates, reproduced
in column 3 for ease of comparison. Taken together, the results in Table 7 sug-
gest that within our chosen bandwidth of plus or minus $1,000 we have correctly
specified a locally linear relationship between EFC and cumulative STEM credit
completion.23
As a final robustness test, we consider the possibility that our results are not cap-
turing true causal effects, but rather idiosyncratic fluctuations in the data at the

23 We conducted similar analyses for all outcomes with statistically significant effects and reach the
same conclusions: (1) the polynomial EFC terms and the interactions between FSAG and the polynomial
terms are not necessary within the plus or minus $1,000 analytic window, and (2) the estimates are
robust to estimating effects using nonparametric local-linear specifications.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
Table 7. Robustness of the estimated effect on cumulative STEM credit completion within seven years to differing window widths around the
FSAG cutoff and specification of the functional form.

(1) (2) (3) (4) (5) (6) (7) (8) (9)


EFC window around the cutoff Checks of functional form

± $800 ± $900 ± $1,000 ± $1,100 ± $1,200 EFC window = ±$1,000

Panel A: College math sample


Eligible for FSAG 3.354 3.561* 3.705** 3.897** 2.857* 2.327 3.575 6.050 3.011*
[2.118] [1.906] [1.800] [1.679] [1.637] [2.617] [3.451] [3.721] [1.711]
EFC (centered at FSAG 2.637 2.463 1.219 0.346 0.621 7.995 19.050 50.338
cutoff) [3.352] [2.562] [2.054] [1.673] [1.530] [8.133] [20.761] [39.445]
FSAG × EFC −0.705 0.071 2.642 4.976** 1.294 −18.856 −25.993 −39.032
[4.519] [3.350] [2.793] [2.317] [2.265] [12.005]
√ [30.997]
√ [59.233]

EFC2 and (FSAG × EFC)2 √ √
EFC3 and (FSAG × EFC)3 √
EFC4 and (FSAG × EFC)4
Observations 2,272 2,573 2,834 3,102 3,368 2,834 2,834 2,834 2,834
P-value on F-test of EFC 0.384 0.549 0.496
polynomials
P-value on F-test of FSAG 0.440 0.885 0.729
× EFC interactions
Panel B: HS Trig+ Sample
Eligible for FSAG 7.580* 8.201** 7.259** 8.208** 7.267** 5.235 6.438 12.948* 6.782**
[4.385] [3.943] [3.533] [3.299] [3.172] [5.061] [6.127] [6.830] [2.722]
EFC (centered at FSAG 7.441 8.176 4.389 4.010 5.286* 20.997 49.877 116.161
cutoff) [6.913] [5.530] [4.379] [3.383] [2.860] [16.346] [46.401] [75.980]
FSAG × EFC −5.240 −5.371 0.153 2.942 −3.279 −46.708** −89.887 −91.368
[9.329] [7.273] [5.708] [4.530] [4.332] [23.154]
√ [71.136]
√ [115.590]

EFC2 and (FSAG × EFC)2 √ √
EFC3 and (FSAG × EFC)3 √
EFC4 and (FSAG × EFC)4

Journal of Policy Analysis and Management DOI: 10.1002/pam


The Effects of Need-Based Grants on STEM attainment / 159

Published on behalf of the Association for Public Policy Analysis and Management
Table 7. Continued.

(1) (2) (3) (4) (5) (6) (7) (8) (9)


EFC window around the cutoff Checks of functional form

± $800 ± $900 ± $1,000 ± $1,100 ± $1,200 EFC window = ±$1,000

Observations 1,035 1,157 1,283 1,408 1,525 1,283 1,283 1,283 1,283
P-value on F-test of EFC 0.317 0.433 0.340
polynomials
P-value on F-test of FSAG 0.521 0.821 0.639
× EFC interactions

Journal of Policy Analysis and Management DOI: 10.1002/pam


*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. Results in columns (1) through (8) are from multiple imputation OLS
specifications that include the following covariates: race dummy variables (black, Hispanic, and other race/ethnicity); female dummy variable; high school senior
year GPA (weighted 4.5 scale); SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented program; parental adjusted
gross income; student age, and whether the student was eligible for the Florida Bright Futures Scholarship. All models also include school fixed effects and a
constant. Results in column (9) are from a triangular kernel-weighted nonparametric local-linear specification that includes the same set of controls.
160 / The Effects of Need-Based Grants on STEM attainment

Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 161

Table 8. Falsification test for whether estimated effects of FSAG eligibility on STEM credit
completion within seven years are unique to the actual FSAG cutoff.

(1) (2) (3) (4)


Eligibility cutoff at EFC = $1,590 (actual) $1,090 $2,090 $2,590

Panel A: College math sample


Eligible for FSAG 3.705** −0.729 0.376 1.127
[1.800] [1.704] [1.756] [1.933]
EFC (centered at FSAG cutoff) 1.219 −2.391 1.440 1.938
[2.054] [2.024] [2.472] [2.396]
FSAG × EFC 2.642 4.233 −3.329 −1.294
[2.793] [2.909] [3.145] [2.906]
Observations 2,834 2,988 2,673 2,519
R2 0.204 0.200 0.195 0.180
Outcome mean above FSAG cutoff 18.27 19.32 17.83 17.95
Panel B: HS Trig+ Sample
Eligible for FSAG 7.259** −1.536 −2.546 3.093
[3.533] [3.523] [3.971] [3.884]
EFC (centered at FSAG cutoff) 4.389 −6.784 4.168 4.266
[4.379] [4.536] [5.096] [4.935]
FSAG × EFC 0.153 6.456 −10.176 0.895
[5.708] [6.829] [6.508] [6.794]
Observations 1,283 1,355 1,232 1,121
R2 0.310 0.288 0.321 0.363
Outcome mean above FSAG cutoff 20.98 22.56 22.03 22.40
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results
are from multiple imputation OLS specifications estimated with an EFC window plus or minus $1,000
around the FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; student age, and whether the student was eligible for the
Florida Bright Futures Scholarship. All models also include school fixed effects and a constant.

EFC cutoff of $1,590. If this were the case, then we might also expect to detect in-
creases in STEM attainment at arbitrarily chosen points along the EFC distribution.
We conduct this falsification test by re-fitting equation (1) using a $1,000 window
around four arbitrarily selected EFC “cutoffs”: (1) at the actual eligibility cutoff of
$1,590 in column 1; (2) at $500 below the actual cutoff in column 2; (3) at $500
above the actual cutoff in column 3; and (4) at $1,000 above the actual cutoff in
column 4. We present the results of this test in Table 8. In columns 2 through 4, the
estimated “effect” of FSAG eligibility on STEM credit accumulation through seven
years is smaller in absolute magnitude than the actual estimate and not distinguish-
able from zero. This suggests that our estimates in column 1, and throughout our
main results, are detecting causal effects rather than random fluctuations in the
data around the EFC cutoff.

CONCLUSION
Increasing college STEM attainment among students with the interest and pre-
paredness to succeed in STEM courses promises to maximize private returns to
college and help to address the anticipated shortage of STEM workers in the labor
force. A barrier to STEM attainment largely overlooked to date is the financial cost
to pursuing study in those fields. Despite a wide array of policy initiatives at the

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
162 / The Effects of Need-Based Grants on STEM attainment

institutional, state, and federal levels, including financial aid for students pursuing
STEM fields, there has been little research investigating the causal impact of grant
aid on students’ credit accumulation and degree attainment in STEM fields. We add
to both the STEM and financial aid literatures by examining the effect of need-based
grant eligibility on credit and degree attainment in STEM fields.
Our results indicate that financial aid policy has a role to play in increasing STEM
attainment in college because students make cost-conscious decisions when choos-
ing what to study. Using an RD design, we find a positive effect of FSAG eligibility
on whether students with sufficient high school academic preparation accumulate
STEM credit. When we adjust our estimates into magnitudes per $1,000 of grant
aid, our results indicate that eligibility for a modest amount of additional grant
aid in the first year of college alone led to students accumulating between 2.9 and
5.6 additional credits after seven years, depending on the sample. While these im-
pacts are modest in absolute magnitude, they are quite large given that students
just below the cutoff at the start of college were equally likely to be above the cutoff
in subsequent years. In relative terms, the effects are also economically significant,
representing increases of 20 to 35 percent over the STEM credit accumulation of
students who were ineligible for the FSAG award in 2000/2001. We also find sug-
gestive evidence that award offers increased bachelor’s degree attainment in STEM
fields by 3 percentage points, although the degree effects are not precisely estimated
and we cannot rule out that award offers had no effect on STEM degree production.
However, if the effects on degree attainment are real, then the FSAG program
would appear to be a cost-effective investment towards increasing the production of
STEM degrees. On the assumption that 300 students (i.e., roughly 10 percent of the
college math sample) were sufficiently close to the cutoff and academically prepared
to study STEM, our results suggest that FSAG award offers induced approximately
eight more students (off a baseline of 13 students) to earn STEM degrees. Since
FSAG-eligible and FSAG-ineligible students in our samples received average cumu-
lative awards of $816 and $1,602 through six years, the total expenditure on students
close to the cutoff (including the 300 STEM-ready students and 400 additional stu-
dents close to the cutoff who were not academically prepared to study STEM) was
$846,300 and the cost per student to produce eight more STEM graduates was
$105,788.24 Given the STEM wage premium among entry-level college-educated
workers, which was estimated at $10,000 in 2013, the social and private benefits of
FSAG would have exceeded the costs within 11 years (Carnevale, Cheah, & Hanson,
2015).25 In addition, because average earnings of STEM majors grow more quickly
than other majors over the course of a career, this simple “back of the envelope”
calculation indicates that FSAG likely has a positive rate of return even if the true
effects on degree attainment are smaller than those we estimate.
A natural question that emerges is the mechanism(s) by which FSAG eligibility
increased STEM attainment. We find that despite increasing attendance at institu-
tions with more STEM offerings, the effects of FSAG eligibility on STEM attainment
are primarily explained by within-institution factors rather than which schools stu-
dents chose to attend. Unfortunately, we were unable to pin down the precise mech-
anisms by which additional grant aid lowered non-institutional obstacles to STEM

24 The marginal number of graduates produced is derived from the estimates in column 7 of Table
5 (panel A). Cumulative FSAG receipt is reported in Table B3. As explained in Footnote 19, students
ineligible for FSAG awards in year one averaged $816 in FSAG aid over six years because some students
just below the eligibility cutoff initially became eligible for awards in later years. The total expenditure
on students close to the cutoff is therefore $846,300 (i.e., 700 × 0.5 × [$816 + $1, 602]).
25 Our estimates also imply that if the FSAG program had been targeted exclusively to STEM-ready
students, the cost per student would have been approximately $45,000 and the total cost would have
been recouped within five years.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 163

attainment. We examined, but find no evidence that award offers increased summer
enrollment, thereby enabling students to distribute STEM coursework throughout
the academic year or more easily re-take STEM courses they had previously failed.
We also investigated whether FSAG eligibility lowered the financial barriers to
pursuing more costly courses by examining variation in effects by STEM field of
study. While certain fields like engineering may require students to incur additional
laboratory and materials costs, other disciplines like mathematics or statistics are
less likely to have additional costs (beyond the costs associated with taking any
course, such as purchasing textbooks). As a result, we might expect to see pro-
nounced effects of need-based aid for students in more cost-intensive courses if
additional need-based aid makes those courses affordable. We find no evidence of
differential effects by STEM field to suggest that effects operated through this chan-
nel. However, because Florida did not charge differential tuition in the years we
observe in this study, it is possible that the students in our samples encountered
similar costs across STEM fields. In other contexts where students pay higher tu-
ition for more expensive programs of study, grant aid may enable cost-constrained
students to afford taking STEM courses. A further mechanism by which FSAG may
have positively impacted STEM outcomes is by reducing the hours students had to
work in college, thereby allowing them to devote more time to their coursework. We
were unable to empirically examine this hypothesis with our data, and it remains
another plausible mechanism that merits further inquiry.
In summary, our results suggest that expanding need-based aid programs may be
a sound investment, in tandem with efforts to address academic readiness and psy-
chosocial barriers to STEM attainment, towards addressing the mismatch between
the STEM training college students in the United States receive and the demands of
employers in the labor market. Our findings also reveal that the potential impact of
interventions designed to raise STEM attainment levels and lower socioeconomic
gaps may not be fully realized if financial barriers to pursuing STEM are overlooked.
More broadly, the results presented here indicate that financial barriers can affect
the academic choices students make after deciding to enroll. Unpacking precisely
how college affordability influences those decisions warrants further research to
help fulfill our labor market needs and to help students reap the fullest returns to
their postsecondary education.

BENJAMIN L. CASTLEMAN is an Assistant Professor of Education and Public Policy


at the Curry School of Education, University of Virginia, Ruffner Hall 262, P.O. Box
400265, Charlottesville, VA 22904 (e-mail: [email protected]).

BRIDGET TERRY LONG is the Saris Professor of Education and Economics at


Harvard Graduate School of Education, 6 Appian Way, Gutman 465, Cambridge,
MA 02138 (e-mail: [email protected]). She is also a Research Associate at
the National Bureau of Economic Research.

ZACHARY MABEL is a Doctoral Candidate at Harvard Graduate School


of Education, 6 Appian Way, Gutman 244, Cambridge, MA 02138 (e-mail:
[email protected]).

ACKNOWLEDGMENTS
The contents of this article were developed under grant R305A060010 from the U.S. Depart-
ment of Education, Institute of Education Sciences, and generous funding from the Bill and
Melinda Gates Foundation. The research was conducted in collaboration with the National
Center for Postsecondary Research (NCPR). We are grateful to Eric Bettinger, William Doyle,

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
164 / The Effects of Need-Based Grants on STEM attainment

Ilyana Kuziemko, Jonah Rockoff, Barbara Schneider, and anonymous reviewers for their
comments on earlier versions. All errors, omissions, and conclusions are our own.

REFERENCES
ACT. (2014). The condition of STEM 2014. Iowa City, IA: ACT.
Andrews, R. & Stange, K. (2016). Price regulation, price discrimination, and equality of
opportunity in higher education: Evidence from Texas. Cambridge, MA: National Bureau
of Economic Research, Working Paper No. 22901.
Arcidiacono, B., Aucejo, E., & Hotz, V. (2016). University differences in the graduation of
minorities in STEM fields: Evidence from California. American Economic Review, 106,
525–562.
Arcidiacono, P. (2004). Ability sorting and the returns to college major. Journal of Economet-
rics, 121, 343–375.
Baum, S., Elliot, D., & Ma, J. (2014). Trends in student aid 2014. Trends in higher education
series. Washington, DC: The College Board.
Bowen, W., Chingos, M., & McPherson, M. (2009). Crossing the finish line: Completing college
at America’s public universities. Princeton, NJ: Princeton University Press.
Burning Glass Technologies. (2014). Real-time insight into the market for entry-level STEM
jobs. Boston, MA: Burning Glass Technologies.
Calonico, S., Cattaneo, M., & Titiunik, R. (2014). Robust nonparametric confidence intervals
for regression-discontinuity designs. Econometrica, 82, 2295–2326.
Carnevale, A., Cheah, B., & Hanson, A. (2015). The economic value of college majors. Wash-
ington, DC: Center on Education and the Workforce, Georgetown University.
Castleman, B., & Long, B. (2016). Looking beyond enrollment: The causal effect of need-
based grants on college access, persistence, and graduation. Journal of Labor Economics,
34, 1023–1073.
Cattaneo, M., Titiunik, R., & Vazquez-Bare, G. (2017). Comparing inference approaches in
RD designs: A reexamination of the effect of Head Start on child mortality. Journal of
Policy Analysis and Management, 36, 643–681.
Cornwell, C., Lee, K., & Mustard, D. (2005). Student responses to merit scholarship retention
rules. Journal of Human Resources, 40, 895–917.
Cornwell, C., Lee, K., & Mustard, D. (2006). The effects of state-sponsored merit scholarships
on course selection and major choice in college. IZA Discussion Paper No. 1953, Institute
for the Study of Labor, Bonn, Germany.
Denning, J., & Turley, P. (2017). Was that smart? Institutional financial incentives and field
of study. Journal of Human Resources, 52, 152–186.
Dynarski, S., & Scott-Clayton, J. (2013). Financial aid policy: Lessons from research. Future
of Children, 23, 67–91.
Espinosa, L. (2011). Pipelines and pathways: Women of color in undergraduate STEM majors
and the college experiences that contribute to persistence. Harvard Educational Review,
81, 209–241.
Evans, B. (2017). SMART money: Do financial incentives encourage college students to study
science? Education Finance and Policy, 12, 342–368.
Fayer, S., Lacey, A., & Watson, A. (2017). STEM occupations: Past, present, and future. Spot-
light on statistics. Washington, DC: U.S. Department of Labor, Bureau of Labor Statistics.
Florida Department of Education. (2006). Articulation Coordinating Committee Meet-
ing Agenda, October 25, 2006. Retrieved May 21, 2014, from http://www.fldoe.org/
articulation/pdf/acc_102506ada.pdf.
Florida Postsecondary Education Planning Commission. (2001). State student financial aid
policy. Council for Education Policy Research and Improvement. Retrieved May 21, 2014,
from http://www.cepri.state.fl.us/pdf/flfinaid.pdf.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment / 165

Griffith, A. (2010). Persistence of women and minorities in STEM field majors: Is it the school
that matters? Economics of Education Review, 29, 911–922.
Hanson, G., & Slaughter, M. (2016). High-skilled immigration and the rise of STEM occu-
pations in the U.S. employment. Cambridge, MA: National Bureau of Economic Research,
Working Paper No. 22623.
Hastings, J., Neilson, C., & Zimmerman, S. (2013). Are some degrees worth more than others?
Evidence from college admission cutoffs in Chile. Cambridge, MA: National Bureau of
Economic Research, Working Paper No. 19241.
Higher Education Research Institute. (2010). Degrees of success: Bachelor’s degree com-
pletion rates among initial STEM majors. Los Angeles, CA: University of California-Los
Angeles.
Imbens, G., & Lemieux, T. (2008). Regression discontinuity design: A guide to practice.
Journal of Econometrics, 142, 615–635.
Kirkeboen, L., Leuven, E., & Mogstad, M. (2016). Field of study, earnings, and self-selection.
Quarterly Journal of Economics, 131, 1057–1111.
Lacey, A., & Wright, B. (2009). Occupational employment projections to 2018. Monthly Labor
Review. Washington, DC: U.S. Department of Labor, Bureau of Labor Statistics. Retreived
September 26, 2017, from http://www.bls.gov/opub/mlr/2009/11/art5full.pdf.
Langdon, D., McKittrick, G., Beede, D., Khan, B., & Doms, M. (2011). STEM: Good jobs now
and for the future. Economics and Statistics Administration Issue Brief #03–11. Washing-
ton, DC: U.S. Department of Commerce.
McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity
design: A density test. Journal of Econometrics, 142, 698–714.
Murnane, R., & Willett, J. (2010). Methods matter: Improving causal inference in educational
and social science research. Oxford, UK: Oxford University Press.
National Center for Education Statistics. (2002). Digest of education statistics 2002. Tables
182 and 203. Washington, DC: United States Department of Education.
National Center for Education Statistics. (2011). Postsecondary awards in sci-
ence, technology, engineering, and mathematics (STEM), by state: 2001 and
2009. NCES publication no. 2011–226. Retrieved September 26, 2017, from
https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid = 2011226.
Newman, J. (2014). State funding for need-based aid averages less than $500 per stu-
dent. The Chronicle of Higher Education, January 29, 2014. Retrieved September
26, 2017, from http://chronicle.com/blogs/data/2014/01/29/state-funding-for-need-based-
aid-averages-less-than-500-per-student/.
Noonan, R. (2017). STEM jobs: 2017 update. ESA issue brief #02–17. Washington, DC: U.S.
Department of Commerce, Economics and Statistics Administration, Office of the Chief
Economist.
Price, J. (2010). The effect of instructor race and gender on student persistence in STEM
fields. Economics of Education Review, 29, 901–910.
Roderick, M., Nagaoka, J., Coca, V., Moeller, E., Roddie, K., Gilliam, J., & Patton, D. (2008).
From high school to the future: Potholes on the road to college. Chicago, IL: Consortium
on Chicago School Research.
Rothstein, J., & Rouse, C. E. (2011). Constrained after college: Student loans and early-career
occupational choices. Journal of Public Economics, 95, 149–163.
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs
for generalized causal inference. Boston, MA: Houghton Mifflin.
Sjoquist, D., & Winters, J. (2015a). State merit aid programs and college major: A focus on
STEM. Journal of Labor Economics, 33, 973–1006.
Sjoquist, D., & Winters, J. (2015b). The effect of Georgia’s HOPE scholarship on college
major: A focus on STEM. IZA Journal of Labor Economics, 4, 1–29.
Stange, K. (2015). Differential pricing in undergraduate education: Effects on degree produc-
tion by field. Journal of Policy Analysis and Management, 34, 107–135.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
166 / The Effects of Need-Based Grants on STEM attainment

Steele, C., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of
African Americans. Journal of Personality and Social Psychology, 69, 797–811.
United States Census Bureau. (2011). State and county quick facts. Retrieved October 25,
2011, from http://quickfacts.census.gov/qfd/index.html.
United States Department of Homeland Security. (2012). STEM designated degree programs
list. Retrieved May 21, 2014, from http://www.ice.gov/sevis/stemlist.htm.
Urquiola, M., & Verhoogen, E. (2009). Class-size caps, sorting, and the regression disconti-
nuity design. American Economic Review, 99, 179–215.
Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and
postsecondary context of support. American Educational Research Journal, 50, 1081–1121.
Wiswall, M., & Zafar, B. (2015). Determinants of college major choice: Identification using
an information experiment. Review of Economic Studies, 82, 791–824.
Zhang, L. (2011). Does merit aid affect degree production in STEM fields? Evidence from
Georgia and Florida. Journal of Higher Education, 82, 389–415.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment

Appendix A: Predictions of Financial Aid Effects on Major Choice


Below, we specify a model of major choice that illustrates the theoretical uncer-
tainty of the impact of financial aid on STEM attainment. First, assume that students
choose a major that maximizes net expected value, with the value to student i from
pursuing major k at institution j captured by:
   
Vi jk = γ Ei jk + ρ Ei jk ∗ NC Ti j + τ Ii jk + π Ii jk ∗ NC Ti j + δi jk. (A.1)

In this model, net value (Vi jk) is a function of four factors: (1) net cost of tuition
to the student (NC Ti j ), which for simplicity is assumed to vary across schools but
is constant over majors within schools; (2) expected post-schooling earnings (Ei jk);
(3) investment costs required to pursue a given major (Ii jk); and (4) non-financial
benefits associated with the major (δi jk), such as the utility derived from studying
intrinsically enjoyable subject matter.
Assuming that a conditional logit model correctly specifies the probability of
choosing major k over an alternative, the marginal effect of lowering net tuition cost
on the share of students choosing major k at school j is:
dShar e jk   
= (ρ + π ) Shar e jk 1 − Shar e jk , (A.2)
dNC Tj

where ρ denotes how the effect of expected earnings on major choice changes when
net costs increase and π captures the analogous interaction effect with respect to
investment costs. On the one hand, it is assumed that ρ ≥ 0 since, all else being
equal, higher cost will increase pressure to pursue lucrative majors. On the other
hand, we expect that π ≤ 0. Higher costs will exacerbate financial constraints to
pursuing majors that demand relatively more investments of time and money. The
combined effect of lowering net cost on the intensive margin is therefore ambiguous
and depends on whether π > ρ, π < ρ, or π = ρ.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment

Appendix B: Additional Figures and Tables

Notes: EFC is divided by $1,000 and centered at the FSAG cutoff. Each point represents the mean of
the covariate within a $100 bin of EFC. The trend lines present uncontrolled, locally-linear regressions
within $1,000 of the aid eligibility cutoff.

Figure B1. Bivariate Relationship Between the Forcing Variable and Selected Co-
variates, with Locally Linear Regressions Fit on Either Side of the Cutoff.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment

Table B1. CPT and SAT math scores by highest math course completed in high school.

(1) (2) (3) (4)


All public Florida high school EFCs ±$1,000 of FSAG
seniors eligibility cutoff

CPT SAT CPT SAT

Algebra 1 41.87 452.20 42.40 442.00


(19.08) (95.66) (18.92) (100.46)
[1,387] [199] [140] [25]
Geometry 47.13 457.44 47.62 445.83
(20.51) (71.94) (20.24) (70.60)
[4,582] [3,592] [436] [391]
Algebra 2 59.15 459.32 58.86 432.39
(22.04) (93.99) (21.62) (86.64)
[4,788] [1,464] [426] [155]
Trigonometry 80.87 546.17 78.85 524.90
(22.71) (72.81) (24.50) (75.85)
[1,411] [2,574] [143] [255]
Pre-calculus 85.01 530.85 88.92 525.56
(22.67) (71.43) (21.23) (77.52)
[1,170] [2,792] [111] [239]

Notes: Means are shown with standard deviations in parentheses and the number of observations in
brackets.

Table B2. STEM credits attempted second semester by first semester GPA, among BFS
scholarship recipients.

(1) (2) (3) (4) (5) (6)

Below BF GPA renewal −0.290*** 0.318** 0.371** 0.334* 0.410* 1.053***


threshold [0.076] [0.145] [0.157] [0.178] [0.229] [0.336]
First semester GPA (centered 0.652*** 0.303** -0.053 −0.008 0.358
at BF renewal cutoff) [0.152] [0.153] [0.254] [0.432] [1.021]
Below BF GPA renewal −0.100 0.343 1.024** 1.312* 5.671***
threshold × GPA [0.243] [0.263]
√ [0.407]
√ [0.724]
√ [1.873]

Additional covariates √ √ √ √
School fixed effects
GPA window (1.00 = 1.00 1.00 1.00 0.75 0.50 0.25
optimal)
Observations 9,000 9,000 8,225 6,307 4,288 2,421
R2 0.002 0.005 0.135 0.139 0.162 0.230
Outcome mean above GPA 3.74 3.74 3.74 3.68 3.78 3.78
renewal cutoff
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results are
estimates from multiple imputation OLS specifications that include a constant. Columns 3 through 6
also include school fixed effects and the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; and student age.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment

Table B3. The effect of FSAG eligibility on cumulative financial aid receipt through
2005/2006.

(1) (2) (3) (4) (5)


Grant aid Total aid
Total grant minus from all
FSAG aid FSAG Loan aid sources

Panel A: College math sample


Eligible for FSAG 785.92*** 1,183.83 397.90 −364.47 819.35
[119.42] [785.03] [722.33] [664.17] [1,218.98]
EFC (centered at FSAG −306.43** −2,134.21** −1,827.78** −277.92 −2,412.13*
cutoff) [124.31] [904.85] [851.80] [851.74] [1,418.60]
FSAG × EFC 262.91 2,211.48* 1,948.57 524.33 2,735.81
[206.73] [1,322.02] [1,225.31] [1,086.43] [2,015.23]
R2 0.290 0.397 0.405 0.203 0.341
Outcome mean above 815.5 9,487 8,672 4,806 14,293
FSAG cutoff
N 2,834
Panel B: HS Trig+ Sample
Eligible for FSAG 920.88*** 1,457.70 536.82 621.62 2,079.33
[207.43] [1,565.69] [1,456.39] [1,459.50] [2,469.09]
EFC (centered at FSAG −178.81 −2,103.23 −1,924.42 157.59 −1,945.63
cutoff) [236.47] [1,830.71] [1,732.98] [1,599.47] [2,715.25]
FSAG × EFC −96.95 2,418.02 2,514.98 2,273.82 4,691.84
[385.39] [2,681.86] [2,483.57] [2,212.48] [4,196.05]
R2 0.402 0.442 0.451 0.291 0.384
Outcome mean above 957.6 12,339 11,381 5,888 18,226
FSAG cutoff
N 1,283
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results
are from multiple imputation OLS specifications estimated with an EFC window plus or minus $1,000
around the FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; student age, and whether the student was eligible for the
Florida Bright Futures Scholarship. All models also include school fixed effects and a constant.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
The Effects of Need-Based Grants on STEM attainment

Table B4. The effect of FSAG eligibility on cumulative STEM credits and BA/BS degrees
earned in STEM fields, among Florida high school graduates not academically prepared for
postsecondary study in STEM fields (N = 3,769).

(1) (2) (3) (4) (5) (6) (7)


Through Year 1 Through Year 3 Through Year 7

STEM STEM STEM STEM STEM STEM BA/BS


credits at- credits credits at- credits credits at- credits degree in
tempted completed tempted completed tempted completed STEM

Eligible for FSAG 0.377* 0.318* 0.93 0.684 1.01 0.558 −0.001
[0.221] [0.184] [0.645] [0.501] [1.095] [0.814] [0.004]
EFC (centered at −0.063 0.13 −0.047 0.155 0.379 0.472 0.004
FSAG cutoff) [0.268] [0.222] [0.844] [0.648] [1.393] [1.011] [0.005]
FSAG × EFC 0.867** 0.394 1.471 1.015 0.471 0.208 −0.01
[0.342] [0.289] [1.092] [0.874] [1.737] [1.315] [0.007]
R2 0.181 0.183 0.158 0.164 0.138 0.147 0.151
Outcome mean 1.47 0.95 5.41 3.64 8.85 6.05 0.003
above FSAG
cutoff
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results
are from OLS/LPM specifications estimated with an EFC window plus or minus $1,000 around the
FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic, and other
race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale); whether the
student was in a gifted and talented program; parental adjusted gross income; student age, and whether
the student was eligible for the Florida Bright Futures Scholarship. All models also include high school
fixed effects and a constant.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management
Table B5. The effect of FSAG eligibility on cumulative STEM credits and BA/BS degrees earned in STEM fields, excluding student-level
academic and demographic covariates.

(1) (2) (3) (4) (5) (6) (7)


Through Year 1 Through Year 3 Through Year 7

STEM STEM STEM STEM STEM STEM BA/BS


credits credits credits credits credits credits degree in
attempted completed attempted completed attempted completed STEM

Panel A: College math sample (N = 2,834)


Eligible for FSAG 0.633 0.756** 2.300* 2.653*** 3.142 3.982** 0.027*
[0.429] [0.384] [1.200] [1.025] [1.936] [1.663] [0.014]
EFC (centered at FSAG cutoff) 0.528 0.371 1.616 1.336 0.954 1.04 0.019
[0.510] [0.450] [1.430] [1.200] [2.279] [1.883] [0.018]
FSAG × EFC −0.169 0.251 −0.342 0.838 1.947 3.356 0.013
[0.633] [0.600] [1.806] [1.644] [2.994] [2.588] [0.024]
R2 0.001 0.002 0.001 0.002 0.001 0.003 0.002
Outcome mean above FSAG cutoff 5.63 4.31 15.99 12.41 23.55 18.27 0.043

Journal of Policy Analysis and Management DOI: 10.1002/pam


Panel B: HS Trig+ Sample (N = 1,283)
Eligible for FSAG 1.649** 1.856*** 4.542** 5.388*** 7.250** 8.853*** 0.047*
[0.715] [0.638] [2.006] [1.749] [3.166] [2.748] [0.025]
EFC (centered at FSAG cutoff) 1.665* 1.614** 5.281** 5.520*** 5.711 6.609** 0.070**
The Effects of Need-Based Grants on STEM attainment

[0.869] [0.763] [2.405] [2.119] [3.652] [3.195] [0.033]


FSAG × EFC −0.185 0.136 −2.522 −1.912 0.601 0.928 −0.059
[1.096] [1.035] [3.118] [2.876] [5.071] [4.414] [0.042]

Published on behalf of the Association for Public Policy Analysis and Management
R2 0.006 0.008 0.005 0.008 0.004 0.008 0.004
Outcome mean above FSAG cutoff 6.51 5.28 18.06 14.44 26.47 20.98 0.059
*** p < 0.01; ** p < 0.05; * p < 0.10.
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results are estimates from OLS specifications that include a constant.
The Effects of Need-Based Grants on STEM attainment

Table B6. The effect of FSAG eligibility on non-STEM college outcomes.

(1) (2) (3) (4) (5) (6) (7)


Through Year 1 Through Year 3 Through Year 7

Non- Non- Non- Non- Non- Non- BA/BS


STEM STEM STEM STEM STEM STEM degree in
credits at- credits credits at- credits credits at- credits non-
tempted com- tempted com- tempted com- STEM
pleted pleted pleted field

Panel A: College math sample (N = 2,834)


Eligible for 0.216 0.445 0.295 0.692 −2.021 −0.602 0.051
FSAG [0.796] [0.764] [2.333] [2.299] [3.885] [3.656] [0.042]
EFC (centered 0.472 0.354 −1.271 −1.749 −5.038 −4.075 0.02
at FSAG [0.976] [0.895] [2.801] [2.747] [4.683] [4.418] [0.051]
cutoff)
FSAG × EFC 0.419 0.802 3.45 4.11 7.421 7.133 0.033
[1.288] [1.171] [3.789] [3.640] [6.109] [5.810] [0.066]
R2 0.169 0.207 0.182 0.213 0.176 0.206 0.214
Outcome 13.29 11.19 42.32 36.06 72.09 61.95 0.367
mean above
FSAG
cutoff
Panel B: HS Trig+ Sample (N = 1,283)
Eligible for −0.852 0.088 −1.961 −0.083 −5.988 −2.459 0.007
FSAG [1.188] [1.186] [3.972] [3.659] [7.077] [6.178] [0.068]
EFC (centered −0.259 0.492 −0.362 0.829 −5.144 −1.71 0.014
at FSAG [1.551] [1.485] [4.954] [4.603] [8.108] [7.272] [0.084]
cutoff)
FSAG × EFC 0.97 0.454 4.673 3.607 12.177 9.392 0.032
[2.180] [2.026] [7.214] [6.720] [11.977] [11.131] [0.121]
R2 0.290 0.290 0.305 0.318 0.296 0.307 0.278
Outcome 12.95 11.22 43.24 37.64 74.57 65.05 0.433
mean above
FSAG
cutoff
Notes: Robust standard errors, clustered at the high school level, are shown in brackets. All results are
from multiple imputation OLS/LPM specifications estimated with an EFC window plus or minus $1,000
around the FSAG cutoff and include the following covariates: race dummy variables (black, Hispanic,
and other race/ethnicity); female dummy variable; high school senior year GPA (weighted 4.5 scale);
SAT math and verbal scores (imputed where missing); whether the student was in a gifted and talented
program; parental adjusted gross income; student age, and whether the student was eligible for the
Florida Bright Futures Scholarship. All models also include high school fixed effects and a constant.

Journal of Policy Analysis and Management DOI: 10.1002/pam


Published on behalf of the Association for Public Policy Analysis and Management

You might also like