Cramming: The Effects of School Accountability on College Study Habits and Performance Colleen Donovan University of California-Berkeley David

Figlio University of Florida and National Bureau of Economic Research Mark Rush University of Florida First draft: October 2005 PRELIMINARY AND INCOMPLETE: PLEASE DO NOT CITE WITHOUT PERMISSION

Corresponding author: David Figlio, Department of Economics, University of Florida, Gainesville, FL 32611-7140, [email protected]. The authors are grateful to the National Institutes of Child Health and Human Development and the National Science Foundation for financial support, and to an anonymous university for access to its admission, registrar and course download records. All errors or omissions are our own.

Cramming: The Effects of School Accountability on College Study Habits and Performance Introduction School accountability the practice of evaluating schools on the basis of the observed performance of students, and rewarding and punishing schools according to these evaluations is ubiquitous in the world today, with nations on every continent experimenting with such policies. In the United States, many states had implemented school accountability policies well in advance of the federal No Child Left Behind Act of 2001, and some states experiments with accountability are currently in their second decade. The stated purpose behind school accountability plans is to encourage schools to more effectively educate students, and especially to ensure that all students are gaining ground toward subject-matter proficiency.

The available evidence indicates that school accountability plans have been generally successful in attaining these objectives. Numerous studies, such as Figlio and Rouse (forthcoming) and Jacob (2004), indicate that low-performing schools facing accountability pressure have improved their students test scores, not just on the highstakes tests but also on tests that are directly comparable to schools in other states. Aggregate studies, such as Carnoy and Loeb (2002) and Hanushek and Raymond (2004), find that states that adopted high-stakes school accountability plans earlier than others have experienced greater growth in average test scores as compared with late-adopting states. The upshot of these studies is that school accountability has had a moderately large effect on average student outcomes.

While there has been considerable research attention paid to the effects of school accountability plans on the standardized test scores of average students or lowperforming students, as well as evidence concerning the incentives embedded within school accountability plans (e.g., Boyd et al., 2002; Cullen and Reback, 2001; Deere and Strayer, 2001; Figlio, 2005; Figlio and Getzler, 2001; Figlio and Winicki, 2004; and Jacob, 2004), to date there has been no published research investigating the effects of these plans on the high end of the academic distribution those students who would surely have attained proficiency in the absence of school accountability plans. It is not obvious how school accountability should affect these students. On the one hand, if school accountability plans and the tests they employ to evaluate schools, are challenging and based on rigorous standards, it could be expected that school accountability plans would have a positive impact on students learning. On the other hand, if schools adhere strictly to the material most likely to be on the high-stakes test and the focus becomes more fully on high-stakes tests as ends in themselves rather than indicators of subjectmatter knowledge, school accountability plans could have the effect of dulling student and teacher creativity and altering the ways in which students approach learning. Both of these potential consequences may manifest themselves in university studies, where study habits and course performance may be improved or diminished due to school accountability plans.

This paper seeks to fill this void in the literature. We exploit data from a state that changed the basis of its accountability system in 1999. This change directly influenced a large number of schools that immediately either transitioned from being threatened with

sanctions to not being threatened at all, or vice versa. Using this identification strategy, we can measure the impact on students of the school they attend either becoming threatened or becoming less threatened. Because the bases of these two accountability systems were so different, we can employ a variety of falsification tests to distinguish the effects of school accountability sanctions from the effects of the variables underlying these sanctions. We compare students in the high school classes of 2000 and 2001 the last high school class affected directly by the old accountability regime (because the high-stakes test grade is the tenth grade) and the first high school class affected directly by the new accountability regime. Because of the possibility for spillovers even though ninth grade is not a high-stakes grade, for instance, schools may have begun gearing up for the tenth grade test, and curricula and teaching styles might not immediately change even when accountability pressures change this empirical approach represents a strong test of the effects of accountability sanction threats on student university performance.

In order to implement this identification strategy, we utilize a remarkable dataset from a large selective public university in the state in question. (The identity of the university and the state in question are suppressed to preserve the anonymity of the institution that provided us with the data.) The university in question provided us with information from admissions records (the students high school attended, grade point average, college entrance exam scores, race/ethnicity, and sex) as well as transcript data when in college (courses taken and grades earned). These data allow us to identify the effects of attending a high school threatened with accountability sanctions on a students subsequent collegiate success.

The university in question also provided us with data that allows us to investigate one other potential consequence of attending a high school threatened with accountability sanctions changes in student study habits. For six large classes in which the universitys students watch lectures online rather than in a traditional classroom setting, we observe the precise timing of when students first downloaded and viewed each lecture. We can therefore measure the degree to which students cram for exams delaying studying for a course until the time prior to the test, rather than keeping up with the course throughout the semester and relate this cramming behavior to the threat of accountability sanctions that the students high school faced during the students tenth grade. While this measure of cramming is only one measure of student study habits, it has the advantage that for one semester Spring 2003 it is directly observed and measured, meaning that we do not need to rely on student self-reports for outcome data.

We observe that school accountability plans have the potential to substantially affect high-achieving students performance and study habits in college. Looking at a range of college courses, school accountability plans have an apparent effect on student course performance at the university in question but not in uniform ways. While removing the accountability threats from the first accountability system appears to have improved student performance in college in general, introducing the new accountability system did not have an equal and opposite effect on collegiate performance. Instead, this introduction had a positive effect in some disciplines, a negative effect in others, and no

apparent effect in still others. We suspect this is due to the fact that the new system introduced in 1999 was strongly standards-based, while the disbanded system was not.

We also observe that school accountability plans apparently alter student study habits in college. Students who attended schools threatened with sanctions under either school accountability system displayed a dramatically increased likelihood of cramming once they attended the university in question. While we lack the ability to explore whether cramming behavior is directly responsible for the relationship between accountability sanction threats and student collegiate performance across a wide range of subjects, we can show that the more students in an online-viewed course crammed, the worse they performed in those classes. Therefore, the evidence suggests that poor student study habits may be partially to blame for the low course performance of students affected by the first accountability system, and that the second accountability system may have been more effective in improving student collegiate performance were it not for the poor study habits that accompanied the systems introduction.

Two accountability systems In 1999, the state in question dramatically changed its school accountability system. Prior to 1999, schools were graded according to aggregate student performance on a lowlevel test of basic skills, a minimum standard of proficiency and not one that was linked to instructional or curricular standards. Beginning in 1999, the state changed its system of evaluating schools to be based on meeting proficiency targets on a new standardsbased test. The new test discriminated performance at a much higher level, and was

aligned to a far greater extent to the subject matter that state educators expected students to master. While the majority of students statewide scored in the highest category of the test used for pre-1999 accountability, only a small fraction of students scored in the highest category of the test used for post-1999 accountability. The result is that the newer accountability system was likely to directly affect the measured learning of all students, including four-year college-bound students, in a high school, while the older accountability system was less likely to directly affect the high-performing students measured academic skills.

But while the two accountability systems differed considerably in their potential direct effects on measured academic performance in high school, both accountability systems had the potential to influence student study skills and the ways in which students approach learning. As measured above, students may learn from their schools facing accountability pressure that test performance is the goal in itself, and may manifest this attitudes by changing their approach to education from deliberate study for the sake of learning to concentrated study for the sake of test performance.1 Both accountability regimes though particularly the new regime could potentially help to instill this new study ethic in high-performing students.

Most important for the purposes of this study is that the changing accountability regimes dramatically affected which schools were threatened with accountability sanctions. Some

Anecdotal evidence abounds regarding these changing attitudes. Schools threatened with accountability sanctions often offer parties to reward students for high aggregate test performance, hold pep rallies to gear students up for the accountability test, and teach test-specific preparation skills to students and their parents.

schools immediately transitioned from being threatened to no longer being threatened, while other schools were affected in the reverse. Table 1 presents evidence of the transitions between accountability systems. Specifically, because we will be employing high school fixed effects in our regression models, we describe the transitions only of high schools where students in both the high school class of 2000 and the high school class of 2001 are represented in our data. We observe that 22 percent of schools changed accountability status from one accountability regime to the other, with the majority of these schools becoming less threatened once the accountability regime shifted. Nine percent of schools were threatened with sanctions in both accountability regimes, while 69 percent of schools were threatened in neither accountability regime. Our analysis will focus on the bottom-left and top-right cells of Table 1 the schools that transitioned from threatened to not threatened, and vice versa.

Table 2 describes the attributes of the students in our study, based on the accountability status of their high school. We observe that the schools that transitioned from unthreatened to threatened and vice versa were somewhat different from one another, and different from the schools whose accountability status did not change over this time period. Specifically, the high schools that were not threatened under the old accountability regime but became threatened under the new accountability regime sent students to the university in question who were more likely to be male, black and Hispanic and who had modestly lower college entrance exam scores and slightly lower adjusted high school grade point averages than did the high schools whose accountability status reversed from one regime to the next. The fact that these differences in student

characteristics across schools are apparent underscores the importance of estimating high school fixed effects models, where we compare students from one graduating class in a given high school to students in the next graduating class from the same high school.

Table 2 also compares one student attribute SAT scores within the same schools across graduating classes. As can be seen, SAT scores improved for enrollees from the high school class of 2001 vis--vis enrollees from the high school class of 2000 for all types of schools. However, we observe that the group of schools that experienced a transition from unthreatened to threatened as the accountability regime changed saw the greatest improvement in students sent to the university, at least according to this metric. This finding underscores the necessity of controlling for student background characteristics in the school fixed effects analyses that follow.

Effects of accountability on collegiate grades The first purpose of this paper is to explore the degree to which school accountability has affected student performance in college. Our approach is to estimate variants of the following model: (Grade)isc = s + 2c + (Xisc + (Accountability threat)sc + ,isc , where X is a vector of student background characteristics (race/ethnicity, sex, high school adjusted grade point average, and SAT scores), s represents the high school attended and c represents the students graduating class. Because we expect outcomes to differ depending on which accountability regime is active at the relevant time for a given student i, we estimate two coefficients one for the estimated effect of removing the

old accountability sanctions and one for the estimated effect of introducing the new accountability sanctions. If the accountability system has a positive effect on student course performance, therefore, one would expect the coefficient on removing threatened sanctions to be negative and the coefficient on introducing threatened sanctions to be positive. If the accountability system has a negative effect on student course performance, the opposite predictions would be true. Given the different potential effects of accountability on course performance, we have no a priori prediction of whether the positive effects or the negative effects of accountability threats would be stronger.

Table 3 presents evidence of the relationship between student grades and accountability sanction threats for students in introductory courses in the 14 most popular introductory subjects at the university in question. Each row represents a different course, and the two estimated coefficients for each regression are reported in the table.2 The first column of the table shows the estimated effects of removing the old accountability sanction threats. We observe consistent evidence that removing the old accountability sanction threats are associated with improved course performance, as measured by course grades (on a fourpoint scale, with a grade of A being 4 points.) In eleven of the 14 subjects the estimated coefficient is positive, and in eight of these it is statistically significant at conventional levels. In the three subjects where the estimated coefficient is negative it is far from statistical significance. Moreover, the estimated effects of dropping the accountability sanction threat are often large in magnitude 0.3 grade points in
2

The coefficient estimates on race/ethnicity, sex, graduation class, SAT scores, and high school grade point average are available upon request from the authors.

mathematics (calculus or pre-calculus), 0.2 grade points in English composition, or nearly 0.8 grade points in anthropology, for example. Given that the mean college grade is a B, these are very large estimated effects.

As can be seen, the estimated effects of introducing the new accountability system are not nearly as consistent. Only five of the 14 estimated coefficients are statistically significant at conventional levels, and those that are statistically significant are not of the same sign. The three courses for which a negative effect of introducing the new accountability sanction threats is estimated (equivalent to a positive estimated effect of dropping the old sanction threat) anthropology, computer science and geology were also among the courses for which there was a positive estimated effect for removing the threat of sanctions. But two other courses statistics and mathematics have positive estimated effects of introducing the new accountability sanctions.

Why might one expect a positive effect of introducing the new accountability sanctions in mathematics and statistics? The answer lies in the nature of the new accountability system, which imposed standards in mathematics, reading and writing for students and schools. If schools were responding to these standards, it follows that mathematics and statistics training may be improved, an effect manifested in higher performance once at the university. But what about English composition? The accountability standards for writing focused on formulaic writing skills that reflect clarity of expression and exposition, but not higher-order writing skills.

10

The results of this analysis illustrates that accountability sanction threats have the potential to lead to either positive or negative effects for students in college. An accountability system based on poorly-defined standards and low-level skills such as the system in place before 1999 is estimated to lead to reduced student performance in college, while a system based on well-defined standards and higher-level skills such as the system put in place in 1999 does not have the same general negative estimated effect (though some negative estimates are found), and may lead to improvements in collegiate performance in the subjects most closely related to the high standards tested.3

Measuring study skills in college We next turn to the effects of school accountability sanction threats on the development of study skills. While it is impossible to directly measure study skills, we can measure an indicator of these skills. In one semester Spring 2003 the university that provided us with the data for this analysis recorded not only the date of download for each onlineviewed class, but also the specific lecture downloaded. We can therefore identify the degree to which students delay watching the lectures one measure of cramming for the exam. Our data consist of observations on 2064 students who attended 249 public high schools in the state for which students in both the high school graduating classes of 2000 and 2001 are present in the data.

Figure 1 illustrates definitively that cramming for exams occurs and is widespread in the online-viewed classes. Each of the six graphs presents time-series plots of the number of
Neither accountability system assesses schools on the basis of science or social studies performance. Among the four introductory science courses, geology is the least mathematical, and therefore probably the least likely to be positively affected by improved mathematics instruction in high school.
3

11

lecture downloads per day, per course. It is immediately apparent that the pattern of downloads exhibits large cycles, and the dates of the exams in each course are obvious by inspection. Table 4 presents similar data in tabular form. Sixteen percent of students download a lecture on the day it was delivered, and another 20 percent download the lecture over the next two days. But three percent download any given lecture on the day of the test, and another nine percent download a lecture one or two days before the test (this is not counting lectures delivered close to the time of the test in question.) All told, one-quarter of lectures delivered more than one week before the exam are first downloaded in the week prior to the exam, and the same fraction is true for those lectures delivered more than two weeks before the exam.

The argument that accountability sanction threats lead to cramming behavior requires that, for some students, cramming is a trait an attribute of the student that persists. Tables 5 and 6 present some evidence that this is the case. We observe (see Table 5) that some students consistently keep up with lectures while others consistently procrastinate. Thirty-one percent of students are never observed downloading a lecture within two days of the lecture delivery, and 57 percent of students download less than one-fifth of lectures within two days of lecture delivery. While few students wait until the last moment to download every lecture, a non-trivial fraction of students delay downloads of many lectures. Over two percent of students view more than half of their lectures for the first time within two days of the exam, and nearly 12 percent first download at least one-fifth of their lectures within two days of the exam. The presence of these patterns suggests that students tend either to keep up with the course or to delay viewing the lectures.

12

Table 6 illustrates that students who tend to cram in one class tend to cram in all of their classes. This table presents a cross-tabulation of students cramming behavior in two courses taken simultaneously, for the set of students for whom behavior in two courses is observed. The top panel investigates the percentage of lectures downloaded within two days of delivery. We observe that if a student in course #1 downloads less than one-third of lectures within two days of delivery, he or she is very likely to do the same (69 percent) in his or her other class, and only has a 15 percent chance of downloading more than two-thirds of the lectures in the other class within two days of delivery. On the other hand, if the student in course #1 downloads more than two-thirds of lectures within two days of delivery, he or she has a 43 percent chance of downloading less than one-third of lectures in his or her other class within two days, but has a 41 percent chance of downloading two-thirds or more of lectures in his or her other class within two days of delivery. The bottom panel presents similar evidence for students who download within two days of the exam. The lesson from these tables is that while cramming in courses does not follow a one-to-one correspondence, there is a very high degree of withinstudent persistence in cramming across classes.4

Table 7 shows that students who cram are somewhat more likely to be male, somewhat more likely to be Hispanic and Asian, and tend to have higher SAT scores than do students who dont cram. While high school grade point averages are slightly higher for cramming students as for those who do not cram, the differences are not great.

There is also considerable within-student persistence within a given course. Students who tend to cram for the first exam in a course tend to cram for the second exam in the course.

13

Given that cramming tends to persist for individual students, the next logical question is whether cramming has a deleterious effect on student outcomes in the class in which cramming occurs. Table 8 presents separate regressions of the relationship between cramming behavior (as measured by the fraction of classes first downloaded within two days of the exam) and a students grade in the class, for each of the six online classes. We observe a consistent pattern (in five of the six classes) between cramming and course performance; holding constant student attributes, students who cram more tend to perform at a lower level in the class. Table 8 also presents separate regressions for students who attended threatened schools in either accountability regime, and those who attended schools that were not threatened in either accountability regime. We again observe consistent evidence of a negative relationship between cramming behavior and student course grades, regardless of the nature of the high school that the student attended, and across courses.5 In sum, student cramming persists across time within a course, and across courses, and is associated with reduced student course performance. This may of course be due to some unobserved attribute, but recall that if anything, cramming students tend to look better on observables (in terms of SAT scores and perhaps in terms of high school grade point average) than do students who cram less. An alternative theory would be that there is another unobserved attribute that affects cramming and student grades that is caused by changes in a school over time. We turn to this possibility next.

For two courses, the estimated relationship between cramming and course grades is only statistically significant for one of the two subgroups of students, though the estimated sign is consistently negative.

14

Do accountability sanction threats induce student cramming? Table 9 presents estimates of the relationship between accountability sanction threats and student cramming behavior. Two measures of cramming are presented the fraction of total lectures downloaded within one week of the exam and the fraction of total lectures downloaded within two days of the exam. We estimate the model: (Fraction of lectures downloaded)isc = s + 2c + (Xisc + (Accountability threat)sc + ,isc , where all notation is as before.6

We observe that cramming behavior is associated with accountability sanction threats. When such a threat is removed, students attending a formerly-sanctioned school subsequently cram less in college than do their predecessors who attended the high school when the school was threatened. When a new threat is introduced, students attending a newly-sanctioned school subsequently cram more in college than did their predecessors who attended the high school before it was threatened. The estimated effects of the new accountability system are greater in magnitude and statistical significance than are the estimated effects of the older accountability system; this finding is consistent with the expectations that the new accountability system may impact higherachieving students more than did the older accountability system.

We also conduct two falsification exercises. The first such exercise, reported in the second column of Table 9, utilizes data from the graduation classes of 1999 and 2000, rather than 2000 and 2001, and assumes that the accountability system switch occurred a

A full set of coefficient estimates is presented in Appendix Table 1.

15

year before it actually did.7 We observe that there is no apparent relationship between cramming and changes in accountability regimes when the accountability regime did not actually change.

Our second falsification exercise involves using the same data as in the actual analysis, but applying the 1998 standards to test scores realized in 1999 and 1999 standards to test scores realized in 1998, for the purposes of calculating which schools would have been sanctioned if the systems, but not the test scores used to determine a schools accountability grades, had been swapped. Again, we find no apparent relationship between measured cramming and this other characterization of school test scores, indicating that it the accountability system itself, and not the actual school performance underlying the accountability system, that apparently led to cramming behavior among an affected schools students.

Conclusion This paper presents the first evidence of the effects of school accountability systems on the long-term human capital development of the affected students. The results are mixed. On the one hand, the evidence is consistent that school accountability sanction threats are associated with changes in student study habits. Students who attended high schools that were threatened with accountability sanctions systematically study differently than did their school-mates who graduated from high school in an adjacent year but who were not directly affected by the schools threat of sanctions. We find evidence that the increased
The number of observations differs from the first and third column because the number of relevant schools and students is different in this exercise. The results of the falsification exercise are virtually unchanged were we to limit the sample of schools to be the same as in the first and third columns.
7

16

student cramming that apparently results from school accountability sanction threats may lead to reduced course performance in the classes in which the student crams.

Whether school accountability sanction threats lead to lower student course performance than would have occurred in the absence of such threats, however, is not as clear. We observe that students attending high schools that were threatened with sanctions under the states old accountability regime tended to perform at a lower level in their coursework at the university, all else equal. But we do not observe this same pattern with regard to the new accountability regime. Students attending high schools threatened with the new sanctions performed at a higher level in mathematics and statistics courses than their school-mates had previously done, all else equal. At the same time, these same students performed at a lower level in some of their other college courses. We suspect that the difference in outcomes across these accountability regimes and across course subjects in the new accountability regime is due to the standards-based nature of the new accountability regime and to the subjects emphasized in the new accountability system.

These results provide potential lessons for the design of accountability systems. The finding that students are apparently better prepared for college coursework in the specific subjects that are emphasized by the accountability system suggests that accountability systems that cover more subject areas may in turn produce better-prepared students. At the same time, the finding that students performed better when their schools were threatened with sanctions under a standards-based accountability system than under an accountability system with lower standards indicates that a system with high-quality tests

17

closely aligned with rigorous standards is likely to yield better student preparation for higher education.

While some school accountability systems may lead to improved preparation in specific subjects, one consistent feature of both types of accountability systems implemented in the state in question is that they apparently lead to altered student study habits, and not necessarily for the better. Students who attended high schools threatened with sanctions under either accountability system tend to cram for exams and apparently perform less well as a consequence when they reach the university. This is not a necessary consequence of school accountability systems, but it seems likely to result when schools place such a heavy emphasis on the outcomes of a specific test, rather than on overall learning outcomes over the entire year. To the extent to which school accountability systems encourage such behavior, they may lead to the unintended consequence of changing the ways in which students approach learning and scholarship.

18

References To be added

19

Figure 1: Time-series patterns of first student downloads of lectures, six electronic platform courses, Spring 2003
.04 .03 15750 15800 15850 0 15700 .01 0 .01 15700 fraction .02 fraction .02

.03

day

15750

day

15800

15850

Course A with four midterm exams

Course B with three midterm exams

fraction

.02

.01

15700

15750

day

15800

15850

0 15700

.01

fraction .02

.03

15750

day

15800

15850

Course C with one quiz and three midterm exams

Course D with three midterms and optional final

.03

.04

fraction .02

.01

15720

15740

15760

day

15780

15800

15820

0 15700

.01

fraction

.02

.03

15750

day

15800

15850

Course E with four midterm exams

Course F with three midterm exams

20

Table 1: Transitions from one accountability system to another, high school classes of 2000 and 2001: students in high schools represented in both years In-state public school graduates only High school threatened with sanctions in high-stakes test year, class of 2001 No Yes 0.689 0.021 0.198 0.091

High school threatened with sanctions in highstakes test year, class of 2000

No Yes

21

Table 2: Attributes of students in study, by high school transition: high school classes of 2000 and 2001 In-state public school graduates only High school transitioned from unthreatened to threatened 0.56 0.13 0.21 0.12 1196 3.8 High school transitioned from threatened to unthreatened 0.48 0.08 0.08 0.13 1218 3.9 High school did not change status

Male 0.49 Black 0.06 Hispanic 0.09 Asian 0.08 SAT score 1239 Adjusted high 3.9 school GPA Comparing attributes of students in each of the two high school graduation cohorts SAT scores: high 1174 1208 1233 school class of 2000 SAT scores: high 1208 1228 1245 school class of 2001

22

Table 3: Estimated effects of changing accountability regimes on student course outcomes: High school fixed effects regressions; clustered standard errors in parentheses Dependent variable: grade in introductory course Introductory course Estimated effect of removing old accountability sanction threat 0.165 (0.200) 0.767** (0.320) 0.214 (0.198) 0.290* (0.163) 0.258** (0.084) 0.215** (0.066) 0.225** (0.082) -0.248 (0.224) 0.634** (0.163) 0.378* (0.227) 0.311** (0.145) 0.128 (0.160) -0.113 (0.354) -0.014 (0.101) Estimated effect of introducing new accountability sanction threat -0.079 (0.191) -0.753** (0.328) -0.296 (0.288) 0.137 (0.288) -0.192* (0.109) 0.115 (0.134) -0.042 (0.108) 0.068 (0.217) -0.554** (0.157) 0.181 (0.478) 0.330* (0.191) -0.233 (0.579) -0.215 (0.293) 0.226* (0.127)

American History Anthropology Astronomy Chemistry Computer Science Economics English Composition Geography Geology Government Mathematics (Calculus) Physics Psychology Statistics

Note: Coefficients marked * (**) are statistically significant at the ten (five) percent level. Clustered standard errors are in parentheses. Regressions also include school fixed effects, time effects, student race/ethnicity variables, student sex, student high school grade point average, and student SAT scores.

23

Table 4: Patterns of first student downloads of lectures, six electronic platform courses, Spring 2003 Timing of download Fraction of first downloads 0.159 0.203 0.253 0.168 0.099 0.086 0.032 Fraction of first downloads, lecture delivered >=7 days before test 0.140 0.177 0.235 0.196 0.116 0.100 0.037 Fraction of first downloads, lecture delivered >=14 days before test 0.137 0.167 0.199 0.271 0.123 0.084 0.019

Day of lecture 1-2 days later 3-6 days later >=7 days later, >=7 days before test >=7 days later, 3-6 days before test >=7 days later, 1-2 days before test >=7 days later, Day of test

24

Table 5: Cramming as a trait: Percentage of lectures downloaded immediately or at the last moment, by student Frequency Fraction of students: Percentage of lectures downloaded within 3 days of delivery 0.311 0.132 0.131 0.098 0.067 0.052 0.051 0.043 0.037 0.036 0.017 0.025 Fraction of students: Percentage of lectures downloaded 2 or fewer days before the exam (for lectures delivered >=7 days before exam) 0.533 0.263 0.102 0.045 0.021 0.015 0.010 0.006 0.002 0.002 0.001 0.001

Never 0.1-9.9% 10-19.9% 20-29.9% 30-39.9% 40-49.9% 50-59.9% 60-69.9% 70-79.9% 80-89.9% 90-99.9% Always

25

Table 6: Cramming as a trait: Persistence of cramming behavior across classes, by student (row percentages) Panel 1: Lectures downloaded 2 or fewer days after delivery (mean: 25.1%) Percentage of lectures downloaded 2 or fewer days after delivery, class #2 0-33.3% 33.4-66.6% 66.7-100% Percentage of 0-33.3% 0.691 0.159 0.151 lectures 33.4-66.6% 0.565 0.272 0.164 downloaded 2 66.7-100% 0.428 0.162 0.410 or fewer days after delivery, class #1 Panel 2: Lectures downloaded 2 or fewer days prior to exam, for lectures delivered >=7 days before exam (mean: 6.5%) Percentage of lectures downloaded 2 or fewer days prior to exam, class #2 0% 0.1-20% >20% Percentage of 0% 0.819 0.128 0.054 lectures 0.1-20% 0.622 0.300 0.078 downloaded 2 >20% 0.516 0.258 0.226 or fewer days prior to exam, class #1

26

Table 7: Attributes of students with different levels of cramming behavior Percentage of lectures downloaded 2 or fewer days before exam 0-33.3% 33.4-66.6% 66.7-100% 0.54 0.55 0.62 0.06 0.09 0.08 0.11 0.12 0.14 0.08 0.08 0.11 1189 1210 1227 3.84 3.83 3.89

Male Black Hispanic Asian SAT score Adjusted high school GPA

27

Table 8: Cramming and student course performance Student-level relationship between course grade and cramming behavior Coefficient on fraction of classes first downloaded 2 or fewer days before exam Students who All students in the Students who class attended threatened attended schools not threatened in either schools in either accountability accountability regime regime -1.875** -2.079** -1.441 (0.649) (1.065) (1.098) -0.371** -0.084 -0.535** (0.141) (0.268) (0.165) -1.558** -2.032* -1.090* (0.462) (0.766) (0.621) -0.446 -1.919 -0.440 (0.458) (2.564) (0.429) -0.810** -0.726* -0.943** (0.239) (0.474) (0.280) -1.046** -0.868* -1.151** (0.258) (0.509) (0.294)

Course A Course B Course C Course D Course E Course F

Note: Each cell represents a separate regression. Regressions also control for student sex, race/ethnicity, SAT scores, and high school grade point average. Coefficients marked * (**) are statistically significant at the ten (five) percent level.

28

Table 9: Estimated effects of changing accountability regimes on student cramming behavior: High school fixed effects regressions; clustered standard errors in parentheses Dependent variable: fraction of total classes first downloaded within week (two days) of exam In-state public school graduates only Specification 1: Actual accountability sanctions included in model (graduation years 2000,2001) 2064 249 Falsification test 1: Assume that accountability switch occurred one year before it did (graduation years 1999, 2000) 2394 265 Falsification test 2:Apply 1998 standards to 1999 scores, and 1999 standards to 1998 scores (graduation years 2000, 2001) 2064 249

Number of students Number of schools present in both years Cramming measure: Fraction of total lectures downloaded within the week before the exam -0.030* -0.007 0.010 Estimated effect of (0.017) (0.019) (0.027) removing old accountability sanction threat 0.078** -0.008 0.016 Estimated effect of (0.035) (0.026) (0.032) introducing new accountability sanction threat Cramming measure: Fraction of total lectures downloaded 2 or fewer days before exam -0.027* -0.014 0.016 Estimated effect of (0.015) (0.017) (0.028) removing old accountability sanction threat 0.062** 0.008 0.015 Estimated effect of (0.032) (0.025) (0.026) introducing new accountability sanction threat

Note: Coefficients marked * (**) are statistically significant at the ten (five) percent level. Clustered standard errors are in parentheses. Regressions also include school fixed effects, time effects, student race/ethnicity variables, student sex, student high school grade point average, and student SAT scores.

29

Appendix Table 1: Full set of coefficient estimates (except for course fixed effects and high school fixed effects): Table 9, specification 1, top panel Variable Old accountability sanctions x post-change New accountability sanctions x postchange Post-change Male Black White Asian Hispanic Have SAT score SAT score Have high school GPA Adjusted high school GPA Coefficient (standard error) -0.030* (0.017) 0.078** (0.035) -0.007 (0.014) 0.016 (0.012) 0.002 (0.041) -0.004 (0.033) 0.050 (0.037) 0.027 (0.038) -0.093 (0.070) 0.00006 (0.00006) -0.075 (0.065) 0.021 (0.017)

Note: Coefficients marked * (**) are statistically significant at the ten (five) percent level. Clustered standard errors are in parentheses.

30