Rohrer, D., & Taylor, K. (2006).

The effects of overlearning and distributed practice on the

retention of mathematics knowledge. Applied Cognitive Psychology, 20, 1209–1224.

The Effects of Overlearning and Distributed Practice on the Retention of Mathematics

Knowledge

Doug Rohrer and Kelli Taylor

University of South Florida

In two experiments, 216 college students learned a mathematical procedure and returned for a test either one or
four weeks later. In Experiment 1, performance on the four-week test was virtually doubled when students
distributed 10 practice problems across two sessions instead of massing the same 10 problems in one session. This
finding suggests that the benefits of distributed practice extend to abstract mathematics problems and not just rote
memory cognitive tasks. In Experiment 2, students solved 3 or 9 practice problems in a single session, but this
manipulation had no effect on either the one-week or four-week test. This result is at odds with the virtually
unchallenged support for the strategy of continuing practice beyond the point of mastery in order to boost long-
term retention. The results of both experiments suggest that the organization of practice problems in most
mathematics textbooks is one that minimizes long-term retention.

Perhaps no mental ability is more a student first masters a skill and then
important than our capacity to learn, but the immediately continues to practice the same skill.
benefits of learning are lost once the material is For example, after a student has studied a set of
forgotten. Such forgetting is particularly 20 vocabulary words until each definition has
common for knowledge acquired in school, and been correctly recalled once, any immediate
much of this material is lost within days or further study of these definitions entails an
weeks of learning. Thus, the identification of overlearning strategy. Overlearning is
learning strategies that extend retention would particularly common in mathematics education
prove beneficial to students and any others who because many mathematics assignments include
wish to retain information for meaningfully long a dozen or more problems of the same type. A
periods of time. Toward this aim, the two distributed or spaced practice strategy requires
experiments presented here examined how the that a given amount of practice be divided across
retention of a moderately abstract mathematics multiple sessions and not massed into just one
procedure was affected by variations in either the session. For example, once a mathematics
total amount of practice or the scheduling of this procedure has been taught to students, the
practice. corresponding practice problems can be massed
Specifically, the two learning strategies into one assignment or distributed across two or
assessed here are known as overlearning and more assignments. As detailed in the general
distributed practice. By an overlearning strategy, discussion, the practice problems in most
mathematics textbooks are arranged so that
Address correspondence to Doug Rohrer, Department of Psychology, students rely on overlearning and massed
PCD4118G, University of South Florida, Tampa, FL 33620, USA. E-
mail: [email protected]. We thank Christina Coghlan, Tammy
practice.
Logan, Kristina Martinez, Erica Porch, and Courtnay Rosene for The strategies of distributed practice and
their assistance with data collection. This research was supported by
two grants from the Institute of Education Sciences, U.S. Department
overlearning are not complementary, and the two
of Education, to Hal Pashler and Doug Rohrer (R305H020061, strategies cannot be compared directly. Instead,
R305H040108). The opinions expressed are those of the authors and
do not necessarily represent the views of the Institute of Education
distributed practice is the complement of massed
Sciences. practice, and the comparison of these two
 Mathematics Learning 2
strategies requires that the total amount of instance, in the Driskell et al. meta-analysis
practice be held constant. For example, one described above, only 7 of the 51 comparisons
group of students might divide 10 problems relied on a retention interval of more than one
across two sessions while another group solves week, and the longest was 28 days. Moreover,
all 10 problems in the same session (as in the largest effect sizes were observed for
Experiment 1). By contrast, assessing the retention intervals lasting less than one hour, and
benefits of overlearning requires a manipulation Driskell et al. astutely observed that benefits of
of the total amount of practice given within a overlearning declined sharply with retention
single session. Thus, one group might be interval.
assigned three problems while another is The possibility that the benefits of
assigned nine problems (as in Experiment 2). overlearning may dissipate with time is also
Thus, because overlearning and distributed supported by several overlearning experiments
practice are orthogonal and not complementary, including an explicit manipulation of retention
it is logically possible that neither, both, or just interval. For example, in Experiment 1 of
one of these strategies could benefit long-term Reynolds and Glaser (1964), some students
retention. Naturally, both strategies have been studied biology three times as much as others,
the focus of numerous previous studies, but the and the high studiers recalled 100% more than
following review of the research literature the low studiers after 2 days but just 7% more
reveals caveats, gaps, and inconsistencies with after 19 days. A similar decline in the test score
regard to the benefits of each strategy for benefits of overlearning was observed in two
conceptual mathematics tasks. recent experiments reported by Rohrer, Taylor,
Overlearning Pashler, Wixted, and Cepeda (2005).
An overlearning experiment requires a Finally, it appears that the benefits of
manipulation of the total amount of practice overlearning are especially unclear in
within a single session, so that one condition mathematics learning because, to our knowledge,
includes more practice than another. Numerous every previously published overlearning
experiments have shown that this increase in experiment relied solely on verbal memory tasks.
practice can raise subsequent test performance Moreover, virtually all of these tasks required
(e.g., Bromage & Mayer, 1986; Earhard, Fried, only rote memory. Thus, the results of these
& Carlson, 1972; Gilbert, 1957; Kratochwill, experiments may not generalize to abstract
Demuth, & Conzemius, 1977; Krueger, 1929; mathematical tasks. This gap in the literature is
Postman, 1962; Rose, 1992). This benefit of surprising in light of the heavy reliance on
overlearning is also supported by a meta-analysis overlearning by students in mathematics courses,
by Driskell, Willis, and Cooper (1992), who as further detailed in the general discussion. In
examined 51 comparisons of overlearning versus summary, because of the uncertainty surrounding
learning-to-criterion in experiments using the long-term benefits of overlearning and the
cognitive tasks and found a moderately large apparent absence of overlearning experiments
effect of overlearning on a subsequent test (d = using mathematics tasks, it is unclear whether
.75). By these data, it is not surprising that overlearning is efficient or even effective when
overlearning is a widely advocated learning long-term retention is the aim.
strategy (e.g., Fitts, 1965; Foriska, 1993; Hall, Distributed Practice
1989; Jahnke & Nowaczyk, 1998). When practice is distributed or spaced, a
Yet a closer review of the empirical given amount of practice is divided across
literature reveals that the benefits of overlearning multiple sessions and not massed into one
on subsequent retention may not be long lasting. session. The duration of time between learning
This is because most previous overlearning sessions is the inter-session interval (ISI). For
experiments have employed a relatively brief example, if 10 math problems are divided across
retention interval (RI), which is the duration two sessions separated by one week, the ISI
between the learning session and the test. For equals one week. By contrast, massed practice
 Mathematics Learning 3
entails an ISI of zero. When practice is Modigliani (1985) observed a spacing effect with
distributed, the retention interval refers to the young children who were asked to memorize five
duration between the test and the most recent multiplication facts (e.g., 8 x 5 = 40). While such
learning session. For example, if a concept is facts are certainly useful, the present study
studied on Monday and Thursday and tested on focuses on mathematical tasks that require more
Friday, the RI equals one day. (Incidentally, than rote memory.
while the benefits of distributing practice across A second reason to question the benefits
sessions is the focus of the present paper, one of distributed practice for non-rote mathematics
can also distribute practice within a session when learning is given by the design of three previous
multiple presentations are separated by unrelated mathematics learning experiments that are often
tasks, e.g., Greene, 1989; Toppino, 1991). cited as evidence of a spacing effect with a non-
Distributed practice often yields greater rote mathematics task. In these experiments, the
test scores than massed practice, and this finding retention interval was shorter for Spacers than
is known as the spacing effect (e.g., Baddeley & for Massers, and this confound undoubtedly
Longman, 1978; Cull, 2000; Bjork, 1979, 1988; benefited the Spacers. For instance, in Grote
Bloom & Shuell, 1981; Carpenter & DeLosh, (1995), the Massers learned only on Day 1 while
2005; Dempster, 1989; Fishman, Keller, & the Spacers’ learning continued from Day 1
Atkinson, 1968; Seabrook, Brown, & Solity, through Day 22. Yet every student was tested on
2005). At very brief retention intervals, however, Day 36, which produced a 35-day RI for the
spaced practice may be no better or even worse Massers and a 14-day RI for the Spacers. This
than massed practice (e.g., Bloom & Shuell, undoubtedly benefited the spacers. The same
1981; Glenberg & Lehman, 1980; Krug, Davis, confound between ISI and RI occurred in the two
& Glover, 1990). By this result, students are mathematics learning experiments reported by
behaving optimally when they mass their Gay (1973). We are not aware of any previously
learning into one session just prior to an exam if published, non-confounded experiment that
they do not need the information after the exam. examines how the distribution of practice affects
At longer retention intervals, though, the benefits the retention of a non-rote mathematics task.
of spacing are often sizeable, and many There are, however, non-experimental
researchers have therefore argued that students studies that have found benefits of distributed
should rely more heavily on distributed practice practice of mathematics. Most notably, perhaps,
in order to extend retention (Bahrick, Bahrick, Bahrick and Hall (1991) assessed the retention of
Bahrick, & Bahrick, 1993; Bjork, 1979; Bloom algebra and geometry by people who had taken
& Shuell, 1981; Dempster, 1989; Schmidt & these courses between 1 and 50 years earlier. A
Bjork, 1992; Seabrook, Brown, & Solity, 2005). regression analysis showed that retention was
Even at longer retention intervals, positively predicted by the number of courses
though, the benefits of spacing are more requiring the same material. For example, much
equivocal for tasks that require abstract thinking of the material learned in an algebra course
rather than only rote memory. In a recent meta- reappears in an advanced algebra course, and the
analysis of 112 comparisons of distributed and completion of both courses therefore provides
massed practice, Donovan and Radosevich distributed practice of this overlapping material.
(1999) found that the size of the spacing effect Thus, the results suggest that distributed practice
declined sharply as task complexity increased may be beneficial in mathematics learning. This
from low (e.g., rotary pursuit) to average (e.g., possibility is assessed here with a controlled
word list recall) to high (e.g., puzzle). By this experiment, albeit with retention intervals that
finding, the benefits of spaced practice may be are measured in weeks rather than years.
muted for mathematical tasks that require more Task
than rote memory. As for mathematical tasks that In the experiments reported here, college
do rely solely on rote memory, a spacing effect students learned to calculate the number of
has been demonstrated. For example, Rea and unique orderings (or permutations) of a letter
 Mathematics Learning 4
sequence with at least one repeated letter. For Experiment 1
example, the sequence abbbcc has 60 The first experiment examined the benefit
permutations, including abbcbc, abcbcb, bbacbc, of distributing a given number of practice
and so forth. The solution is given by a formula problems across two sessions rather than
that is presented and illustrated in the Appendix. massing the same practice problems into one
While the number of permutations can also be session. As shown in Figure 1A, the Spacers
obtained by listing each possible permutation, attempted 5 problems in each of two sessions
this alternative approach fails when the number separated by one week, whereas the Massers
of permutations is large and the problem must be attempted the same 10 problems in session two.
completed in a relatively brief amount of time Each group received a tutorial immediately
(as in the present study). No student saw a given before their first practice problem, and the
letter sequence more than once. A total of 17 students were tested one or four weeks after their
different sequences were used in the two final practice problem.
experiments, and these are listed in the Method
Appendix. Participants. All three sessions were
Base Rate Survey completed by 116 undergraduates at the
We assessed the base rate knowledge of University of South Florida. The sample
this task for our experiment participants by included 95 women and 21 men. An additional
giving a brief test to a sample of students drawn 39 students completed the first session but failed
from the participant pool used in Experiments 1 to show for either the second or third session.
and 2. We expected that few if any of the None of the students participated in Experiment
students would be able to perform the task 2.
because we have never encountered this Design. There were two between-subjects
particular kind of permutation problem in an variables: Strategy (Space or Mass) and
undergraduate level textbook. Retention Interval (1 or 4 weeks). Thus, each
Method student was randomly assigned to one of four
Participants. The sample included 50 groups: Spacers with 1-week RI, Spacers with 4-
undergraduates at the University of South week RI, Massers with 1-week RI, and Massers
Florida. These included 43 women and 7 men. with 4-week RI.
None participated in Experiments 1 or 2. Procedure. The students attended three
Procedure. After a brief demographic sessions. At the beginning of the first session,
survey, students were given three minutes to each student was randomly assigned to one of
solve the following three problems: aabbbbb, the four conditions listed above. Students were
aaabbbb, and abccccc. These sequences yield not told what tasks awaited them in future
answers of 21, 35, and 42, respectively. sessions. It is not known whether some students
Results and Discussion practiced the procedure outside of the
Not surprisingly, none of the students experimental sessions, although there was no
correctly answered any of the three problems. extrinsic reward for test performance. If any self
Many of them attempted to solve the problems review did occur, we know of no reason why its
by listing every permutation, but none exhibited prevalence would vary among Spacers and
knowledge of the necessary formula. Thus, this Massers.
mathematics procedure appears to be unknown All students completed a tutorial, two
to the participant pool used in Experiments 1 and practice sets, and a test. Students read the tutorial
2. Furthermore, even if some participants in immediately before their first practice set. The
these experiments did possess any relevant tutorial included two pages of instruction (3 min)
knowledge before the experiment, the use of and written solutions to Problem 11 (3 min) and
random assignment ensured that their presence Problem 12 (2 min) of the Appendix. The first
would not be a confounding variable. practice set included Problems 1 – 5 of the
Appendix, in that order, and the second practice
 Mathematics Learning 5
set included Problems 6 – 10 of the Appendix, in (SE = 1.6%) while the Spacers averaged only
that order. Each problem appeared in a booklet 85% (SE = 3.2%), F (1, 108) = 6.79, p < .05, ηp2
with only one problem per page. Students were = .06. This difference was due to very poor
allotted 45 s to solve each problem. Immediately performance by a subset of Spacers who
after each attempt, students were shown the apparently forgot the procedure during the one
solution for 15 s before immediately beginning week inter-session interval. Thus, the one-week
the next problem. ISI introduced a disadvantage for the Spacers,
The first two sessions were separated by but this worked against the spacing effect rather
one week. The Spacers completed the first five- than for it.
problem practice set in session one and the Test. The mean percentage accuracy for
second five-problem practice set in session two. the five test problems is shown in Figure 1B. As
The Massers completed both practice sets in illustrated, the Spacers and Massers were not
session two, without any delay between the two reliably different at the 1-week RI, but the
sets. Spacers sharply outscored the Massers at the 4-
One or four weeks after the second week RI. This parity at one week caused the
session, every student returned for a test. The test main effect of Strategy (Space vs. Mass) to fall
included the five test problems listed in the short of statistical significance, F (1, 106) = 3.67,
Appendix in the order shown. The five test p = .06, ηp2 = .03. Not surprisingly, the main
problems were presented simultaneously, and effect of RI was reliable, F (1, 106) = 12.92, p <
students were allotted five minutes to solve all .001, ηp2 = .11. The reliance of the spacing effect
five problems. No feedback was given during the on retention interval was evidenced by an
test. interaction between ISI and RI, F (1, 106) =
Results 7.21, p < .01, ηp2 = .06. This pattern was further
Learning. The tutorial was sufficiently confirmed by Tukey tests showing that the
effective, as evidenced by performance on the difference between Spacers and Massers was
five problems given to every student reliable at the 4-week RI (p < .05) but not the 1-
immediately after the tutorial. Of the 116 week RI. These post hoc tests also showed that
students, 65 scored a perfect five, 27 scored four, the difference between the one- and four-week
12 scored three, 6 scored two, 3 scored one, and test scores was significant for the Massers (p <
3 scored zero. All further analyses excluded the .05) but not the Spacers.
students who correctly answered zero (n = 3) or Discussion
one (n = 3) of these five problems. It appears that For the longer retention interval of four
the three students with scores of one were weeks, the distribution of 10 practice problems
guessing, as four of the five correct answers were across two sessions was far more useful than the
multiples of ten. Notably, this inclusion criterion massing of all 10 problems in the same session.
of at least two correct is based solely on Thus, these data provide an instance of the
performance during the first session because the spacing effect for a non-rote mathematics
additional reliance on performance during the learning task in a non-confounded experiment.
second practice set (which was delayed for the At the one-week retention interval, there was no
Spacers) would have confounded the experiment. reliable difference between the two strategies
Naturally, the Massers and Spacers performed among students who were tested only one week
equivalently on the first practice set because the after learning. This result is consistent with
procedures for these groups did not diverge until previous findings demonstrating no spacing
after the first practice set was complete. effect or even massing superiority at sufficiently
Specifically, the Massers averaged 88% (SE = short retention intervals, as described in the
2.3%) and the Spacers averaged 87% (SE = introduction. Despite this ambiguity after one
2.5%), F < 1. week, though, the spacing superiority after four
However, for the second set of five weeks suggests that long-term retention, which is
practice problems, the Massers averaged 94%
 Mathematics Learning 6
the focus of the present paper, is better achieved that each of the nine problems was presented
by distributing practice problems across sessions. equally often. This was done to equate the
Experiment 2 selection of practice problems given to Lo and
The second experiment assessed the Hi Massers. In addition, the nine problems given
effect of overlearning on retention by varying the to the Hi Massers were presented in one of three
number of practice problems within a single different orders so that the first three problems
session. The Hi Massers attempted 9 practice corresponded to the only three problems given to
problems, whereas the Lo Massers attempted the same number of Lo Massers. The other
only 3 practice problems (as detailed in Figure aspects of the procedure, including the five-
1C). Thus, the Hi Massers relied heavily on problem test, were the same as those in
overlearning. Because this increase in the Experiment 1.
number of practice problems produced a Results
concomitant increase in time devoted to practice, Learning. The tutorial was again
this manipulation is effectively a manipulation of sufficient to produce learning, as demonstrated
total practice time. Students were tested either by students’ performance on the three practice
one or four weeks later. As detailed in the problems completed immediately after the
introduction, many researchers have found tutorial. Specifically, 65 of the 100 students
benefits of overlearning on a subsequent test, but correctly answered all three problems, 23 scored
the bulk of these experiments relied on relatively two, 10 scored one, and 2 scored zero. As in
brief retention intervals. Experiment 1, students with scores of zero or
Method one were excluded from further analysis.
Participants. All three sessions were As expected, there was no reliable
completed by 100 undergraduates at the difference between Hi and Lo Massers on the
University of South Florida. The sample first three problems because these two groups
included 83 women and 17 men, and none underwent the same procedure until after these
participated in Experiment 1. An additional 17 three problems were completed. Specifically, the
students completed the first session but failed to Hi Massers averaged 90% (SE = 2.3%) and the
show for the second session. Lo Massers averaged 88% (SE = 2.3%), F < 1.
Design. We manipulated two between- For the additional six practice problems given
subjects variables: Practice Amount (Hi or Lo) only to the Hi Massers, accuracy averaged 95%
and Retention Interval (1 or 4 weeks). Thus, each (SE = 1.3%).
student was randomly assigned to one of four Test. As shown in Figure 1D, there was
groups: Hi Massers with 1-week RI, Hi Massers virtually no difference between the Hi and Lo
with 4-week RI, Lo Massers with 1-week RI, and Massers on either the one- or four-week test.
Lo Massers with 4-week RI. Consequently, an analysis of variance revealed
Procedure. Each student attended two no main effect of Practice Amount (F < 1) and
sessions, separated by one or four weeks. At the no interaction between Practice Amount and
beginning of the first session, each student was Retention Interval (F < 1). Not surprisingly, the
randomly assigned to one of the four conditions main effect of retention interval was significant,
listed above. Each student then observed a F (1, 84) = 33.16, p < .001, ηp2 = .28.
tutorial consisting of screen projections that Discussion
included the complete solutions to Problems 10, The increase in the number of practice
11, and 12 of the Appendix, in that order. problems given during a single learning session
Immediately after this tutorial, all students began had virtually no effect on subsequent test scores
the practice problems. Each Hi Masser was given at either retention interval. This null effect of
nine problems (which were Problems 1 through overlearning is not well explained by a lapse in
9 of the Appendix), and each Lo Masser was attention by the Hi Massers during their
assigned three of these nine problems. The three additional six practice problems because these
problems assigned to each Lo Masser varied, so
 Mathematics Learning 7
problems were solved with 95% accuracy. Thus, proceed well beyond that minimally necessary
as fully described in the general discussion, these for an immediate, correct first reproduction” (p.
results provide no support for the oft cited claim 181). Fitts concluded that, “The importance of
that overlearning boosts long-term retention and continuing practice beyond the point in time
therefore cast doubt on the utility of mathematics where some (often arbitrary) criterion is reached
assignments that include many problems of the cannot be overemphasized” (p. 195). And Hall
same type. wrote, “The overlearning effect would appear to
General Discussion have considerable practical value since continued
The two experiments assessed the practice on material already learned to a point of
benefits of distributed practice and overlearning mastery can take place with a minimum of effort,
on subsequent test performance. In Experiment and yet will prevent significant losses in
1, distributing 10 practice problems across two retention” (p. 328). In contrast to these
sessions instead of massing all 10 problems in conclusions, the results of Experiment 2 revealed
the same session had no effect on one-week test no effect of overlearning on retention.
scores but virtually doubled four-week test Conceptually, the minimal effect of
scores. In Experiment 2, increasing the number overlearning on retention can be interpreted as an
of problems solved in a single session from three instance of diminishing returns. That is, with
to nine had virtually no effect on test scores on each additional amount of practice devoted to a
either the one- or four-week test. In brief, the single concept, there is an ever smaller increase
extra effort devoted to additional problems in test performance. Thus, after the initial
produced no observable benefit, whereas the exposure to a concept, the first one or two
distribution of a given number of number of practice problems might yield a large increase in
practice problems produced benefits without any a subsequent test score. Yet each additional
extra effort. practice problem provides an ever smaller gain
The results of previous experiments have until, ultimately, any additional practice within
provided little support for distributed practice the same session will yield very little gain.
with non-rote mathematics tasks. As detailed in It should be noted, though, that a small
the introduction, three previously published amount of overlearning may be useful if
experimental findings that are cited as instances overlearning is strictly defined as any practice
of a spacing effect with a non-rote mathematics beyond one correct problem. By this definition,
task are, in fact, confounded in favour of the overlearning occurs even when a student
spacing effect. The results of Experiment 1, correctly solves only two or three problems,
however, suggest that the superiority of which means that even the Lo Massers in
distributed practice over massed practice extends Experiment 2 relied on a small amount of
to these more abstract cognitive tasks. overlearning. Consequently, the results of
Consequently, we concur with those authors who Experiment 2 do not support the extreme view
have urged greater reliance on distributed that students should be assigned only one
practice as a means of boosting long-term problem of each type in a given session. Instead,
retention (Bahrick & Hall, 1991; Baddeley & the results suggest that students who correctly
Longman, 1978; Bjork, 1979, 1988; Dempster, solve several problems of the same kind have
1989; Reynolds & Glaser, 1964; Schmidt & little to gain by working more problems of the
Bjork, 1992: Seabrook, Brown, & Solity, 2005). same type within the same session. After these
With regard to overlearning, however, the first problems are solved correctly, students
present results strongly conflict with the could devote the remainder of the practice
numerous claims about its utility as a learning session to problems drawn from previous lessons
strategy (e.g., Fitts, 1965; Foriska, 1993; Hall, in order to reap the benefits of distributed
1989; Jahnke & Nowaczyk, 1998). Indeed, the practice.
strategy of overlearning is widely advocated. As A final caveat concerns two important
Jahnke and Nowaczyk advised, “Practice should limitations on the extent to which the present
 Mathematics Learning 8
finding will generalize. First, all of the number of practice problems, but only a few of
participants in the present experiments were these problems relate to the immediately
college students, and it is not known whether the preceding lesson. Additional problems of the
results of both experiments would have been same type then appear perhaps once or twice in
observed with much younger students. However, each of the next dozen or so assignments and
we suspect that young children would provide once again after every fifth or tenth assignment
qualitatively similar results because such thereafter. In brief, the number of practice
parallels have been observed in previous problems relating to a given topic is no greater
distributed practice experiments (e.g., Seabrook, than that of typical mathematics textbooks, but
Brown, & Solity, 2005; Toppino, 1991). Second, the temporal distribution of these problems is
because the experiments reported here relied on a increased dramatically.
test that required students to solve problems While the distributed practice format
identical to those presented during the practice should improve retention, it might also prove
session (albeit with different numerical values), more challenging than the massed practice
it is unknown whether the benefit of distributed format. With a massed practice format, students
practice or the futility of overlearning would who have solved the first few problems have
have occurred if the test had required students to little difficulty with the remaining problems of
apply their previous learning in novel ways (i.e., the same type. Indeed, they merely need to
assessed what is known as transfer). repeat the procedure. A distributed practice
The Organization of Practice Problems in format, however, ensures that each practice set
Mathematics Textbooks includes many different challenges. Thus, the
Many mathematics textbooks rely on a challenge and long-term returns of a distributed
format that fosters both overlearning and massed practice format provide an example of what
practice. In these textbooks, virtually all of the Bjork and his associates have called a “desirable
problems for a given topic appear in the difficulty” (Christina & Bjork, 1991; Schmidt &
assignment that immediately follows the lesson Bjork, 1992). Despite this difficulty, however,
on that topic. This format fosters overlearning some students might find the mixture of
because each assignment includes many problems more interesting than a group of
problems of the same kind. This format also similar problems.
fosters massed practice because further problems A distributed practice format is used in
of the same kind are rarely included in the Saxon series of mathematics textbooks (e.g.,
subsequent assignments. As an illustration, we Saxon, 1997). While numerous non-controlled
examined every problem in the most recent studies have compared Saxon textbooks to other
editions of four textbooks in pre-algebra textbooks, we are not aware of any published,
mathematics or introductory algebra that are very controlled experiments with these textbooks.
popular in the United States. The proportion of However, there may be little information to be
the problems within each assignment that gained from an experiment in which students are
corresponded to the immediately preceding randomly assigned to a condition that uses a
lesson, when averaged across assignments, Saxon or non-Saxon textbook because the
equalled between 75% and 92% for the four numerous differences between two such
books. Thus, the format of these practice sets textbooks would confound the experiment. For
facilitates overlearning and massing. example, if such an experiment did show
Fortunately, there is an alternative format superior retention for users of the Saxon
that minimizes overlearning and massed practice textbook, it is logically possible that this benefit
while emphasizing distributed practice, and it was the result of textbook features other than the
does not require an increase in the number of distributed practice format. (Neither author has
assignments or the number of problems per had any affiliation with Saxon Publishers;
assignment. With this distributed-practice however, the first author is a former mathematics
format, each lesson is followed by the usual
 Mathematics Learning 9
teacher who has used both Saxon and non-Saxon program record the date of each problem
mathematics textbooks in the classroom.) attempt.
Perhaps a more informative experiment In addition to its effect on retention, a
would compare two groups of students who distributed-practice format can also facilitate
underwent instruction programs that differed learning because it allows students ample time to
only in the temporal distribution of practice master a particular skill. For instance, if a student
problems. For example, a class of students could is unable to solve the problems of a given type in
be divided randomly into two groups, with each a single lesson, a distributed-practice format will
group participating in the same class activities. provide further opportunities throughout the
Every student would also receive a packet that year.
included the same lessons in the same order. The Conclusion
selection of practice problems would also be The results of Experiment 1 suggest that
identical, but the practice problems of each type the retention of mathematics is markedly
would be distributed or massed. improved when a given number of practice
Textbook publishers could adopt a problems relating to a topic are distributed across
distributed-practice format with little trouble or multiple assignments and not massed into one
cost. They would merely rearrange the practice assignment. Moreover, this benefit of distributed
problems in the next edition of their textbooks, practice can be realized without increasing the
regardless of whether the lessons are changed as number of practice problems included in a
well. Oddly, practice problems typically receive practice set typical of most mathematics
relatively little attention from publishers and textbooks. Specifically, rather than require
textbook authors, and the practice problems are students to work far more than just a few
often written by sub-contracted writers. Yet the problems of the same kind in the same session,
practice sets are as least as important as the which had no effect in Experiment 2, each
lessons. In fact, as many mathematics teachers practice set could instead include problems
will attest, a majority of their students never read relating to the most recent topic as well as
the lessons and instead devote all of their problems relating to previous topics. This
individual effort to the practice sets. distributed practice format could be easily
In addition to the implications for adopted by the authors of textbooks and CAI
textbook design, the benefits of distributed software.
practice are equally applicable to the design of Any resulting boost in students’
the algorithms used in computer-aided mathematics retention might greatly improve the
instruction (CAI). Unlike textbooks, the mathematics achievement, and there is little
programs can provide individualized training and doubt that the mathematics skills of most
error-contingent feedback, and an increasing students need improving. In one recent report on
number of educators and agencies have urged mathematics achievement, less than one third of
greater reliance on such technologies (e.g., a sample of U.S. students received a rating of “at
Department for Education and Skills, United or above proficient” (Wirt et al., 2004). Such
Kingdom, 2003). Yet virtually all currently reports often lead people to conclude that
available CAI programs are designed to foster students are not learning, but it may be that many
learning rather than retention. Of course, such mathematical skills and concepts are learned but
programs could be easily adapted to incorporate later forgotten. The prevalence of such forgetting
distributed rather than massed practice, and may partly reflect the widespread reliance on
students’ compliance to a distributed practice practice schedules that proved to be the worst
schedule could be verified by ensuring that the strategies in the experiments reported here.
 Mathematics Learning 2
References
Baddeley, A. D., & Longman, D. J.A. (1978). Dempster, F. N. (1989). Spacing effects and
The influence of length and frequency of their implications for theory and practice.
training sessions on the rate of learning to Educational Psychology Review, 1, 309-330.
type. Ergonomics, 21, 627-635. Department for Education and Skills, United
Bahrick, H. P., Bahrick, L.E., Bahrick, A.S., & Kindgom (2003). Fulfilling the potential.
Bahrick, P. E. (1993). Maintenance of (DfES publication 0265 / 2003). DfES
foreign-language vocabulary and the spacing Publications.
effect. Psychological Science, 4, 316-321. Donovan, J. J., & Radosevich, D. J. (1999). A
Bahrick, H. P., & Hall, L. K. (1991). Lifetime meta-analytic review of the distribution of
maintenance of high school mathematics practice effect: Now you see it, now you
content. Journal of Experimental don't. Journal of Applied Psychology, 84,
Psychology: General, 120, 20-33. 795-805.
Bjork, R. A. (1979). Information-processing Driskell, J.E., Willis, R. P., & Copper, C. (1992).
analysis of college teaching. Educational Effect of overlearning on retention. Journal
Psychologist, 14, 15-23. of Applied Psychology, 77, 615-622.
Bjork, R. A. (1988). Retrieval practice and the Earhard, B., Fried, C., & Carlson, G. (1972).
maintenance of knowledge. In M.M. Interference, overlearning, and anticipation
Gruneberg, P.E., Morris, & R.N. Sykes time. Journal of Experimental Psychology,
(Eds.). Practical aspects of memory II 94, 345-347.
(pp. 391-401). London: Wiley. Fishman, E. J., Keller, L., & Atkinson, R. C.
Bloom, K.C., & Shuell, T.J. (1981). Effects of (1968). Massed versus distributed practice in
massed and distributed practice on the computerized spelling. Journal of
learning and retention of second- Educational Psychology, 59, 290-296.
language vocabulary. Journal of Fitts, P. M. (1965). Factors in complex skill
Educational Research, 74, 245-248. training. In R. Glaser (Ed.), Training
Bromage, B. K., & Mayer, R. E. (1986). research and education (pp. 177-197). New
Quantitative and qualitative effects of York: Wiley.
repetition on learning from technical Foriska, T. J. (1993). What every educator
texts. Journal of Educational Psychology, should know about learning. Schools in the
78, 271-278. Middle, 3, 39-44.
Carpenter, S.K., & DeLosh, E.L. (2005). Gay, L. R. (1973). Temporal position of reviews
Application of the testing and the spacing and its effect on the retention of
effects to name learning. Applied mathematical rules. Journal of Educational
Cognitive Psychology, 19, 619-636. DOI: Psychology, 64, 171-182.
10.1002/acp.1101 Gilbert, T. F. (1957). Overlearning and the
Christina, R.W., & Bjork, R.A. (1991). retention of meaningful prose. Journal of
Optimizing long-term retention and General Psychology, 56, 281-289.
transfer. In D. Druckman & R.A. Bjork Glenberg, A. M., & Lehmann, T. S. (1980).
(Eds.), In the mind’s eye: Enhancing Spacing repetitions over 1 week. Memory &
human performance (pp. 23-56). Cognition, 8, 528-538.
Washington DC: National Academy Greene, R. L. (1989). Spacing effects in
Press. memory: evidence for a two-process account.
Cull, W. L. (2000). Untangling the benefits of Journal of Experimental Psychology:
multiple study opportunities and repeated Learning, Memory and Cognition, 15, 371-
testing for cued recall. Applied Cognitive 377.
Psychology, 14, 215-235.
 Mathematics Learning 2
Grote, M. G. (1995). Distributed versus massed Rohrer, D., Taylor, K., Pashler, H., Wixted, J. T,
practice in high school physics. School & Cepeda, N. J. (2005). The effect of
Science and Mathematics, 95, 97-101. overlearning on long-term retention. Applied
Hall, J. F. (1989). Learning and memory, 2nd Ed. Cognitive Psychology, 19, 361-374. DOI:
Boston: Allyn & Bacon. 10.1002/acp.1083
Jahnke, J.C., & Nowaczyk, R. H. (1998). Rose, R. J. (1992). Degree of learning,
Cognition. Upper Saddle River, NJ: Prentice interpolated tests, and rate of forgetting.
Hall. Memory & Cognition, 20, 621-632.
Kratochwill, T., Demuth, D.M., & Conzemius, Saxon, J. (1997). Algebra I (3rd Ed.). Norman,
W.C. (1977). The effects of overlearning on OK: Saxon Publishers.
preschool children's retention of sight Schmidt, R. A., & Bjork, R. A. (1992). New
vocabulary words. Reading Improvement, 14, conceptualizations of practice: Common
223-228. principles in three paradigms suggest new
Krueger, W. C. F. (1929). The effect of concepts for training. Psychological Science,
overlearning on retention. Journal of 3, 207-217.
Experimental Psychology, 12, 71-78. Seabrook, R., Brown, G.D.A., & Solity, J.E.
Krug, D., Davis, T. B., & Glover, J. A. (1990). (2005). Distributed and massed practice:
Massed versus distributed repeated reading – From laboratory to classroom. Applied
a case of forgetting helping recall. Journal of Cognitive Psychology, 19, 107-122. DOI:
Educational Psychology, 82, 366-371. 10.1002/acp.1066
Postman, L. (1962). Retention as a function of Toppino, T. C. (1991). The spacing effect in
degree of overlearning. Science, 135, 666- young children’s free recall: Support for
667. automatic-process explanations. Memory &
Rea, C. P., & Modigliani, V. (1985). The effect Cognition, 19, 159-167.
of expanded versus massed practice on the Wirt, J., Choy, S., Rooney, P., Provasnik, S.,
retention of multiplication facts and spelling Sen, A., & Tobin, R. (2004). The Condition
lists. Human Learning: Journal of Practical of Education 2004 (NCES 2004-077). U.S.
Research & Applications, 4, 11-18. Department of Education, National Center
Reynolds, J. H., & Glaser, R. (1964). Effects of for Education Statistics. Washington, DC:
repetition and spaced review upon retention U.S. Government Printing Office.
of a complex learning task. Journal of
Educational Psychology, 55, 297-308.
 Mathematics Learning 2
Appendix

Students were taught to calculate the number of unique orderings (or permutations) of a letter sequence

with at least one repeated letter (e.g., abbbcc). For n items and k unique items, the number of

permutations equals n! / (n1! n2! … nk!), where ni = number of repetitions of item i. For example, abbbcc

includes six letters (n = 6) and three unique letters (k = 3), and the letters a, b, and c appear 1, 3, and 2

times, respectively (n1 = 1, n2 = 3, n2 = 2). Thus, by the formula, the number of permutations equals

6! / (1! 3! 2!)

= (6Χ5Χ4Χ3Χ2Χ1) / [(1)Χ(3Χ2Χ1)Χ(2Χ1)]

= (6Χ5Χ4) / (2)

= 60.

Tutorial and Learning Session (see the procedures of each experiment for details)
1. abccc 20
2. abcccc 30
3. aabbbbb 21
4. abbcc 30
5. aaabbb 20
6. aabbbb 15
7. aabb 6
8. abbccc 60
9. abccccc 42
10. aabbbbbb 28
11. abbcccc 105
12. abcccccc 56

Test
abcc 12
aabbb 10
aabbcc 90
aaabbbb 35
aaaabbbb 70
 Experiment 1
A Learning Procedure B Test Results
100%
1 week apart

9
75% Spacers
70%

Accuracy
session one session two 64%

Spacers 5 problems 5 problems Massers 32%

Massers − 10 problems

0%
1 4
Retention Interval (weeks)

Experiment 2

C Learning Procedure D Test Results

100%

session one 69%

Accuracy

67%
Hi Massers
Hi Massers 9 problems
Lo Massers 3 problems
Lo Massers 28%
27%

0%
1 4
Retention Interval (weeks)

Figure 1. Learning Procedure and Test Results for Experiments 1 and 2. Accuracy

represents the mean percentage correct. Error bars reflect plus or minus one standard error.

Figure 1