International Journal of Educational Development 81 (2021) 102326

Contents lists available at ScienceDirect

International Journal of Educational Development

journal homepage: www.elsevier.com/locate/ijedudev

Modelling the long-run learning impact of the Covid-19 learning shock:

Actions to (more than) mitigate loss
Michelle Kaffenberger
University of Oxford, Oxford, United Kingdom

A R T I C L E I N F O A B S T R A C T

Keywords: This paper uses a calibrated “pedagogical production function” model to estimate the potential long-term losses
International education to children’s learning from the temporary shock of Covid-19 related school closures. It then models possible gains
Learning loss from two mitigation strategies. Without mitigation, children could lose more than a full year’s worth of learning
Remediation
from a three-month school closure because they will be behind the curriculum when they re-enter school and will
Covid-19
fall further behind as time goes on. Remediation when children return to school reduces the long-term learning
loss by half, but still leaves children more than half a year behind where they would have been with no shock.
Remediation combined with long-term reorientation of curriculum to align with children’s learning levels fully
mitigates the long-term learning loss due to the shock and surpasses the learning in the counterfactual of no
shock by more than a full year’s worth of learning. Systems need to begin planning now for remediation pro­
grammes, and as they do so they should build programmes and train teachers in ways that can continue to
produce benefits beyond the period immediately following reopening.

1. Introduction large deficits in later test scores, suggesting affected children learned
less each year after they returned to school because of the short-term
At its peak the Covid-19 pandemic forced more than 1.6 billion interruption (Andrabi et al., 2020). One possible explanation is that
children temporarily out of school. While many education systems have the curriculum and instruction did not adapt to the children’s lower
attempted varying degrees of remote learning, it is widely accepted that learning levels upon re-entry into school and hence the affected children
the closures will produce substantial losses in learning (World Bank, fell further and further behind.
2020; Kuhfeld et al., 2020). A serious concern is that these short term Leading education experts have called for adaptation of instruction
learning losses could continue to accumulate after children return to when children return following the Covid-19 related closures. Rukmini
school, resulting in large and permanent learning losses as many chil­ Banerji, CEO of Pratham, the NGO in India which pioneered the
dren who fall behind during school closures never catch up. “Teaching at the Right Level” approach, has said education systems
A recent study suggests that even temporary school closures can should focus on “helping children catch up on basic foundational skills”
result in large medium-term lost learning. Andrabi et al. (2020) analyse when children return to school (World Bank Live, 2020). The World
the impact of the 2005 Pakistan earthquake on children’s learning four Bank has called for education systems to begin planning for large-scale
years later by comparing households that were close to the fault line remedial programmes (World Bank, 2020), and a consortium made up of
with similar households that were farther away and not affected by the UNESCO, UNICEF, the World Bank, the World Food Programme, and
quake. Schools in the affected area were closed for an average of 14 UNHCR has called for remediation to mitigate learning loss with a focus
weeks, a little more than 3 months. However, four years later children in on literacy and numeracy for primary age children (UNESCO et al.,
the affected areas were not just three months behind, they were the 2020).
learning equivalent of 1.5 years of schooling behind.1 How much learning might be lost in the long run from the Covid-19
The direct effect of the school closures alone cannot account for such shock if nothing is done, especially if losses accumulate as the Pakistan

E-mail address: [email protected].

1
Their test scores were 0.24 standard deviations behind unaffected children and the average annual gain is only .15. The study also finds that, due to government
and international relief efforts, affected households were no worse off economically than unaffected households (in terms of wealth, consumption, or infrastructure),
so the learning affects are not consequences of economic hardships.

https://doi.org/10.1016/j.ijedudev.2020.102326
Received 9 October 2020; Received in revised form 2 December 2020; Accepted 6 December 2020
Available online 18 December 2020
0738-0593/© 2020 The Author. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

study suggests they could? And how much of a difference might miti­ The PPF is what, on average, child i with skill level s would learn if
gation strategies make? This paper uses an existing pedagogical pro­ they attended grade G. In general terms this is represented as:
duction function model (Kaffenberger and Pritchett, 2020), calibrated to
Learning process (LP) = LPG(si)
replicate learning trajectories in low- and middle-income countries, to
model the possible outcomes. To model learning loss, I introduce a Drawing on the findings of the emerging literature on learning pro­
learning loss shock for children currently in grade 3 and simulate how files, Kaffenberger and Pritchett (2020) assumes a trapezoidal functional
their learning is affected through grade 10.2 form for the PPF, as in the equation:
I find that if learning in grade 3 is reduced by one-third, roughly the ⎧
amount of time many children are likely to be out of school, learning ⎪
⎪
⎪
⎪
levels in grade 10 (compared to a counterfactual of the same children ⎪
⎪
⎨ (
with no shock) are a full year lower. This is similar to the cumulative PPF(LP(w, h, r, π ), s ) = hmin + r si − (πG
G i
⎪
learning loss identified by Andrabi, Daniels, and Das (2020). Second, I ⎪
⎪
⎪
⎪
find that if learning in grade 3 is reduced by half, which could reflect ⎪
⎩
missing one third of a year of school plus additional learning regression w
while away from school (as in the phenomenon of “summer learning 0 if si < π G −
2
loss” (Slade et al., 2017)), then learning in grade 10 is 1.5 years lower w) w w
than the counterfactual of no shock. −
2
) if πG −
2
< si < π G +
2
I then model two remediation approaches. w
i G
The first models short-term remediation efforts when these children 0 if s > π +
2
return to grade 4. It assumes one-third of the grade 3 curriculum is
covered during grade 4 before moving on to grade 4 topics. Starting in Where the learning in grade G of student i of initial skill s is a function
grade 5, instruction reverts to the previously established (pre-pandemic) the width w, height h, slope r, and center πG of the trapezoid, as illus­
curriculum and instructional levels. This is a “short-term” remediation trated in Fig. 1.
model. Modelling this with the more conservative assumption of the loss Two distinct features follow from this functional form. First, the PPF
of one-third of grade 3 learning from school closures, such short-term assumes there is a range of initial skill levels within which children learn
remediation mitigates about half of the grade 10 learning deficit, and above and below which they do not. If the instructional process is
reducing the long-term impact of the shock to one-half of a school year. too advanced relative to student skill level (e.g. teaching division to
The final scenario models an instruction reorientation strategy which children who cannot recognize numbers) or too rudimentary (e.g.
combines short-term remediation with long-term adaptation of in­ teaching number recognition to students ready for geometry) no new
struction to children’s learning levels. The steps that education systems skills are gained. The PPF or instructional process at grade G is centred
will need to take to conduct remedial education, including instituting on a specific skill level, πG , and the width of the PPF, the range of initial
formative assessments to identify children’s learning levels, training and child skills over which the instructional process produces learning, is the
empowering teachers to conduct such assessments and adapt their in­ parameter w. Therefore a child too far behind (si < πG − w2 ) will learn
struction and pedagogical practices to students’ levels and needs, and nothing from attending grade G (Fig. 1).
prioritising children’s attainment of essential skills, are well-proven Second, the trapezoidal shape has a slope parameter, r, so that
strategies for improving learning outside of a crisis context (Teaching learning can vary across the initial student distribution. Kaffenberger
at the Right Level (TaRL), 2018; Piper et al., 2018). The final model and Pritchett (2020) assumes an upward sloping trapezoid, with r>0, so
considers the outcomes if systems both conduct remedial instruction in that high performers learn more per year than low performers.3 hmin is
grade 4 as described above and reorient instruction and practices to the amount learned by the child with the lowest initial skill level that
children’s learning levels on a long-term basis. This scenario not only
fully mitigates the effect of the shock but increases grade 10 learning
above the counterfactual of no shock by more than a full year’s worth of
instruction.
Without the urgent and immediate attention of education systems to
the question of how they will handle the learning losses from the tem­
porary school closures, the consequences for today’s children will be
long-run and large. Actions to protect children from these losses must be
a top priority, even while the crisis persists.

2. Modelling learning

Modelling learning requires a specification of the parameters that

drive the learning process. Changes to the parameters then allow
modelling of counterfactual scenarios, such as shocks to the learning
process or various policy priorities or scenarios. Kaffenberger and
Pritchett (2020) develops a pedagogical production function (PPF)
which models the learning gained by children at different points in a
student distribution in a year of schooling. In this paper, I use this PPF to Fig. 1. Modelling the learning process with a trapezoidal pedagogical pro­
duction function.
model the Covid-19 learning shock. This section provides a brief over­
view of the model, and more details can be found in Kaffenberger and
Pritchett (2020).

3
This is based on the literature on learning profiles showing that low per­
2
The choice of grade 3 is illustrative, as some learning has already occurred, formers often learn less per year than high performers. It could just as easily be
but enough years remain to model long-term learning loss. The shock could be reversed, such as in a remedial environment, and modelled so that low per­
modelled for children in any grade. formers learn more and high performers learn less.

2
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

learns anything at all, and hmax is the amount learned by the child with Table 1
the highest initial skill level that learns anything. Calibrated parameters for reproducing average PISA-D scores.
Fig. 1 illustrates this trapezoidal PPF learning process. Parameter PISA-D calibrated parameters
A final parameter, pace, p, represents the shift in the PPF from one
W (width) 153
grade to the next, as the level of instruction shifts to the next grade level. hmax 49
To model learning, the learning process is iterated by applying the PPF hmin 26
to an initial distribution of student skills to produce a new distribution r (slope) 0.15
based on the learning acquired in grade G. These dynamics are illus­ P (pace) 45
N(π1, σ1) N(020)
trated in Fig. 2. The PPF then shifts to the right according to the pace p to
produce the learning for grade G+1.
Kaffenberger and Pritchett (2020) calibrate the model to replicate countries, to model dropout after each grade. In this paper I assume
average grade 10 learning in mathematics in the seven low- and enrolment and dropout stay constant at pre-pandemic levels, making the
middle-income countries that participated in the PISA for Development learning loss estimates optimistic if some children do not return to
(PISA-D) assessment.4 This is just one option for calibration and allows school (more details on this in Section III.C.).
us to model long term learning (through at least grade 10).5 For the
current modelling, this calibration allows us to model a “typical” 3. Modelling the Covid-19 learning shock and mitigation
learning process in a low- or middle-income country.6 approaches
PISA assesses children who are 15 years old and in school and in at
least grade 7. Eligible 15-year-olds are on average in grade 10. In OECD I use the calibrated PPF to model five scenarios. The first is the
countries, as a comparison, 89 percent of 15-year-olds are eligible, and counterfactual of cohort learning with no shock, representing business-
PISA is standardized so that the mean score of these participating chil­ as-usual schooling. This serves as the comparison point for learning loss
dren is 500 and the standard deviation is 100. Among the PISA-D due to the shock – it is the counterfactual of no schooling disruption.
countries, 43 percent of 15-year-olds were eligible, the average score Then I model two different learning loss scenarios, followed by two
of participating children was 324, and the standard deviation was 74. In mitigation approaches.
our calibration, we assume dropout is endogenous and that low per­
formers dropout first. More details on the calibration process and
dropout assumptions are provided in Kaffenberger and Pritchett (2020). 3.1. How much long-term learning may be lost?
We calibrate the PPF so that the top 43 percent of the grade 10 distri­
bution roughly replicates the observed PISA-D results, with the combi­ Using the calibrated PPF described in Section II, Kaffenberger and
nation of parameters that comes closest to replicating the PISA-D results Pritchett (2020) estimates average grade 10 cohort learning among
given in Table 1. In Kaffenberger and Pritchett (2020) and in the PISA-D countries. While the PISA-D assessment only provides informa­
following modelled scenarios, we use grade attainment data from the tion on the learning of the portion of the cohort that took the assessment,
World Bank’s EdAttain database, averaged across the seven PISA-D with the parameterized model it is possible to estimate learning trajec­
tories and learning outcomes among the full cohort of in- and
out-of-school children. This will serve as the base case counterfactual of
learning with no shock.
The calibrated PPF produces average cohort learning at age 15 of 213
and cohort standard deviation of 126, on the PISA scale of OECD average
500 and standard deviation 100. This mean is much lower and the
standard deviation much larger than those observed for the PISA-D
eligible population, as cohort learning now includes a much longer
left tail of low performers (i.e. those who never started or dropped out of
school and were therefore ineligible for PISA-D). This distribution of
cohort learning is the counterfactual to which the following modelled
outcomes are compared.
To model Covid-19 related learning loss, I introduce a shock for the
cohort of grade 3 students and model their learning trajectories and
outcomes through grade 10.7 Today’s grade 3 students will be in grade
10 in 2027, three years before the SDG target completion date, making
this a relevant cohort for understanding not only implications for long
term learning loss but also repercussions for reaching international
Fig. 2. Initial and end of grade student skill distribution. learning goals. I reduce their grade 3 learning gains by one-third, the
equivalent of about a three-month school closure,8 and, in this initial
scenario, assume no remedial efforts are made when children return but
that schools return to “business as usual” curriculum and teaching. I also
4
The PISA-D programme adapted the traditional PISA assessment to lower
assume, for simplicity, (here and in the subsequent scenarios) no addi­
income countries and included capacity building for participating countries to tional school dropout due to the closures so that dropout follows the
conduct the assessment. Participating countries included Cambodia, Ecuador,
Guatemala, Honduras, Paraguay, Senegal, and Zambia.
5 7
The model could be calibrated to other data sources, and this will be a focus The model could be run with a shock to any grade cohort. Grade 3 was
in future work. chosen as some learning has already occurred, and enough schooling remains to
6
As a comparison point, for the six PISA-D countries that also have learning model long term consequences of the shock.
8
poverty measures by the World Bank, the average learning poverty level is 67 A similar assumption has been used in other modelling efforts (Cummiskey
percent. Across all low- and middle-income countries with learning poverty and Stern, 2020). Given that the majority of low-income countries have no
measures learning poverty is 53 percent and in low-income countries it is 89 announced distance learning plan, it is not unreasonable to assume that
percent, so the PISA-D countries are within reasonable levels for these income learning will effectively cease while children are out of school (Carvalho and
levels. Hares, 2020).

3
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

same trajectory as in the counterfactual. Assuming no changes in 3.2. How much difference could remedial efforts make?
dropout makes this a conservative estimate of learning loss, as discussed
further in Section III.C. There are steps that can be taken to mitigate some or all of these
Similar to Andrabi, Daniels, and Das (2020), the simulation finds that devastating outcomes. It has been widely acknowledged that remedial
reducing learning by one-third of a school year in grade 3 reduces later efforts will be needed when children return to school. A joint framework
learning by a much larger amount. When this cohort of current grade 3 by UNESCO, UNICEF, the World Bank, the World Food Programme, and
students reaches grade 10, their learning on average is a full year lower UNHCR has called on education systems to implement large scale
than what it would have been had there been no shock (Fig. 3).9 While remediation programmes (UNESCO et al., 2020). The World Bank has
this may at first seem extreme, the mechanism is clear. The lost learning said that where full cohorts have missed content, especially in founda­
puts children behind the curriculum, and without remediation they tional subjects, “plans for teaching essential missed material should be
cannot keep up. They begin to fall outside the range of the PPF (i.e. integrated with plans for resuming progress through the curriculum,”
outside the range of the curriculum and instruction) and cannot engage (World Bank, 2020). How much could such remediation efforts mitigate
with the material. By grade 10, nearly three-quarters (72 percent) of the long-term effects of the learning shock?
children who are still in school have fallen outside the range of the PPF For modelling mitigation, I use the more conservative learning loss
and hence are making no learning gains (Fig. 4).10 assumptions from Section III.A., reducing grade 3 learning by one-third,
Sustainable Development Goal 4 calls for all children to achieve and first model remediation when children re-enter school in grade 4.
minimum proficiency in reading and mathematics by 2030. One defi­ This assumes that education systems cover the material missed during
nition for minimum proficiency, established by UNESCO Institute for school closures when children return to school. For example, if grade 3
Statistics (UIS), is achieving a Level 2 on the PISA scale, roughly children missed the last third of the school year, when they re-enter
equivalent to a score of 400 (UNESCO, 2018). Because the model is school in grade 4, they will cover the part of the grade 3 material they
calibrated to the PISA scale, it is possible to estimate the proportion of missed before moving on to the new grade 4 material. This is modelled
children who will achieve the SDG in each modelled scenario. The by reducing the curricular pace from grade 3 to grade 4 by one-third,
percent of the current grade 3 cohort that would reach the SDG goal of representing some of the grade 3 topics being covered in grade 4.
minimum proficiency in mathematics by grade 10 drops from 7 percent After grade 4 the curriculum reverts to the previously established levels
with no shock to just 3 percent reaching the SDG goal with the shock (and pace). This is the equivalent of a short-term remediation effort.
(Fig. 5). Short-term remediation makes up for some of the long-term learning
I next model another variation of learning loss, reducing grade 3 loss. Average grade 10 cohort learning with remediation is half of a
gains by one-half. This could be thought of as the equivalent of direct school year higher than without remediation. It does not, however, fully
learning loss from school closure and additional learning regression make up for the learning loss of the shock. This is because instructional
during the time out of school. “Summer learning loss” is an established time required for remediation reduces the time available for the regular
phenomenon in high-income countries. A recent analysis suggests that in grade level instruction, so learning losses are partially but not fully
the United States, school closures due to Covid-19 could mean children compensated. Cohort learning in grade 10 is still 0.55 years behind the
return in the next school with less than 50 percent of the learning gains counterfactual of no shock (Fig. 3). In the remediation scenario the
they would have had in math, and for some grades children could be a percent of the grade 10 cohort who reach the SDG for mathematics is 4
full year behind the gains they would have under normal circumstances percent, compared with 3 percent with the shock and no remediation,
(Kuhfeld and Tarasawa, 2020). Research on this topic is more sparse for and 7 percent in the counterfactual with no shock (Fig. 5).
lower income countries, but at least one study indicates that such “grade Finally, I model a scenario of remediation plus longer term “reor­
transition” loss does occur in lower income countries and can be severe ientation” of instruction. This scenario acknowledges the opportunity
(Slade et al., 2017). These suggest that even a 50 percent reduction in that education systems have to “build back better,” and particularly to
the grade 3 gains may be conservative in terms of the learning loss do so building on capabilities they may gain in implementing remedi­
students experience due to Covid-19 closures. ation programmes. Evidence suggests that curricula, and resulting in­
In this scenario, the grade 10 learning deficit far surpasses the initial struction, in many developing (and developed) countries are
loss of one-half of a year’s learning. In grade 10, today’s grade 3 cohort overambitious, covering many topics with limited time allocated to
has gained 1.5 years less learning than if the shock had not occurred. The each. Teachers under pressure to complete the curriculum must rush
percent of in-school children who have fallen outside the range of the through the content before students can fully grasp the new knowledge.
PPF (i.e. behind the level of instruction) and are learning nothing is As a result, many children miss out on foundational and essential skills
higher in every grade following the shock, reaching more than 80 and fall further and further behind (Beatty and Pritchett, 2015; Glewwe
percent in grade 10. Finally, this larger shock further reduces the percent et al., 2009). This is represented in the model as children fall outside the
of the cohort who reaches the SDG target for math to just 2 percent. The range of the PPF, unable to keep up with the pace of instruction, and stop
percent of the cohort with learning levels below 200, considered very making learning gains.
low, rises to 73 percent. In Uganda, for example, a study of the national curriculum found
that foundational English competencies receive very little emphasis
before children are expected to move on to higher order skills (Atuhurra
and Alinda, 2017). In 2015 Tanzania reformed its grade 1 and 2
9
Treating grade 1 as a baseline, the average in-school grade 1 child in the curricula, which at the time consisted of eight subjects including
counterfactual of no shock learns 36 points on the PISA-like scale per year, and Vocational Skills and Information and Communication Technology. The
in this shock scenario grade 10 children are 38 points behind the counterfactual reform radically simplified the curriculum, placing 80 percent of
of no shock. instructional time on foundational literacy and numeracy, and pre­
10
The long term learning losses, in terms of years of learning lost, are larger liminary evidence shows large gains in the foundational subjects as a
for children in lower grades at the time of the shock because the losses have result (Mbiti and Rodriguez-Segura, 2020). These studies, and others,
more years of schooling across which to accumulate, and smaller for children in
suggest that reorienting curriculum to children’s ability levels and
higher grades. For example, alternative specifications show that an initial loss
ensuring adequate coverage of topics so that children can gain compe­
of one-third of a year for grade 1 students results in grade 10 learning that is 1.5
years lower, while initial loss of one-third of a year for grade 6 students results tency can substantially improve learning. The “Teaching at the Right
in grade 10 learning that is 0.5 years lower. An interactive data visualization Level” approach, pioneered by Pratham, provides further evidence that
tool is available for exploration of further scenarios: https://riseprogramme. such tailoring of instruction to children’s ability levels can have large
org/tools/simulating-learning impacts on children’s learning, and such programmes are now

4
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

Fig. 3. Modelling long-term lost learning from Covid-19 shock and mitigation strategies for the current grade 3 cohort: Equivalent years of learning behind/ahead in
grade 10 compared to counterfactual of no shock.

Fig. 4. Percent of in-school children learning zero, due to falling behind the level of instruction, at each grade level. Covid-19 learning loss shock occurs in grade 3.

expanding and scaling throughout Africa (Teaching at the Right Level pedagogy (Piper et al., 2018), to help teachers put new practices and
(TaRL), 2018). curriculum into practice. Such efforts will look different in different
If systems implement effective remediation when schools reopen, systems, and the exact form would need to take the context and the
many of the building blocks for such adaptations of instruction to meet system’s existing capabilities and constraints into account.
children where they are would be put in place. For a system to imple­ In such a scenario, in which an education system not only conducts
ment remediation efforts effectively, as modelled in the first mitigation remediation immediately upon return to school, but also carries the
scenario, teachers and schools require some ability to conduct formative capabilities it gains in formative assessment, adaptation of instruction,
assessments, to determine children’s learning levels when they return, and ensuring all children master foundational skills into the future, what
and to identify the subjects in need of remedial attention. It also requires could be the result for long-term learning? In this final scenario I build
the ability to adapt instruction to accommodate these needs, tailor on the first mitigation scenario, modelling the shock that reduces grade
content, and adjust curriculum. Doing so will require that teachers 3 learning gains by one-third plus remediation as described above. In
receive training and professional development and that they are allowed addition, I reorient the curriculum for the remaining school years
and empowered to adapt the curriculum they typically are required to through grade 10 and assume that after grade 4 the pace is “optimised”
fully cover. to children’s learning levels. The “optimised” curricular pace draws
If systems embrace the current crisis as an opportunity to “build back from Kaffenberger and Pritchett (2020) which, for the calibrated PPF
better”, they could build on these remediation efforts to maintain good model, identifies the curricular pace that maximizes grade 10 learning
practices in the long run. This could include ongoing use of formative outcomes. This optimised pace is a pace of 35 on the scale used for the
assessments as a part of standard instructional practices, adjustment (or model, a reduction of 22 percent from a pace of 45 which reproduces
reform) of curriculum to better match the level and pace of children’s PISA-D learning outcomes (and is “overambitious” per children’s
learning, and ongoing support, such as through coaching or structured learning levels). This implies better alignment between instruction and

5
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

Fig. 5. Percent of grade 10 cohort scoring in each band.

Remediation + instruction reorientation produces learning
gains far surpassing the counterfactual of no shock. On PISA-
like scale; 400 roughly corresponds with the SDG goal of
“minimum proficiency”. Note: The PISA scale is standardized
to a mean among OECD countries of 500 and standard devia­
tion of 100. The SDG goal of minimum proficiency roughly
corresponds to a score of 400. Children scoring 50 or less either
never started school or dropped out prior to grade 3 and so are
unaffected by the shock and mitigation efforts.

children’s learning levels and paces in each grade. By reducing the pace, The results from all four scenarios and the counterfactual of no shock
fewer children fall behind the level of instruction because sufficient time are summarized in Table 2. It is clear that across all measures, the
is spent on content before moving on. With the slower pace, more remediation plus instruction reorientation scenario performs better even
children stay in the range of the PPF (and continue learning) longer, than the counterfactaul of no shock, truly representing a "build back
increasing learning outcomes. better" scenario.
This scenario that combines short-term remediation with long-term On one hand, it may be surprising that relatively simple efforts to
reorientation of instruction to children’s learning levels not only fully tailor instruction to children’s ability levels could produce such large
mitigates the long-term learning loss due to the shock, but also surpasses learning gains. These results, however, are in line with the large impacts
the learning in the counterfactual of no shock by more than a full year’s achieved by many programmes that have worked to reorient instruction
worth of learning. Remediation combined with long term reorientation to children’s ability levels. Pratham’s TaRL approaches, for example,
of instruction produces average cohort learning of 259 in grade 10 on have effect sizes ranging from 0.08 to 0.70 standard deviations for
the PISA-like scale, a whopping 2.3 years’ more learning than if the relatively short-term programmes (ranging from 10-day remedial camps
shock had gone unmitigated (Figure 3). This is also 1.3 years’ more to a full year of reoriented instruction for foundational subjects) (TaRL,
learning than the counterfactual of no shock occurring. With remedia­ 2018). A computer-aided learning programme that adapted instruction
tion and instruction reorientation, 27 percent of the cohort achieves the to children’s individual learning levels achieved improvements of 0.29
SDG—nine times more than had the shock gone unmitigated, and even standard deviations over just a 4.5 month period in India (Muralidharan
nearly four times more than the counterfactual of no shock (Figure 5). et al., 2019). The Tusome programme in Kenya achieved impacts of
This scenario achieves such large learning gains because the reor­ 0.6–1.0 standard deviations in English and Kiswahili learning outcomes
ientation of instruction enables more children to continue learning for after one year through a multifaceted programme that reoriented liter­
longer. The percentage of in-school grade 10 children who are learning acy instruction to ensure all children were learning (Freudenberger and
zero (because they have fallen below the range of instruction) in the Davis, 2017; Piper et al., 2018; Wilichowski et al., 2020). It is entirely
remediation plus reorientation scenario is just 16 percent, compared conceivable that adjusting curriculum pacing and instructional focus to
with 72 percent with the unmitigated shock, and 43 percent in the be in line with children’s pace of learning could produce such large
counterfactual of no shock (Fig. 4). Because instruction moves at a pace long-term effects.
with which children can keep up, they continue learning and gaining On the other hand, perhaps it is not surprising at all that a multi­
new competencies. faceted effort to conduct remediation combined with long-term reor­
Much of the gains come from reducing the number of children ientation of instruction would produce such large gains. Here, it is worth
scoring between 100 and 200 on the PISA-like scale, and greatly keeping in mind that the large, modelled impacts have incorporated a
increasing the percent scoring above 400 (Fig. 5). major shock that, if left unmitigated, would have reduced long term

Table 2
Summary of modelled outcomes of Covid-19 learning loss shocks and mitigation scenarios.
Grade 10 average Equivalent years of learning Percent of in-school Percent of grade 10 cohort Percent of grade 10
cohort learning ahead/behind of children in grade 10 above PISA 400 (achieving cohort below PISA 200
(PISA-like scale) counterfactual in grade 10 learning zero minimum proficiency) (very low learners)

Counterfactual – no shock 213 N/A 43 % 7% 51 %

Shock reduces grade 3 learning by 175 − 1.04 72% 3% 65 %
1/3
Shock reduces grade 3 learning by 157 − 1.55 82% 2% 73 %
1/2
Shock reduces grade 3 learning by 193 − 0.55 61% 4% 58 %
1/3 + remediation
Shock reduces grade 3 learning by 259 1.27 16% 27 % 36 %
1/3 + remediation +
instruction reorientation

6
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

learning by a full school year. An effort that can so fully mitigate a major following reopening. The present crisis presents an opportunity for
shock to learning and far surpass the counterfactual learning if there had systems to build back better.
been no shock deserves attention and consideration by education sys­
tems planning for reopening. Acknowledgements

3.3. Limitations to the modelled learning loss This work was funded by the RISE Progamme, which is supported by
the United Kingdom’s Foreign, Commonwealth, and Development Of­
The modelling of learning losses in the above scenarios has limita­ fice; the Australian Government’s Department of Foreign Affairs and
tions. The simulations assume no additional dropout as a result of school Trade; and the Bill & Melinda Gates Foundation. I am grateful to Lant
closures, rather assuming that enrolment and completion rates maintain Pritchett and Luis Crouch for comments on an earlier draft of this article.
pre-shock levels once schools reopen. This suggests that the modelled
learning loss may be optimistic, and actual learning losses could be even References
worse if many children do not return to school. After being out of school
for an extended time, some (or many) children may not return (UNESCO Al-Samarrai, S., 2020. The Impact of the Covid-19 Pandemic on Education Financing.
World Bank. http://pubdocs.worldbank.org/en/734541589314089887/Covid-a
et al., 2020). During their time out of school, some children may be put nd-Ed-Finance-final.pdf.
to work to help support their household and have to remain in Andrabi, T., Daniels, B., Das, J., 2020. Human Capital Accumulation and Disasters:
employment once schools reopen. Reductions in children returning to Evidence From the Pakistan Earthquake of 2005. RISE Working Paper Series, 20/
039. https://doi.org/10.35489/BSG-RISE-WP_2020/039.
school would further reduce learning outcomes (increasing learning Atuhurra, J., Alinda, V., 2017. Basic Education Curriculum Effectiveness Analysis in East
loss) due to the shock. Africa: Using the ‘Surveys of Enacted Curriculum’ Framework to Describe Primary
This model also does not build in macro shocks such as reductions in Mathematics and English Content in Uganda. MPRA. Paper No. 79017.
Beatty, A., Pritchett, L., 2015. Slow down, you’re going too fast: matching curricula to
education spending or losses of parental income. The World Bank fore­ student skill levels. Int. J. Educ. Dev. 40, 276–288.
casts that education budgets in 2020 could fall by as much as 4.2 percent Beatty, A., et al., 2018. Indonesia Got Schooled: 15 Years of Rising Enrolment and Flat
in Sub-Saharan Africa and 6.4 percent in South Asia, though there is Learning Profiles. RISE Working Paper Series, 18/026. https://doi.org/10.35489/
BSG-RISE-WP_2018/026.
much uncertainty in making such forecasts at this stage in the crisis Carvalho, S., Hares, S., 2020. More from our database on school closures: New education
(Al-Samarrai, 2020). The same World Bank report states that in policies may be increasing educational inequality. Center for Global Development
low-income countries households contribute, on average, 29 percent of Blog. https://www.cgdev.org/blog/more-our-database-school-closures-new-educ
ation-policies-may-be-increasing-educational.
education funding, and households are being hit hard economically in
Cummiskey, C., Stern, J., 2020. Calculating the Educational Impact of COVID-19 (Part
the crisis. Reduced income will reduce households’ abilities to invest in II): Using Data From Successive Grades to Estimate Learning Loss. RTI Shared Blog.
education. Remittances are also expected to drop significantly, and ed­ https://shared.rti.org/content/calculating-educational-impact-covid-19-part-ii-usi
ucation is often among the top uses of remittances by receiving house­ ng-data-successive-grades-estimate.
de Ree, J., et al., 2018. Double for nothing? Experimental evidence on an unconditional
holds. While education spending is poorly correlated with learning teacher salary increase in Indonesia. Q. J. Econ. 133 (May 2), 993–1039. https://doi.
outcomes (World Bank, 2018; Beatty et al., 2018; de Ree et al., 2018), org/10.1093/qje/qjx040, 2018.
large reductions, if maintained in the long run, could have detrimental Freudenberger, E., Davis, J., 2017. Tusome External Evaluation Midline Report.
Management Systems International for USAID. https://pdf.usaid.gov/pdf_docs/PA00
effects on outcomes. Thus, the scenarios simulated in this paper may be MS6J.pdf.
even more optimistic if economic constraints further reduce schooling Glewwe, P., Kremer, M., Moulin, S., 2009. Many children left behind? Textbooks and test
attainment and quality of instruction for those in school. scores in Kenya. Am. Econ. J. Appl. Econ. 1 (1), 112–135. https://www.sciencedirec
t.com/science/article/pii/S0738059314001217.
Finally, the model does not currently allow differentiation across Kaffenberger, M., Pritchett, L., 2020. Failing to Plan? Estimating the Impact of Achieving
countries. It is calibrated to the average learning across the countries Schooling Goals on Cohort Learning. RISE Working Paper. SERIEs. https://doi.org/
participating in the PISA-D assessments. Future work aims to develop 10.35489/BSG-RISE-WP_2020/038, 20/038.
Kuhfeld, M., Tarasawa, B., 2020. The COVID-19 Slide: What Summer Learning Loss Can
country-specific calibrations, but doing so is beyond the scope of the Tell Us About the Potential Impact of School Closures on Student Academic
current paper. Achievement. NWEA White Paper.
Kuhfeld, M., et al., 2020. Projecting the Potential Impacts of COVID-19 School Closures
on Academic Achievement. EdWorkingPaper. Available:, pp. 20–226 https://
4. Conclusion
edworkingpapers.com/sites/default/files/ai20-226-v2.pdf.
Mbiti, I. and Rodriguez-Segura, D. (forthcoming), Evaluating Curriculum Reforms in
The Covid-19 pandemic, which began as a health crisis, has also had Developing Countries: Evidence from Tanzania. RISE Working Paper Series.
Muralidharan, K., Singh, A., Ganimian, A., 2019. Disrupting education? Experimental
tragic economic and educational consequences. The model presented in
evidence on technology-aided instruction in India. Am. Econ. Rev. 109 (4),
this paper suggests that the long-term repercussions for children’s 1426–1460. https://doi.org/10.1257/aer.20171112.
learning could be devastating, with today’s grade 3 students losing as Piper, B., et al., 2018. Scaling up successfully: lessons from Kenya’s tusome national
much as 1.5 years’ worth of learning (or more) by the time they reach literacy program. J. Educ. Chang. 19 (2018), 293–321. https://doi.org/10.1007/
s10833-018-9325-4.
grade 10 as a consequence of their time out of school. Governments can, Slade, T.S., et al., 2017. Is ‘Summer’ reading loss universal? Using ongoing literacy
however, introduce measures that mitigate some or all of these conse­ assessment in Malawi to estimate the loss from GraDe-transition breaks. Res. Comp.
quences. The model suggests that effective remediation efforts imme­ Int. Educ. 12 (4), 461–485. https://doi.org/10.1177/1745499917740657.
Teaching at the Right Level (TaRL), 2018. Evidence. https://www.teachingattheright
diately upon return to school could reduce long-term learning loss for level.org/evidence/.
the cohort of grade 3 students by half. Beyond immediate efforts, there is UNESCO, 2018. Metadata for the Global and Thematic Indicators for the Follow-up and
an opportunity for systems to use the skills gained from implementing Review of SDG 4 and Education 2030. Available: http://tcg.uis.unesco.org/wp-conte
nt/uploads/sites/4/2019/04/sdg4-metadata-global-thematic-indicators-en.pdf.
large-scale remediation programmes to reorient instruction to better UNESCO, UNICEF, The World Bank, The World Food Programme, UNHCR, 2020.
match children’s skill levels in the long run. Such efforts, the model Framework for Reopening Schools. Accessed: https://unesdoc.unesco.org/ark:
suggests, could not only fully mitigate the consequences of the shock but /48223/pf0000373348.
Wilichowski, A., et al., 2020. Building Back Better: Accelerating Learning When Schools
also surpass learning outcomes compared to the counterfactual of no
Reopen, and What Kenya’s Tusome Program Can Teach Us. World Bank Blogs. http
shock. s://blogs.worldbank.org/education/building-back-better-accelerating-learning-whe
All of these mitigation efforts require planning. As systems continue n-schools-reopen-and-what-kenyas-tusome.
their remote learning programmes, they will also need to begin planning
for reopening, putting in place the tools for remedial programmes and, if
feasible, beginning to train teachers remotely. As they do so, they should
consider how they can build programmes and train teachers in ways that
can continue to produce benefits beyond the period immediately

7
M. Kaffenberger International Journal of Educational Development 81 (2021) 102326

World Bank, 2018. World Development Report 2018: Learning to Realize Education’s World Bank Live, 2020. The Covid-19 Pandemic: Shocks to Education and Policy
Promise. World Bank, Washington, DC. https://doi.org/10.1596/978-1-4648-1096- Responses. Accessed: https://live.worldbank.org/covid-19-shocks-ed
1. ucation-policy-responses#discussion.
World Bank, 2020. The COVID-19 Pandemic: Shocks to Education and Policy Responses.
May 2020 Available: https://openknowledge.worldbank.org/bitstream/handle/109
86/33696/148198.pdf.