COMMENTARY

COMMENTARY
Inequality in learning is a major concern after
school closures
Herman G. van de Werfhorsta,b,1

The COVID-19 pandemic has come with many non- Possible explanations for a learning delay gap are
pharmaceutical interventions to curb the spread of the related to parental involvement with education (5), so-
virus. One of the most significant measures has been cioeconomic differences in information and communi-
to close schools for in-school education, at the pri- cations technology (ICT) access and skills among
mary, secondary, and tertiary levels of schooling. students and the schools they attend (6, 7), and par-
While such interventions have been demonstrated ents’ ability to help with homework during the school
to be effective in reducing the spread of the virus closures (8). More studies are needed to understand
(1), they come at a cost: an intermittence of the learn- these mechanisms. Following the “digital divide” lit-
ing process that is not easily repaired with online in- erature, providing students with technological devices
struction. The well-designed study by Engzell et al. alone is unlikely to solve the problem, as inequality in
(2) shows, convincingly, that the “learning loss” due digital skills and usage are additional sources of the
to school closures is severe (although learning delay digital gap between socioeconomic groups (9).
may be a more appropriate term). Using data from In interpreting effect sizes, Engzell et al. (2) esti-
primary schools in The Netherlands and analyzing mate that the average learning progress has been
test scores on externally standardized tests, the study close to zero during the closures. The overall learning
reports a learning loss of 3.16 percentiles on a com- delay of 3.16 percentiles equals 0.08 SDs. Comparing
posite index of math, reading, and spelling, an effect this to the estimated SD growth from American stud-
that varies in size by socioeconomic background and ies and the World Bank of 0.40 SD learning growth
school composition. per year under normal circumstances, the study con-
At least as concerning as the overall learning delay cludes that the loss equals 8 wk of lost progress, sim-
is the reported inequality by socioeconomic back- ilar to the 8-wk school closure (close to one-fifth of a
ground and school composition. Using school records school year). If, instead of using American or global
of parental educational attainment (classifications used estimates on school progress under normal circum-
to determine weighted student funding for schools with stances, we use Dutch estimates on the spelling test
children from disadvantaged backgrounds), the study that is analyzed [available for spelling growth between
shows that children of very low-educated parents (i.e., Dutch grades 6 and 7, roughly from age 10 y to 11 y
none of the parents have more than lower-secondary (10)], we can further benchmark the estimated 3.03
education; in total, 8% of the families) suffer more from percentile learning loss in spelling for this grade, in
school closure than children from more-educated back- SI Appendix, table S11 of ref. 2. The national SD
grounds. Using a more fine-grained measure of school- growth equals 0.64 SD growth on the midyear test
level disadvantage, supplementary analyses show that and 0.45 SD at the end-of-year tests between grades 6
the learning delay is much stronger in schools with a and 7, averaging 0.55 SD (calculations based on table
higher share of disadvantaged children (which could be 2.1 of ref. 10). Assuming 3.03 percentile loss equals
simply compositional, as the school-level indicators (3.03/3.16*0.08 =) 0.077 SD, the learning delay would
were not available at the individual student level). Other then be roughly 0.077/0.55 = 0.14 of a school year,
European studies on the impact of COVID-19−related equaling 5.6 wk of learning delay over the 8-wk clo-
school closures show similar patterns (3, 4), with stron- sure (assuming a 40-wk school year). Following these
ger delays among children with disadvantaged back- rough estimates, the closure weeks then worked at a
grounds or schools with higher concentrations of 30% efficiency. The study by Engzell et al. (2) goes a
disadvantage. long way in interpreting the effect sizes, and they too

a
Department of Sociology, University of Amsterdam, Amsterdam 1001 NA, The Netherlands; and bAmsterdam Centre for Inequality Studies
(AMCIS), University of Amsterdam, Amsterdam 1001 NA, The Netherlands
Downloaded at Philippines: PNAS Sponsored on September 27, 2021

Author contributions: H.G.v.d.W. wrote the paper.

Competing interest statement: H.G.v.d.W. is chair of the Programme Committee of the National Cohort Study on Education (Nationaal
Cohortonderzoek Onderwijs) that is mentioned in the paper.
This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY).
See companion article, “Learning loss due to school closures during the COVID-19 pandemic,” 10.1073/pnas.2022376118.
1
Email: [email protected].
Published April 21, 2021.

PNAS 2021 Vol. 118 No. 20 e2105243118 https://doi.org/10.1073/pnas.2105243118 | 1 of 3

 Fig. 1. Possible school closure effects for the educational career.

present some estimates that suggest there may be some, but Education (Nationaal Cohortonderzoek Onderwijs) (13), student
strongly reduced, efficiency of the online learning process that monitoring data can be connected to further school careers, and
replaced in-school instruction. student socioeconomic and migration background, in more detail
Two control variables have been inserted in the difference- than in ref. 2. Recent reports on the effects of the school closure
in-differences models of ref. 2: an overall trend over years and using these data show comparable results (14): a decline of be-
the number of days between the midyear and end-of-year tests. tween 14% (spelling) and 25% (reading) relative to prepandemic
Without these control variables, the negative impact of school years, and strong socioeconomic gaps in the decline also in more
closures is less strong. In defense of their inclusion of the trend fine-grained measures. Gaps by migration background (net of so-
parameter, the placebo analysis had more expected null effects cioeconomic background) are smaller but found for reading.
than without a trend control. The trend parameter is positive, Importantly, the rich Dutch data infrastructure also enables
suggesting that, across the years 2017−2020, school performance researchers to closely monitor student progression in the years to
growth has increased, while other student assessment data on come. To help students get back on track, schools also can be
students in Dutch grade 6 show a stable or declining level of lit- informed regularly about their students’ performance in compar-
eracy (11) and mathematics (12). This could be due to the differ- ison to schools that have a wide range of similar characteristics.
ence between levels and growth (i.e., lower growth at higher Moreover, researchers can study school careers beyond primary
levels). Unfortunately, the study does not show trend parameters and secondary education. This is relevant, as the effects of school
in the comparison years only (the placebo analysis is only reported closures may be different across the school career. From a socio-
graphically, and not in table form with all coefficients). logical perspective, primary education is mostly concentrated on
The control for number of days between tests requires some- the foundation of basic skills, secondary education is mostly con-
what more thought. Under normal circumstances, a larger number centrated on differentiation in learning trajectories, and tertiary
of days between tests comes with higher performance due to education is mostly focused on credentialization: acquiring qual-
prolonged learning, but, in the treatment (pandemic) year, the ifications that provide access to the labor market (Fig. 1). Impacts
second test was taken later, while performance was lower. This of school closures on inequalities in learning (the dotted arrows)
means that the number of days between tests functions as a may be most prominent in primary education, because the foun-
suppressor variable, as the treatment year is positively associated dation is laid for future learning progression. Closures can further-
with number of days between tests, and negatively associated more impact socioeconomic inequalities in differentiation in
with student performance, while the relationship between number secondary education, especially when the transition from primary
of days and student performance is positive. As the leverage of to secondary education happens after periods of closure, or when
the (positive) slope originates from the comparison years, a linear standardized tests have been cancelled during school closures
control for number of days for all years together is likely not a (15). Swiss research showed that online instruction during school
functional form that would fit the data well, but the model does closures was equally as effective as in-school teaching at the
not allow differential effects by treatment status. The robustness secondary level, while primary education was more negatively
checks in ref. 2 report, indeed, that the negative effect of school affected by online instruction (16). Inequalities in tertiary educa-
closures is reduced by 12% if the control variable is omitted. tion may also be less strongly affected by closures, as the creden-
The study of Engzell et al. (2) demonstrates how much can be tialization process continues. A study on a 1968 French protest,
learned about student progression during the pandemic if the which led students to be given their upper-secondary bacca-
Downloaded at Philippines: PNAS Sponsored on September 27, 2021

data infrastructure is up to the task. It uses well-known externally lauréat diploma easily, showed that these students have not suf-
validated and standardized tests, made available to the researchers fered from it. More of them graduated from university, and they
through an alliance with a provider of analytics services to schools. received higher earnings compared to a counterfactual when the
Recently, other data on the same and other student assessments diploma had been as selectively awarded as before (17).
have become available for researchers. Building upon new secure Governments and schools face a heavy task: How can the delay
register-based school career data of the National Cohort Study on in learning be remedied? Some governments are willing to spend

2 of 3 | PNAS van de Werfhorst

https://doi.org/10.1073/pnas.2105243118 Inequality in learning is a major concern after school closures
 substantial funds to this challenge. The Dutch government has, for that such interventions are focused and intensive as these are
instance, allocated EUR 8.5 billion over 2.5 y, or 3.4 billion per known conditions for success. Countries across the globe can ben-
year, which equals around 8% of the yearly budget for the Ministry efit from the lessons learned from such interventions, provided that
of Education, Culture and Sciences. A high-quality data infrastruc- a solid research infrastructure exists to monitor their effects.
ture can help the field to understand the effectiveness of promis-
ing (inequality-reducing) interventions such as offering after-school Acknowledgments
tutoring (18), enrolling students in summer programs (19), and H.G.v.d.W.’s research is supported by a Vici Grant by the Netherlands Organi-
employing additional teachers and teacher assistants (20), provided sation for Scientific Research, Grant 453-14-017.

1 N. Haug et al., Ranking the effectiveness of worldwide COVID-19 government interventions. Nat. Hum. Behav. 4, 1303–1312 (2020).
2 P. Engzell, A. Frey, M. D. Verhagen, Learning loss due to school closures during the COVID-19 pandemic. Proc. Natl. Acad. Sci. U.S.A. 118 e2022376118 (2021).
3 J. E. Maldonado, K. De Witte, “The effect of school closures on standardised student test outcomes” (Discussion Paper DPS20.17, KU Leuven Department of
Economics, 2020).
4 S. Rose et al., “Impact of school closures and subsequent support strategies on attainment and socio-emotional wellbeing in Key Stage 1” (Interim Pap. 1,
National Foundation for Educational Research, 2021).
5 M. Castro et al., Parental involvement on student academic achievement: A meta-analysis. Educ. Res. Rev. 14, 33–46 (2015).
6 U. Samuelsson, T. Olsson, “Digital inequality in primary and secondary education: Findings from a systematic literature review” in Media and Education in the
Digital Age: Concepts, Assessments, Subversions, M. Stocchetti, Ed. (Peter Lang, 2014), pp. 41–62.
7 H. van de Werfhorst, E. Kessenich, S. Geven, The digital divide in online education. Inequality in digital preparedness of students and schools before the start of
the COVID-19 pandemic. SocArXiv [Preprint] (2020). https://osf.io/preprints/socarxiv/58d6p/ (Accessed 30 March 2021).
8 T. Bol, Inequality in homeschooling during the Corona crisis in the Netherlands. First results from the LISS Panel. SocArXiv [Preprint] (2020). https://doi.org/
1031235/osf.io/hf32q (Accessed 30 March 2021).
9 A. Scheerder, A. van Deursen, J. van Dijk, Determinants of Internet skills, uses and outcomes. A systematic review of the second- and third-level digital divide.
Telemat. Inform. 34, 1607–1624 (2017).
10 R. Engelen, P. Roumans, M.-A. Keizer, I. Jongen, Wetenschappelijke verantwoording taalverzorging groep 6 tot en met 8 [Scientific Justification Language
Assessment Grades 6 to 8] (Cito, 2016).
11 J. Gubbels, A. Netten, L. Verhoeven, Vijftien jaar leesprestaties in Nederland [Fifteen Years of Reading Performance in the Netherlands] (Radboud University,
2017).
12 M. Meelissen, E. Hamhuis, L. Weijn, Leerlingprestaties in de exacte vakken in groep 6 van het basisonderwijs: Resultaten TIMSS-2019 [Student Performance in
Science and Mathematics in Grade 6 of Primary Education: Results TIMSS-2019] (Universiteit Twente, 2021).
13 C. Haelermans et al., Using data to advance educational research, policy, and practice: Design, content, and research potential of the Netherlands Cohort Study
on Education. Eur. Sociol. Rev. 36, 643–662 (2020).
14 C. Haelermans, M. Jacobs, L. Van Vugt, “Factsheets eerste schoolsluiting n.a.v. COVID-19 crisis. Technische toelichting [Factsheets first school closure in the
COVID-19 crisis. Technical supplement]” (Netherlands Initiative for Educational Research, 2021). https://www.nationaalcohortonderzoek.nl/factsheets. Accessed
30 March 2021.
15 S. Geven, Track recommendation procedures and the cancellation of the school leavers’ test during the COVID-19 pandemic in the Netherlands. SocArXiv
[Preprint] (2020). https://doi.org/1031235/osf.io/rdw3m (Accessed 30 March 2021).
16 M. J. Tomasik, L. A. Helbling, U. Moser, Educational gains of in-person vs. distance learning in primary and secondary schools: A natural experiment during the
COVID-19 pandemic school closures in Switzerland. Int. J. Psychol., 10.1002/ijop.12728 (2020).
17 E. Maurin, S. McNally, “Vive la révolution! Long term returns of 1968 to the angry students” (IZA Discuss. Pap. 1504, Institute of Labor Economics, 2005).
18 J. Dietrichson, M. Bøg, T. Filges, A.-M. Klint Jørgensen, Academic interventions for elementary and middle school students With low socioeconomic status: A
systematic review and meta-analysis. Rev. Educ. Res. 87, 243–282 (2017).
19 J. S. Kim, D. M. Quinn, The effects of summer reading on low-income children’s literacy achievement from kindergarten to grade 8: A meta-analysis of classroom
and home interventions. Rev. Educ. Res. 83, 386–431 (2013).
20 S. W. Hemelt, H. F. Ladd, C. R. Clifton, Do teacher assistants improve student outcomes? Evidence from school funding cutbacks in North Carolina. Educ. Eval.
Policy Anal., 10.3102/0162373721990361 (2021).
Downloaded at Philippines: PNAS Sponsored on September 27, 2021

van de Werfhorst PNAS | 3 of 3

Inequality in learning is a major concern after school closures https://doi.org/10.1073/pnas.2105243118