Abstract

This paper evaluates the effects of school closure during the Covid-19 pandemic on learning loss and learning inequalities in Rio de Janeiro. It presents longitudinal data for 671 children (5/6 years old) enrolled in the second year of preschool (compulsory Education in Brazil) in two cohorts (2019 and 2020). All children were assessed at the start and end of the school year, and value-added models were used to estimate the impact of school closure from April to December 2020 on language and mathematics development. Results suggest a learning loss of 0.23 and 0.25 standard deviations for language and mathematics. This equates to children having learned around 65% of what they would have in face-to-face interactions. Those from low socioeconomic status families were more affected; they learned only 48% of what they would have in normal conditions. Results suggest an increase in learning inequalities during the 2020 academic year. Implications for policy and the need for an Education recovery plan focused on disadvantaged children are discussed.

Learning Loss; Learning Inequality; Pandemic; Longitudinal Data

Resumo

O artigo estima os efeitos do fechamento das escolas para atividades presenciais durante a pandemia do Covid-19 no aprendizado e nas desigualdade de aprendizagem. Utiliza dados longitudinais de 671 crianças (5 e 6 anos), matriculadas no 2º ano da pré-escola em 2019 e 2020 na cidade do Rio de Janeiro. Todas as crianças foram avaliadas no início e ao final do ano letivo e modelos de valor agregado foram utilizados para estimar o impacto do fechamento das escolas. Os resultados sugerem impactos de 0,23 e 0,25 desvio padrão para Linguagem e Matemática. A coorte de 2020 aprendeu o equivalente a 65% tendo a coorte de 2019 como parametro e crianças de nível socioeconomico mais baixo foram mais fortemente impactadas, tendo aprendido apenas 48% do estimado para condições normais com aulas presenciais. Os resultados sugerem um aumento das desigualdades de aprendizagem durante a pandemia. Implicações para política educacional e ações para um plano nacional de recuperação do aprendizado são discutidos.

Perda de Aprendizado; Desigualdades de Aprendizagem; Pandemia; Desenvolvimento Infantil; Estudo Longitudinal

Resumen

Este artículo evalúa los efectos del cierre de escuelas durante la pandemia de Covid-19 sobre la pérdida de aprendizaje y las desigualdades de aprendizaje en Río de Janeiro. Presenta datos longitudinales para 671 niños (5/6 años) matriculados en el segundo año de preescolar (educación obligatoria en Brasil) en dos cohortes (2019 y 2020). Todos los niños fueron evaluados al comienzo y al final del año escolar y se utilizaron modelos de valor agregado para estimar el impacto del cierre de la escuela de abril a diciembre de 2020 en el desarrollo del lenguaje y las matemáticas. Los resultados sugieren una pérdida de aprendizaje de 0,23 y 0,25 desviaciones estándar para lenguaje y matemáticas. Esto equivale a que los niños hayan aprendido alrededor del 65% de lo que habrían aprendido en las interacciones cara a cara. Los de familias de nivel socioeconómico bajo se vieron más afectados; aprendieron solo el 48% de lo que tendrían en condiciones normales. Los resultados sugieren un aumento de las desigualdades de aprendizaje durante el año académico 2020. Se discuten las implicaciones para la política y la necesidad de un plan de recuperación de la educación centrado en los niños desfavorecidos.

Pérdida de Aprendizaje; Desigualdad de Aprendizaje; Pandemia; Pandemia; Datos longitudinales

1 Introduction

In response to the Covid-19 outbreak, governments from more than 190 countries suspended face-to-face instruction in schools, affecting nearly 94% of the world’s student population (UNESCO, 2020UNESCO. Global monitoring of school closures caused by COVID-19. New York, 2020. Available from: https://en.unesco.org/covid19/educationresponse . Access in: 2021 Oct. 8.
https://en.unesco.org/covid19/educationr... ). The length of school closures varied across different countries and led to concerns about consequences for student learning, an increase in learning inequality, and school dropouts (CAMPOS, 2020; GOMES et al ., 2021; WORLD BANK; UNESCO; UNICEF, 2021).

Since 2020, many publications have been produced by governments, multilateral organizations, research institutes, and Universities aiming to understand the impact of school closure on multiple school outcomes. Initially, the studies analyzed data from past events that also led to school closures (natural disasters, other pandemics, teacher strikes, or even the school break during the summer), trying to estimate its effect on students’ performance and learning inequality (BELOT; WEBBINK, 2010BELOT, M.; WEBBINK, D. Do teacher strikes harm educational attainment of students? Labour, [s. l.], v. 24, n, 4, p. 391-406, Dec. 2010. https://doi.org/10.1111/j.1467-9914.2010.00494.x
https://doi.org/10.1111/j.1467-9914.2010... ; DOWNEY; HIPPEL; BECKETT , 2004). Then, around the second semester of 2020, publications focused on cohorts affected by the Covid-19 pandemic. Most of the publications analyzed data from developed countries in Europe or the US, and very few presented data from developing countries, including the region of Latin America (STRINGER; KEYS, 2021; UK DEPARTMENT OF EDUCATION, 2021UK DEPARTMENT OF EDUCATION. Learning during the pandemic: quantifying lost learning: report 3 of 5 on learning during the 2020 coronavirus (COVID-19) pandemic. Coventry, 2021. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/998934/6803-3_Learning_during_the_pandemic-_quantifying_lost_learning.pdf . Access in: 2021/ Oct. 2.
https://assets.publishing.service.gov.uk... ; ENGZELL, 2021; MALDONADO; DE WITTE, 2020; WORLD BANK, 2021).

This paper reports the results of a longitudinal study using unique data from the city of Rio de Janeiro, Brazil. The research design compares the progress of two cohorts of pupils ages 5-6 who attended the second year of preschool (compulsory Education in Brazil) in 2019 and 2020. Value-added models were used to estimate the impact of school closure from April to December 2020 on language and mathematics development. It is the first study to estimate the impact of school closure on young children’s cognitive development in Latin American countries, so far as we can tell. The paper seeks to answer two main research questions. The first one asks how big an impact school closure had on pupils’ cognitive development (language and mathematics). The second asks how big an impact school closure had on learning inequality, focusing on disadvantaged children.

Previous publications have indicated that school closure due to Covid-19 produced learning loss across different age groups and the effects were more prominent for disadvantaged pupils, widening the attainment gap across different educational systems (EEF, 2020; ENGZELL, 2021; MALDONADO; DE WITTE, 2020; STRINGER; KEYS, 2021; UK DEPARTMENT OF EDUCATION, 2021UK DEPARTMENT OF EDUCATION. Learning during the pandemic: quantifying lost learning: report 3 of 5 on learning during the 2020 coronavirus (COVID-19) pandemic. Coventry, 2021. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/998934/6803-3_Learning_during_the_pandemic-_quantifying_lost_learning.pdf . Access in: 2021/ Oct. 2.
https://assets.publishing.service.gov.uk... ). Many countries have tried to mitigate learning losses by providing remote instructions using asynchronous or synchronous platforms. Different remote teaching strategies have been assessed on learning, especially in developed countries, with mixed results about their benefits (CARLANA; LA FERRARA, 2021; EEF, 2020; ENGZELL; FREY; VERHAGEN, 2021).

In Brazil, the implementation of these tools has varied substantially (BARBIERIA; CANTANELLI; SCHMALZ, 2021; CAMPOS; VIEIRA, 2021; FCC, 2020; OLIVEIRA et al ., 2021). Private schools or public schools in wealthier areas have implemented different programs to ensure students’ synchronous learning and online participation. Our data suggest that children enrolled in private schools have access to better online learning resources (including high-speed internet), a richer home learning environment, and more help from parents who could work from home during 2020. Taken together, these differences could exacerbate educational inequalities.

The paper is divided into six sections, including the introduction. Section 2 discusses previous research focused on the impact of school closure on children’s cognitive development at the start of compulsory Education. Section 3 presents the study design, sample, and key instruments used during data collection. Section 4 presents the study limitations. Section 5 presents the main findings on school closure effects on pupils’ cognitive development. Section 6 presents the findings for learning inequalities. Section 7 presents the discussions of the main findings. The last one, Section 8, presents preliminary conclusions and implications for future policy.

2 Impact of school closure on students’ learning: what do we know?

The disruption in Education caused by the Covid-19 pandemic is without parallel. Global simulations and more recent studies using actual measures collected during/after the crisis suggest a sizeable negative effect on children’s learning, a higher risk of school dropout, and an increase in learning inequalities, among other adverse effects on students’ development and well-being (UNESCO, 2021; WORLD BANK; UNESCO; UNICEF, 2021).

Globally schools remained closed for 224 days, but the figures varied greatly comparing different regions and countries (WORLD BANK, 2021). For example, in Brazil, most public schools remained closed for almost the entire academic year in 2020 and started to reopen slowly in 2021 (BARBIERIA; CANTANELLI; SCHMALZ, 2021; FCC, 2021). On the other hand, private schools started to reopen earlier (around September/October of 2020), possibly exacerbating inequalities between public and private schools (CASTRO, 2021CASTRO, D. L. Público ou privada: o impacto da rede escolar na aprendizagem e desigualdades escolar. Dissertation (Master in Education) – Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2021.).

Children from disadvantaged households had fewer opportunities to learn during school closures. Remote teaching was implemented in most countries, including mid and low-income. However, the quality and support to transition to remote learning were limited. Data collected during the pandemic shows that children from low socioeconomic status were less likely to engage and benefit from remote learning. The main reasons are the lack of connectivity, devices, a room to study, and support from parents or caregivers. It is a challenging scenario that will likely increase the learning gap and produce a mid to long-term impact on Education (BARNETT; JUNG, 2020BARNETT, S. W.; JUNG, K. The pandemic and preschool education in five charts, National Education Ppçicy Center Newsletter, 2020 Sep. 10.; CAMPOS; VIEIRA, 2021; FCC, 2021; BARNETT; JUNG, 2020BARNETT, S. W.; JUNG, K. The pandemic and preschool education in five charts, National Education Ppçicy Center Newsletter, 2020 Sep. 10.; PASCAL et al ., 2020).

Researchers around the world are trying to answer three main questions:

1. The impact of school closure on learning

2. The impact of school closure on learning inequalities

3. The impact of remote learning during school closure

An increasing number of studies using actual data from the Covid-19 cohorts can provide an initial diagnosis of the current crises (EEF, 2020; ENGZELL et al ., 2021; MALDONADO; DE WITTE, 2020; UK DEPARTMENT OF EDUCATION, 2021UK DEPARTMENT OF EDUCATION. Learning during the pandemic: quantifying lost learning: report 3 of 5 on learning during the 2020 coronavirus (COVID-19) pandemic. Coventry, 2021. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/998934/6803-3_Learning_during_the_pandemic-_quantifying_lost_learning.pdf . Access in: 2021/ Oct. 2.
https://assets.publishing.service.gov.uk... ; WORLD BANK, 2021). For example, Stringer and Keys (2021) produced a systematic review commissioned by the UK government. The summary of international evidence suggests that students affected by school closures in the second quarter of 2020 lost 2 to 3 months compared with the academic milestones their cohorts would be expected to reach. The researchers also highlight those losses were bigger in mathematics than in reading, and young students tend to be more adversely affected than older peers.

Most of the robust evidence produced so far used data collected during the pandemic in high-income countries in Europe and the United States. One study used data collected in Mexico for students between 10 and 15 years old suggests a bigger negative impact according to family SES in a range of 0.34-0.45 standards deviation in reading and 0.62-0.82 in mathematics (HEVIA et al ., 2022). In South America, the Secretary of Education from São Paulo State commissioned a study (not peer-reviewed) and made it public by the end of 2020. The analysis presented an interrupted time-series design and estimated the impact of school closure on children at ages 11-12 and 14-15. Results suggest that, on average, students learned only 28% of what they would have in normal conditions with face-to-face interactions. They have also estimated a greater risk of dropout – more than threefold (Seduc-SP, 2021). The lack of studies in South America and other developing countries is a genuine concern, especially for young pupils. This study aims to provide robust evidence about the impact of school closure on children starting compulsory Education in Rio de Janeiro. An independent study (peer-reviewed) using the same data from the State of São Paulo and differences-in-differences strategy estimated a loss of 0.32 standard deviations in test scores for students in secondary Education. The study indicates that, during the pandemic, students have learned 27.5% of the in-person equivalent (LICHAND et al ., 2022).

The impact of remote learning was also assessed in different studies, and overall results suggest that children learned less when compared to face-to-face instructions (EEF, 2020; ENGZELL; FREY; VERHAGEN, 2021). Nonetheless, different strategies have been tried and assessed during the crises and some initiatives, such as online tutoring after school, seem to produce positive results (CARLANA; LA FERRARA, 2021; HASSAN et al ., 2021). Those are exciting findings that could be adopted as part of recovery plans that many countries have designed.

Most evidence does not consider young children starting compulsory Education and or preschool. The few studies that focus on this age group did not measure children’s learning during the pandemic and, instead, collect data from parents or teachers (CAMPOS; VIEIRA, 2021). This is a gap in the literature, and it is essential to understand its impact, especially considering the growing evidence suggesting that young students benefit less from remote teaching (STRINGER; KEYS, 2021). This is a big concern considering that evidence from the iPIPS project has shown the importance of the first year at school for long-term outcomes and the fact that this is the period when children will learn the most during their entire academic path (BARTHOLO et al ., 2020). Producing reliable data about the impact of school closure is key to designing and implementing recovery plans to reduce student dropout, learning loss, and learning inequalities over time.

3 Research design, sample and instruments

The study uses longitudinal data collected in two cohorts (2019 and 2020) involving 671 children enrolled in the same 21 schools in Rio de Janeiro. All children were assessed using an adaptation to the PIPS (Performance Indicator for Primary Schools) at the start and end of the school year (BARTHOLO et al ., 2020). The 2019 cohort involved 460 children who had the opportunity to attend face-to-face activities at school during the entire year. The 2020 cohort involved 211 children who attended only a few weeks of in-person activities in school during 2020.

The research presents a quasi-experimental design with pre and post-test data with treatment (cohort 2020) and control (cohort 2019) groups. It uses longitudinal data for language and mathematics development. It compares progress in the two cohorts – both attending the second year of preschool – as part of compulsory Education in Brazil since 2009. ¹ 1 In 2009, Constitutional Amendment n. 59 extended compulsory education for children ages 4 and 5 (preschool). In the city of Rio de Janeiro, schools closed, stopping face-to-face instructions in the third week of March 2020, and started slowly to reopen during October and November of 2020. On average, private schools reopened earlier than subsidized non-for-profit private and public schools in Rio de Janeiro.

Before the present research, thirty-six schools were invited in 2019 to be part of a longitudinal study, and they were invited to participate in the new Covid-19 impact study. Unfortunately, some schools declined, claiming safety protocols due to the pandemic. Others stopped offering preschool or closed due to children’s massive dropouts in 2020 (more frequent among low-fee for-profit private schools). Thus, the present study presents a non-probabilistic sample of 21 private (for-profit) and subsidized not-for-profit private schools in Rio de Janeiro. All children enrolled in the second year of preschool in those 21 schools were invited to participate in the study. Parents and children’s consent was obtained before data collection.

Hierarchical linear models were used to estimate the impact of school closure on children’s learning by comparing the progress of the two cohorts. The value-added models produce robust evidence about the effect of school closure.

The sample presented a diverse profile of pupils and school characteristics, including a mixture of private (for-profit) and subsidized not-for-profit private schools (CASTRO, 2021CASTRO, D. L. Público ou privada: o impacto da rede escolar na aprendizagem e desigualdades escolar. Dissertation (Master in Education) – Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2021.). This is an important characteristic of the study sample because it allows a better understanding of the potential impact of school closure on learning inequalities. In 2019, in the city of Rio de Janeiro, about 61.7% of enrollment for children ages 4-5 (preschool) were in public schools and 38.3% in private schools (for-profit and not-for-profit). Table 1 presents additional information about the sample and instruments used in data collection.

The cognitive test (PIPS adaptation do the Brazilian context) measures two dimensions, language and mathematics. It is a one-to-one assessment conducted by a trained researcher for an average of 10 to 20 minutes. The researcher asks the questions and records the answers on a tablet. A booklet is used in the assessment and presents images, figures, letters, words, and numbers to the children. The software on the tablet is adjusted such that wrong and correct answers are used to decide whether to stop a section or continue with the next question. Each test session presents items of increasing difficulty, which enables testing with a desirable minimum duration (BARTHOLO et al ., 2020).

The test presents the following subdimensions: a) Handwriting – the child is asked to write his/her name; b) Vocabulary – identify objects embedded within a series of images; c) Ideas about reading – assessment of concepts of recognition of the written language; d) Phonological awareness – rhymes and repetitions; e) Letters Identification – a fixed order of letters; f) Word recognition and reading – words, sentences, and comprehension; g) Ideas about mathematics– assessment of understanding of mathematical concepts (for example, size and volume); h) Counting and ability to use numbers; i) Sums – addition and subtraction problems presented without symbols; j) Shape identification; k) Digit identification; l) Mathematical problems – including sums with symbols (BARTHOLO et al ., 2020).

Parental questionnaires provided additional information about family and children’s characteristics. Data were collected using three complementary strategies: a) an online questionnaire using a secure link; b) a short telephone interview; c) paper and pencil. Table 2 summarizes all the variables included in the models, considering both cohorts.

3.1 Study Limitations

There are three main threats to the external and internal validity of the study. First, the sample is non-probabilist and is not representative of Rio de Janeiro city. The fact that some schools in 2020 declined to continue in the study or closed imposed a reduction on the number of schools participating in the study.

Attrition in the longitudinal study is also a concern. There are two sources of attrition: a) pupil assessment – parents that did not provide written consent or children did not complete the cognitive assessment in a particular wave (total of 9.5%) and b) parents’ questionnaire: parents that did not complete online or paper and pencil questionnaire. The final sample with 671 children excluded children with only one measure (start or end of the year). Missing data for parents’ questionnaires were handled with imputation at the child level using R (CASTRO, 2021CASTRO, D. L. Público ou privada: o impacto da rede escolar na aprendizagem e desigualdades escolar. Dissertation (Master in Education) – Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2021.). Data in Table 2 suggests that both cohorts present a similar starting point (wave 1), which is relevant for future comparisons between the two groups.

The time when data was collected is relevant to estimating the impact of school closure on children’s cognitive development. Ideally, the time of data collection for both cohorts (2019 and 2020) should be very similar. However, the time varied across schools and cohorts. To compensate for this, children’s attainment was adjusted to estimate scores for 1st April (start of the year) and 1st December (end of the year). This controls for maturation and school effects. The adjustment considered the individual gain between waves one and two ⁴ 4 The number of days between the first and second wave for the 2020 cohort (covid) was bigger when compared to the 2019 cohort. This is a threat to internal validity (maturation effect) since the comparison of both groups should consider a similar number of days between the first and second wave. Therefore, Rasch measures were adjusted to 1st April and 1st December to estimate the results for a similar number of days between data collection for both cohorts. .

4 Impact of school closure on learning

The first analysis presents evidence for the impact of school closure on children’s development in language and mathematics. The central hypothesis is that students who experienced the second year of preschool in 2020 would learn less than the 2019 cohort, who had the opportunity to attend face-to-face instructions throughout the entire school year. Figures 1 and 2 present the Rasch scores (BOONE, 2016BOONE, W. J. Rasch analysis for instrument development: Why, when, and how? CBE Life Science Education, Bethesda, v. 15, n. 4, p. 1-7, Winter 2016. https://doi.org/10.1187/cbe.16-04-0148
https://doi.org/10.1187/cbe.16-04-0148... ) for language and mathematics development for both cohorts at the start and end of the year.

Figure 1
Mathematics development in the second year of preschool – cohorts 2019 and 2020

Figure 2
Language development in the second year of preschool – cohorts 2019 and 2020

The charts indicate a similar starting point for both cohorts at the beginning of the second year of preschool for math and language measures. In addition, the data indicate that children from both cohorts developed throughout the school year. However, the 2019 cohort shows larger gains than the 2020 cohort, suggesting a potential effect of the pandemic on children’s cognitive development, both in language and mathematics.

Table 3 presents the coefficients of multilevel models estimating children’s learning during the second year of preschool for language and mathematics. The initial measure of the child’s proficiency is the main predictor of the model. However, there are also other explanatory variables in the model – child’s sex and age, family’s socioeconomic status (Family SES), home learning environment (HLE), daycare attendance, average school socioeconomic status (School SES), and a variable that identifies the 2019 and 2020 cohorts.

The coefficient that identifies the 2020 cohort (Covid-19) in both models is statistically significant and negatively associated with learning. The results indicate that children in the 2020 cohort, who experienced the Covid-19 pandemic in their second year of preschool, learned at a slower pace compared to other children who attended the same schools in 2019.

The observed effects can be transformed into standard deviation units, effect sizes, or even months of learning. This standardization of results is relevant because it allows us to compare the results presented in this report with other studies carried out in Brazil or other countries. Table 4 presents the estimated coefficients transformed to standard deviation units (Cohen’s D), effect sizes using the calculation proposed by Tymms (2004)TYMMS, P. Effect sizes in multilevel models. In: SCHAGEN, I.; ELLIOT, K.(org.) But what does it mean? The use of effect sizes in educational research. Slough, UK: National Foundation for Educational Research, 2004. p. 1-10. for multi-level models, and months of learning considering the proposal of Higgins et al . (2013).

The estimated coefficients for the impact of the pandemic on learning suggest an average negative effect of -0.23 for language and -0.25 for mathematics using Cohen’s D (Standard Deviation). This would be equivalent to 3 to 4 months of learning loss (HIGGINS et al ., 2013). Using TYMMS’s (2004)TYMMS, P. Effect sizes in multilevel models. In: SCHAGEN, I.; ELLIOT, K.(org.) But what does it mean? The use of effect sizes in educational research. Slough, UK: National Foundation for Educational Research, 2004. p. 1-10. calculation, the effect size suggests -0.28 and -0.30 for language and mathematics or four months of learning loss. The magnitude of the effects estimated with Brazilian data is slightly bigger than the reported in some international studies – in general, losses of approximately 2 to 3 months (STRINGER; KEYS, 2021; UK DEPARTMENT OF EDUCATION, 2021UK DEPARTMENT OF EDUCATION. Learning during the pandemic: quantifying lost learning: report 3 of 5 on learning during the 2020 coronavirus (COVID-19) pandemic. Coventry, 2021. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/998934/6803-3_Learning_during_the_pandemic-_quantifying_lost_learning.pdf . Access in: 2021/ Oct. 2.
https://assets.publishing.service.gov.uk... ).

Another way to think about the effects of the pandemic is to describe learning in 2020 as a percentage of a regular school year (2019). Using Cohen’s D (Standard Deviation), the values estimated in the study suggest that children who experienced the second year of preschool in 2020 learned 66% in language and 64% in mathematics compared to the children’s learning in 2019.

It is relevant for teachers, headteachers, school principals, and parents to understand what children failed to learn during the 2020 school year. Two examples can help exemplify the challenge of diagnosis and further planning for teachers. First, comparing the two cohorts in mathematics development, it is possible to observe that by the end of the school year in 2019, approximately 60% of children were able to: a) identify two-digit numbers, b) do informal maths (add and subtract small amounts supported by pictures) and, c) simple formal sums such as 4 + 1 =. However, in the 2020 group, only 50% of children got these items right.

In language development, in 2019, approximately 60% of children were able to identify 18 letters that were presented in a booklet. In the 2020 cohort, only 45% of children could identify the same set of letters. In terms of vocabulary, in 2019, 56% of children answered 20 items correctly (which would be equivalent to 90% of the test). Only 43% of children in 2020 achieved the same result. In the group of children who had the opportunity to attend regular classes in preschool, the majority (approximately 77%) at the end of preschool were able to identify where writing began in a book and distinguish between text and image. In the 2020 group, 60% of children finished preschool with the same skills.

5 Impact of school closure on learning inequalities

The second research question addresses the impact of school closure during the pandemic on learning inequality, focusing on disadvantaged children. Since 1990, access to early childhood Education and primary and secondary Education has increased in Brazil. In addition, an increase in GDP spending was also observed in the same period, along with a modest improvement in national standardized tests (Prova Brasil). These are significant achievements; nonetheless, research before the pandemic highlighted that the learning inequality gap in elementary schools had increased when we considered the results of students of different SES and ethnic/race backgrounds (ALVES, 2020ALVES, M. T. G. Caracterização das desigualdades educacionais com dados públicos: desafios para conceituação e operacionalização empírica. Lua Nova, São Paulo, v. 110, p. 1-28, maio-ago. 2020. https://doi.org/10.1590/0102-189214/110
https://doi.org/10.1590/0102-189214/110... ; ALVES; SOARES; XAVIER, 2016; SOARES; DELGADO, 2016SOARES, J. F.; DELGADO, V. M. S. Medida das desigualdades de aprendizado entre estudantes de ensino fundamental. Estudos em Avaliação Educacional, v. 27, n. 66, p. 754-780, set.-dez. 2016. https://doi.org/10.18222/eae.v27i66.4101
https://doi.org/10.18222/eae.v27i66.4101... ).

The hypothesis is that school closure will affect more children from families with lower socioeconomic status, widening the cognitive development gap. This hypothesis was built on the premise that the conditions that families and children had throughout the implementation of remote Education and access to additional support from schools and other public services varied greatly depending on the families’ socioeconomic status. It should also be mentioned that national programs coordinated by the Ministry of Education were absent to ensure that disadvantaged children had access to remote learning (FCC, 2021). Also, data collected with the families suggests an association between family SES and the home learning environment index. Moreover, previous research in Rio de Janeiro collecting data in 2017, 2018, and 2019 indicated that the home learning environment is a good predictor of children’s learning (BARTHOLO et al., 2020; CASTRO, 2021CASTRO, D. L. Público ou privada: o impacto da rede escolar na aprendizagem e desigualdades escolar. Dissertation (Master in Education) – Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2021.), even when controlled by family socioeconomic status. The lack of face-to-face instructions, with meaningful interactions between children and their peers and teachers, could exacerbate the impact of the home learning environment on learning inequalities.

Figure 3
Percentage of families that always or frequently carry out activities, according to socioeconomic status

Table 5 presents coefficients for multilevel models estimating children’s learning during the second year of preschool for language and mathematics. Both models are very similar to the ones presented in Table 3 , except for one additional variable – interaction between family SES and the dummy variable that identifies the 2020 cohort (Covid-19). The analysis estimates whether children with different socioeconomic status indicators and who experienced the pandemic during the second year of preschool had higher or lower mean gains in learning.

The results suggest an increase in learning inequalities for language with an effect size of 0.17, which is calculated by comparing children a standard deviation below the mean with children a standard deviation above the mean for the SES indicator. These are big differences of up to 2/3 months of learning. For mathematics, the patterns are similar; however, the coefficient, equivalent to an effect size of 0.09 (1/2 months of learning), is not statistically significant, although it is our best estimate of the impact of Covid-19 on learning inequalities. The values estimated in the study suggest that disadvantaged children (low SES families) learned around 48% of what they would have in face-to-face interactions. Children from high SES families learned more - around 75% of their normal conditions.

6 Discussion

The study presents robust evidence about the impact of school closure on children’s cognitive development as they associate learning throughout the school year (based on longitudinal data for all children) with school closure during the pandemic. This is an essential feature of this study and deserves to be highlighted when compared to other studies that seek to estimate the impact of the pandemic on learning. Furthermore, associating the pandemic with children’s learning (and not simply a measure of student proficiency) improves the quality of causal inference.

The magnitude of the effects estimated with Brazilian data is slightly larger than the reported in some international studies using data from developed countries – general losses of 2 to 3 months (STRINGER; KEYS, 2021; UK DEPARTMENT OF EDUCATION, 2021UK DEPARTMENT OF EDUCATION. Learning during the pandemic: quantifying lost learning: report 3 of 5 on learning during the 2020 coronavirus (COVID-19) pandemic. Coventry, 2021. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/998934/6803-3_Learning_during_the_pandemic-_quantifying_lost_learning.pdf . Access in: 2021/ Oct. 2.
https://assets.publishing.service.gov.uk... ). The results can be partially explained by the conditions available for schools to implement remote activities and families to support pupils in remote activities and the total number of weeks without face-to-face activities in schools. Nonetheless, the impact reported in Mexico (HEVIA et al ., 2021) or the public schools in the State of São Paulo, Brazil (SEDUC-SP, 2021), was bigger than the ones described in the paper. A critical feature of this study is that it has collected data from private schools, including middle and upper-middle-class families in Rio de Janeiro. The study commissioned by the State Secretary of Education in São Paulo Public Schools suggests that students learned only 28% of what they would have in face-to-face interactions as opposed to 65% in non-for-profit private and for-profit private schools in Rio de Janeiro. These significant differences should be considered additional evidence of an increase in learning inequalities, especially when considering the stratification between public and private schools in Brazil.

The results also corroborate international findings suggesting a slightly bigger impact in mathematics and increased learning inequalities, especially for language. There are a few hypotheses that can help to explain an increase in learning inequalities during 2020. First, the association between the home learning environment index and family SES is important. Previous research before the pandemic suggests that the home learning environment is a good predictor of children’s learning even after controlling for prior attainment and family SES (BARTHOLO et al ., 2020). The disruption of Covid-19 in Education is likely to increase the impact of the home learning environment on children’s cognitive development. There is also the fact that disadvantaged children had less opportunity to interact with their teachers and peers during remote learning. Evidence produced in Brazil suggests a lack of national guidelines for remote learning and a shortage of resources to enable public and not-for-profit private schools to provide good quality remote learning (BARBIERIA; CANTANELLI; SCHMALZ, 2021; FCC, 2021). The lack of good quality connectivity, devices, and support from their parents (many could not work from home during the pandemic) are the key reasons.

Future studies should incorporate more data, increase the sample size, and see if mathematics coefficients also suggest a statistically significant impact on learning inequalities. It is also essential to have a heterogeneous sample of children enrolled in public, not-for-profit, and for-profit private schools to thoroughly understand the impact of school closure on learning inequalities. Assessing only children in public schools tends to underestimate learning inequalities in the Brazilian context.

7 Conclusion

This paper provides the first evidence of the effects of school closure during the 2020 Covid-19 pandemic on young pupils starting compulsory Education in Brazil. The data presented is exceptionally rich since it associates pupils’ learning at the start and end of the school year for two cohorts (2019 and 2020) in the second year of preschool. Value-added models provide robust evidence for the impact of school closure on learning loss and learning inequalities.

Schools in Brazil, especially public schools, stayed closed for longer periods than observed in other countries (CAMPOS, 2021CAMPOS, C. O. (coord.). Levantamento internacional de retomada das aulas presenciais. [S. l.]: Vozes da Educação, 2021. Available from: https://vozesdaeducacao.com.br/wp-content/uploads/2020/10/Levantamento-internacional_Retomada-presencial-das-aulas.pdf . Access in: 2021/10/12.
https://vozesdaeducacao.com.br/wp-conten... ). The lack of coordination by the Ministry of Education along with ineffective policies to reduce Covid-19 transmissions created a very challenging scenario for families and schools (CASTRO et al ., 2021). Most schools implemented several synchronous and asynchronous strategies to keep learning when schools could not reopen for face-to-face interactions. Remote learning is a big challenge, especially for young children and/or families from low SES with limited access to connectivity and devices to allow interactions between pupils and their teachers.

The effects presented are slightly higher than those estimated in developing countries, but smaller than estimated in Mexico or State public schools in São Paulo, which focused on older students. One important characteristic of the study is to include private schools with a heterogeneous group of families considering SES and a focus on younger children, starting compulsory Education. The results described in this paper can be seen as a benchmark for future studies, especially those assessing the impact on public schools. The impact on children attending public schools is likely larger and a recovery plan focused on disadvantaged children should be the focus of policymakers.

The disruption to Education caused by the Covid-19 pandemic constitutes the worst Education crisis on record (THE WORLD BANK; UNESCO; UNICEF, 2021) and three steps have not been taken in Brazil and other countries in the region that are crucial and urgent. First, a national diagnosis on the impact of school closure on students in different age groups. The effect is likely to vary considering family SES, pupils’ age, and region in the country. Good quality evidence can help identify groups and areas that have been more affected and guide future compensatory policies. Second, it is necessary to elaborate a recovery plan based on the best available evidence of what works to foster learning for those students who were left behind and had little or no access to remote learning and/or to school during 2020/2021. Third, the plan should be broadly discussed with society and more resources need to be placed in Education to accelerate learning trajectories in the next years. The absence of coordination by the Ministry of Education (Federal Government) is likely to increase inequality among regions, States, or cities. Poor areas and smaller cities must have the necessary support to implement their recovery plans and prevent inequality to rise among States and cities in the years following the pandemic.

Acknowledgment

The study was approved by the Research Ethics Committee of Centro de Filosofia e Ciências Humanas/Universidade Federal do Rio de Janeiro (CAAE: 33865520.2.0000.5582, Opinion No. 4.228.330). The authors are grateful for the partnership and support received throughout the study from all the schools and families. The study was funded by Fundação Maria Cecília Souto Vidigal. Initial findings were presented at the IX International Simposium from Núcleo Ciência pela Infância . The report can be downloaded at https://www.fmcsv.org.br/pt-BR/biblioteca/impacto-covid-criancas/

References

Publication Dates

History