The COVID-19 pandemic forced a global shift to remote learning, raising concerns about its impact on educational outcomes, particularly in middle-income countries like Brazil. Prior to the pandemic, Brazil already faced a "learning crisis," with significant numbers of students lacking basic literacy and numeracy skills. Remote learning exacerbated these pre-existing challenges due to factors such as limited internet access, lack of dedicated study spaces, and insufficient parental support. The pandemic added further pressures, including increased child labor and violence. International organizations estimated millions of additional students would drop out due to school closures. Quantifying learning losses is crucial for informed policy decisions regarding school reopening, balancing health risks with educational benefits. Existing research, primarily from high-income countries and often with limited generalizability, provided insufficient evidence on the impact of remote learning in secondary education in middle-income countries during the pandemic. This study aims to fill this gap by leveraging a natural experiment in São Paulo State, Brazil.

The existing literature on the impact of remote learning on educational outcomes presents several limitations. Much of the existing research focuses on high-income countries and often examines tertiary education or charter schools, limiting the generalizability to the context of middle-income countries and public secondary schools. Studies focusing on middle-income countries often use remote learning as a way to expand access to education in previously underserved areas, which differs from the counterfactual of in-person learning. Studies evaluating learning losses during the pandemic frequently rely on simulations, structural models, or suffer from comparability issues. The few studies that employ appropriate counterfactuals often rely on strong assumptions about the relationship between school closures and changes in instruction mode. This research uniquely addresses these gaps by focusing on a specific context within a middle-income country.

This study uses a natural experiment in São Paulo State, Brazil, where in-person classes were held during the first quarter of 2020 but shifted to remote learning thereafter. The researchers utilized quarterly standardized tests administered throughout 2020, allowing for a differences-in-differences analysis comparing student outcomes before and during the pandemic. A further natural experiment, based on staggered school reopening in the fourth quarter of 2020, provided a triple-differences strategy to evaluate the impact of resuming in-person classes. The authors employed ordinary least squares (OLS) regressions with grade fixed effects and school-level clustered standard errors. Propensity score matching was used to account for selection effects in test scores. Heterogeneous treatment effects were examined by student characteristics (age, gender, race), neighborhood income, and prior experience with online learning. Dropout risk was defined as the absence of math and Portuguese grades for a given quarter, reflecting cumulative student disengagement. The analysis controlled for the effects of COVID-19 local disease activity. Data included quarterly student attendance, scorecard grades, and standardized test scores for 6th to 12th graders between 2018 and 2020, comprising millions of observations.

The study reveals substantial negative impacts of remote learning on both dropout rates and test scores. The differences-in-differences analysis indicated a 365% increase in dropout risk (from approximately 10% to 35%), representing a significant rise in student disengagement. Regarding academic performance, naive comparisons initially suggested improved test scores in 2020, but the differences-in-differences analysis, after controlling for selection effects through propensity score matching, revealed a 0.32 standard deviation decrease in test scores. This corresponds to students effectively learning only 27.5% of the in-person equivalent. The authors estimate that students lost approximately 0.009 standard deviations of learning per week during remote learning. Resuming in-person classes, even partially, led to a 20% increase in test scores for high school students, compared to the control group. Heterogeneous treatment effects showed that negative impacts were more pronounced for girls, non-white students, students from low-income neighborhoods, and schools lacking prior experience with online academic activities. Learning losses were particularly severe in math compared to Portuguese. Notably, the negative effects of remote learning did not systematically vary with local COVID-19 disease activity.

These findings highlight the significant negative consequences of remote learning for secondary education in Brazil during the pandemic. The dramatic increase in dropout risk and substantial learning losses underscore the need for interventions to address the educational challenges brought on by widespread school closures. The positive effect of resuming in-person classes, even partially, emphasizes the importance of in-person instruction for effective learning. The observed disparities in the effects of remote learning across different student groups emphasize the need for targeted support to mitigate inequalities and ensure equitable access to education. The study's findings contribute to the ongoing debate about school reopening during pandemics, providing crucial evidence for policy decisions.

This study provides strong evidence of the detrimental impact of remote learning on secondary education in Brazil during the COVID-19 pandemic. The substantial increases in dropout rates and significant learning losses highlight the importance of in-person instruction and the need for targeted support for vulnerable student populations. Future research could explore the long-term effects of these learning losses, the efficacy of different pedagogical approaches in remote learning contexts, and strategies for more effective and equitable remote education in middle-income countries. Further research is warranted to investigate the influence of parental education levels on the impact of remote learning.

The study's reliance on administrative data may introduce measurement error in capturing dropout rates. The focus on public schools in São Paulo State limits the generalizability of the findings to other contexts or school systems. The study's analysis does not capture the experiences of private schools or consider the influence of parental education on outcomes.