Gun violence in US K-12 schools has dramatically increased over the past two decades, leading to heightened pressure on school administrators to implement preparedness measures, primarily through drills. These drills vary widely in their format, from simple lockdowns to elaborate simulations involving simulated gunfire and masked actors. However, the long-term impact of these drills on the well-being of students, teachers, parents, and the broader school community remains poorly understood. Existing research, largely based on small-scale studies with methodological limitations such as self-reported surveys and behavioral observations in individual schools, presents conflicting results. Some studies suggest improved lockdown procedure compliance, while others highlight increased anxiety, fear, and a sense of decreased safety. Critically, there's a lack of research assessing the long-term mental health consequences of these drills, which are experienced by almost all students. This study aims to address this gap by using a large-scale, data-driven approach to assess the lasting impact of school shooter drills on the mental health and community dynamics within affected school communities.

The existing literature on school shooter drills is mixed and methodologically limited. Ethical concerns and the thankfully low frequency of actual active shooter events hinder studies focusing on lives saved by drills. Previous research has primarily relied on behavioral observations or surveys to assess drill effectiveness and related emotional responses. Some studies suggest improved performance in following lockdown procedures, while others show continued difficulties. Similarly, some participants reported feeling more prepared after drills, while others felt less safe or more anxious. These studies are often limited to individual schools or districts, limiting generalizability. A key limitation is the lack of research examining the long-term mental health impacts of drills on students, teachers, and parents, despite concerns that drills may be developmentally inappropriate, traumatizing, and increase fear and anxiety. This study addresses this gap by applying a large-scale analysis of social media data to examine these long-term impacts across diverse school communities.

This study utilizes a mixed-methods approach combining large-scale social media data analysis with focus group interviews. Drill dates and locations were initially identified through a survey of student, teacher, and parent volunteers from a gun violence prevention organization. This survey collected data on drill dates, school type, and location. Social media data was then collected for a 90-day period before and after each drill. Three data collection strategies were employed: (1) analyzing Twitter posts from individuals following the official Twitter accounts of the schools; (2) analyzing Twitter posts geo-located within the school districts; and (3) analyzing Reddit posts from communities associated with the school names. A control group of an equal number of posts was also selected from Twitter, geographically distanced from the schools. In total, 54 million posts were analyzed. Machine learning classifiers, previously trained and validated on Reddit data, were used to identify social media posts expressing symptoms of stress, anxiety, and depression. The Linguistic Inquiry and Word Count (LIWC) tool was used to analyze psycholinguistic features related to cognitive processes, interpersonal focus, and social concerns. Interrupted time series (ITS) analysis was employed to assess changes in mental health and psycholinguistic indicators before and after drills. Four robustness checks were conducted to strengthen causal inferences: (1) temporal alignment of data based on drill dates; (2) isolating the effects of other events like mass shootings; (3) comparing actual time series to a counterfactual predicted by an ARIMA model; and (4) comparing the treatment group to a control group of non-school-related Twitter posts. Finally, six focus groups comprising 34 students, parents, and teachers provided qualitative data to triangulate the quantitative findings. The focus groups were conducted via Zoom and analyzed using inductive and iterative semi-open coding.

Analysis of social media data revealed a significant increase in expressions of stress, anxiety, and depression following school shooter drills. The long-term change (LC) showed a 42% increase in stress/anxiety and a 38.7% increase in depression in the 90 days following drills. Immediate changes (IC) were also observed. Lexical analysis showed a shift in word usage, with positive words becoming less frequent and words related to school safety and support becoming more frequent after drills. Analysis of psycholinguistic indicators showed increases in perceptual processing (10.78% LC), first-person plural pronouns (47.79% LC), and expressions of social concerns and work-related interactions (33.7% and 106.18% LC respectively). These findings suggest heightened awareness, increased interpersonal interactions, and a greater focus on community engagement following drills. Causality analysis, using the four robustness checks, provided strong support for a causal link between the drills and the observed changes in mental health and community outcomes. The effects were not solely attributable to other events like mass shootings and were significantly larger than those observed in the control group. Subgroup analyses revealed that high school communities and teachers experienced the most significant changes. Focus group interviews corroborated the quantitative findings, with participants reporting increased anxiety, fear, and stress related to drills. Teachers described their own emotional distress and the impact on students, some of whom experienced panic attacks. Parents reported their own anxieties and challenges in supporting their children. Despite the negative mental health impacts, the increased community engagement observed in social media data suggested the drills inadvertently fueled conversations and collective action around school safety.

This study provides the first large-scale, rigorous empirical evidence demonstrating the significant negative impact of school shooter drills on the psychological well-being of K-12 school communities. The substantial increases in anxiety, stress, and depression observed across diverse schools and over a sustained period suggest that current drill practices may be counterproductive, particularly in light of the lack of strong evidence demonstrating their effectiveness in saving lives. The findings highlight the need to carefully consider the potential psychological harm of drills and explore alternative proactive strategies for improving school safety. The increase in civic engagement, while a positive outcome, shouldn't overshadow the detrimental effects on mental health. The study’s findings underscore the importance of prioritizing evidence-based, trauma-informed approaches to school safety.

This research offers compelling evidence that current school shooter drills negatively impact the mental well-being of school communities. The findings strongly suggest a need for a critical re-evaluation of these practices and a shift towards proactive, evidence-based strategies that foster a positive school climate, provide access to mental health resources, and utilize effective threat assessment programs. Future research should investigate the long-term consequences of drill exposure, explore the effectiveness of alternative safety measures, and examine how different drill formats influence psychological outcomes. The study's large-scale, multi-method approach provides a valuable model for future research on the impacts of school safety interventions.

This study has some limitations. While the machine-learning classifiers were validated, clinical inferences should be made cautiously. The study relies on publicly available social media data, which may not represent the views of all school community members, particularly younger students and those not active on social media. Self-selection bias inherent in social media data should also be considered. The study's time frame was limited, and longer-term follow-up studies are needed. Finally, the study did not collect detailed information about the specific types and implementations of the drills, limiting a precise analysis of the influence of specific drill formats on outcomes.