The global surge in mental health conditions strains healthcare systems, highlighting the need for accessible mental healthcare. Self-guided interventions, offering on-demand access to coping strategies, show promise in expanding care access. However, these interventions often present significant cognitive and emotional challenges, such as the complexity of cognitive restructuring (identifying thinking traps and reframing thoughts), leading to limited engagement and adoption. This research investigates how human-language model (HLM) interaction can improve the accessibility and effectiveness of self-guided cognitive restructuring interventions. Unlike previous small-scale or simulated studies, this research utilizes a large-scale, ecologically valid field study to understand how individuals experiencing mental health challenges interact with HLM-supported interventions. The study addresses the lack of knowledge regarding end-user preferences and the potential for demographic biases in language model performance. The central research question explores how to design and evaluate an effective and equitable HLM-based cognitive restructuring intervention.

This work builds upon existing research in three key areas: digital mental health interventions, AI for mental health, and the design of HLM interaction systems. Digital mental health interventions have focused on text-based support, peer-to-peer networks, and on-demand therapy platforms. Studies on self-guided interventions have explored various formats, including apps for meditation, mood tracking, and emotion regulation. Existing literature emphasizes the challenges of designing effective self-guided interventions without professional support and the high dropout rates often observed. The field of AI for mental health has explored machine learning for measuring mental health constructs, building virtual assistants and chatbots, and developing AI systems to assist mental health providers. Previous computational work on cognitive restructuring has largely been limited to small-scale, wizard-of-oz studies. Finally, research on HLM systems focuses on their application in diverse areas such as creative writing and programming, providing a foundation for studying their potential in mental health interventions. This research extends previous work by evaluating a large-scale, real-world implementation of an HLM-based cognitive restructuring intervention and investigating its equity across diverse populations.

The nine-month study involved iterative design, prototyping, and evaluation with mental health experts. The study used Mental Health America's (MHA) website, a platform reaching millions of users. Participants (N=15,531) were MHA visitors who opted into the study and provided informed consent (minors with informed assent and parental waiver). A mixed-methods approach combined quantitative and qualitative data. The study first formulated design hypotheses based on feedback from early prototypes and collaborations with mental health professionals. These hypotheses focused on assisting users with cognitively and emotionally challenging processes, contextualizing reframes through situations and emotions, integrating psychoeducation, facilitating interactive reframe refinement, and ensuring safety. A five-step HLM-based cognitive restructuring tool was then developed. Participants described their negative thought, situation, and emotion. The system then used a language model to suggest potential thinking traps, provide definitions, and offer multiple reframe suggestions as starting points. Participants could iteratively refine reframes through manual edits and additional model suggestions. Safety mechanisms included content filtering to avoid unsafe or inappropriate content generation. Quantitative measures included changes in emotion intensity, reframe relatability, helpfulness, memorability, and skill learnability. Qualitative data came from open-ended feedback questions. Randomized controlled trials were conducted to evaluate the impact of individual design hypotheses, ablating specific features (e.g., removing psychoeducation or the option for interactive refinement). Finally, equity analysis assessed intervention effectiveness across various demographics and issues (categorized using a GPT-3 model trained on manually labeled data). To improve equity, particularly for adolescents, the study explored modifying language model suggestions to be simpler and more casual.

The study found that 67.64% of participants experienced a positive shift (reduction) in emotional intensity, and 65.65% reported helpfulness in overcoming negative thoughts. Participants with higher initial emotion intensity reported a greater reduction in emotion intensity post-intervention, indicating the potential for greater benefit in those with more intense negative emotions. The majority of participants found the generated reframes relatable, helpful, and memorable. Qualitative feedback highlighted the system's help in overcoming cognitive and emotional barriers, providing a less triggering experience, and enabling exploration of multiple perspectives. Randomized trials revealed that contextualizing thoughts through situations led to more helpful reframes without increasing dropout rates. Integrating psychoeducation had limited impact on overall effectiveness. Increased interactivity, through the option to seek additional reframe suggestions, resulted in a 23.73% greater reduction in emotion intensity. Participants seeking actionable reframes reported superior outcomes. Equity analysis revealed disparities across demographics and issues. Adolescents, males, and those with lower education levels reported worse outcomes; those with higher education levels and those addressing work or parenting issues reported better outcomes. An intervention tailoring language model suggestions to be simpler and more casual for adolescents led to a 14.44% increase in reframe helpfulness for the 13-14 age group.

This study demonstrates the potential of HLM interaction to support self-guided cognitive restructuring, addressing limitations of traditional self-guided interventions. The findings support several design hypotheses: personalization improves effectiveness, especially when incorporating situational context, and greater interactivity enhances outcomes. The observed disparities highlight the need for equitable design considerations. The success of simplifying language for adolescents suggests that tailoring interventions to specific demographics is crucial. This research provides valuable insights for designing effective and equitable HLM-based mental health interventions. The structured nature of the intervention, closely following established therapeutic practices, appears to contribute to positive outcomes, suggesting that even with high user engagement, careful consideration is needed to maintain adherence to evidence-based therapies. The potential for over-reliance on language model assistance warrants future research to ensure the development of a user’s independent learning and practical application of cognitive restructuring skills.

This paper presents the design and evaluation of a novel HLM-based system for self-guided cognitive restructuring, demonstrating its effectiveness in reducing emotional intensity and overcoming negative thoughts. Key design principles, such as personalization and interactive refinement, were shown to improve outcomes. Disparities across demographics highlighted the need for equitable design, with promising results from tailoring the intervention for adolescents. Future research should explore more sophisticated personalization strategies, adaptive difficulty levels, and longer-term outcome evaluations. The study's open-source code and detailed methodology contribute to the advancement of HLM-based mental health interventions.

The study's reliance on a single platform might limit the generalizability of findings. The relatively small number of older adults and participants from certain racial/ethnic groups limits the conclusions that can be drawn about equity across all demographics. The focus on short-term outcomes necessitates future research to evaluate long-term effects. The study's reliance on self-reported data introduces potential biases. The effectiveness of the intervention depends on the quality of the language model, so future development of language models will change the effectiveness.