Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring

The paper addresses accessibility barriers in self-guided mental health interventions, which can be cognitively demanding and emotionally triggering without clinician support. It investigates whether human–language model interaction can improve a core CBT skill—cognitive restructuring—when delivered at scale to real users seeking care. The authors pose three research questions: RQ1 on designing a language-model-supported self-guided cognitive restructuring intervention; RQ2a on overall effectiveness for reducing negative emotions and overcoming negative thoughts; RQ2b on how individual design hypotheses affect outcomes; and RQ3 on equity across subpopulations and strategies to improve it. The study situates the work in the context of growing mental health needs and limited access to care, proposing LM assistance to identify thinking traps and generate reframes, and emphasizing safety, personalization, interactivity, and fairness.

Related work spans: (1) Digital mental health interventions, including self-guided CBT/DBT tools and apps for mood tracking, emotion regulation, and loneliness, with noted challenges in engagement and dropout. Prior CBT digitizations often digitize worksheets with limited guidance. (2) AI for mental health, including NLP to measure therapeutic constructs (empathy, strategies, engagement), chatbots/assistants, and LM-based reframing methods. Prior cognitive restructuring work largely used small-scale, lab-based, or wizard-of-oz studies; some LM methods generate reframes and analyze reframe qualities (e.g., avoiding overly positive tone). (3) Human–LM interaction design for real-world tasks (creative writing, coding, brainstorming). The paper extends this literature with a large-scale, ecologically valid deployment, randomized trials of design choices (contextualization, psychoeducation, interactivity), and equity analysis with adaptations for adolescents.

Study setting and participants: Deployed a self-guided cognitive restructuring tool on Mental Health America (screening.mhanational.org), a large mental health platform. IRB-approved; informed consent (adults) and assent (minors) obtained. Participants were platform visitors aged 13+. Of 43,347 consenting users, 15,531 completed outcome surveys and were analyzed (dropout 64.17%, comparable to typical rates for self-guided tools). Multiple randomized trials ran in parallel on independent participant subsets.

Intervention design (five-step workflow): (1) Participant inputs current negative thought. (2) Participant provides a recent situation (context). (3) Participant reports current emotion and its intensity (1–10). (4) LM-assisted identification of thinking traps: a GPT-3 model fine-tuned on 13 thinking traps ranks likely distortions with likelihoods; definitions and examples provided as psychoeducation; model top-1 accuracy 62.98%. (5) LM-assisted reframing: a retrieval-enhanced in-context GPT-3 approach (k=5 similar examples from expert-authored (situation, thought, reframe) triples) generates multiple reframe suggestions. Users select or write their own reframe, then can iteratively refine via additional LM help: make it more relatable, figure out next steps/actions, or seek supportive/validating language. Top-p sampling used to generate multiple suggestions.

Safety: Combined Azure OpenAI classification-based content filtering (hate/sexual/violence/self-harm categories) with rule-based regex filters (≈50 patterns for suicide/self-harm phrases). Participants could flag inappropriate suggestions; crisis hotline info provided.

Design hypotheses tested: H1 assistance on cognitively/emotionally challenging steps; H2 contextualization via situations and emotions; H3 integrating psychoeducation; H4 facilitating interactive refinement; H5 safety mechanisms.

Randomized controlled trials (RCTs):

• Contextualization via situation: enabled vs disabled.
• Contextualization via emotion: enabled vs disabled.
• Psychoeducation: added vs removed.
• Interactivity: option to get more LM suggestions (actionable/empathic/personalized) enabled vs disabled.
• Equity adaptation: adolescent-focused simplification—reframes rewritten to lower reading complexity and more casual tone vs standard.

Measures: Pre/post change in emotion intensity (−10 to +10 computed from 1–10 ratings), and post-use 1–5 Likert ratings for reframe relatability, helpfulness, memorability, and skill learnability. Two-sided t-tests used; 95% bootstrapped CIs reported in figures. Qualitative feedback collected via open-ended questions. Additional analyses included issue categorization (finetuned GPT-3, accuracy 73%) and demographic subgroup outcomes.

Overall effectiveness (N≈1,922 for primary outcomes):

• 67.64% reported reduced emotion intensity; 24.56% no change; 7.80% a negative shift (most were −1). Mean reduction = 1.90 (SD 1.29) on −10 to 10 scale.
• Reframes: relatability mean 3.84; helpfulness mean 3.33 (65.65% agreement); memorability mean 3.52 (70.49% agreement); skill learnability mean 3.39 (67.38% agreement).
• Higher initial emotion (>7/10) saw larger reduction (2.13 vs 0.95; p < 1e−54) but lower relatability (−8.29%), helpfulness (−11.46%), memorability (−9.26%), and learnability (−12.04%).

Design ablations (RCTs):

• Contextualization via situation: +2.80% reframe helpfulness (3.31 vs 3.22; p≈0.019); similar completion rates; relatability unchanged (N=1,636).
• Contextualization via emotion: −3.86% relatability (3.72 vs 3.87; p<0.001; N=4,016); completion unaffected. Likely due to model not incorporating emotions in generation.
• Psychoeducation: no significant quantitative improvements in outcomes (N=1,850), though qualitative feedback valued definitions/examples/strategies.
• Interactivity (option to seek more LM help): +23.73% greater reduction in emotion intensity (2.19 vs 1.77; p≈0.019; N=2,165); other outcomes not significantly different. Among those with the option, users who engaged further reported higher helpfulness (+5.57%; 3.41 vs 3.23; p<1e−5) and learnability (+4.86%; 3.45 vs 3.29; p<0.001; N=992). Seeking actionable reframes was associated with significantly better outcomes across all five measures; empathic requests improved emotion reduction (+21.86%), helpfulness (+5.52%), and learnability (+5.14%); personalization requests showed no significant differences.

Equity analyses:

• Issues: Hopelessness and Loneliness showed worse outcomes (e.g., Hopelessness: lower emotion reduction −41.27%, helpfulness −16.61% vs population means). Parenting and Work issues showed better outcomes (e.g., Work: higher emotion reduction +22.22%, helpfulness +10.97%). Tasks & Achievement showed lower relatability and learnability; participants often sought more concrete actions.
• Demographics: Adolescents (≤17) had worse outcomes across relatability, helpfulness, memorability, learnability; males reported lower helpfulness (−7.52%) and learnability (−5.88%); lower education (Middle School) showed worse outcomes, while Graduate education showed improvements across all reframe metrics. Race/ethnicity effects were mixed; some smaller subgroups showed lower specific outcomes.
• Adolescent adaptation RCT: Simplifying and making reframes more casual increased outcomes. Ages 13–14: +8.60% relatability (4.04 vs 3.72; p=0.0161) and +14.44% helpfulness (3.17 vs 2.77; p=0.0049; N=148). Ages 15–17: +15.58% helpfulness (3.19 vs 2.76; p=0.0042; N=174). No significant effects for adults (≥18). Reading complexity analyses showed adolescents wrote at lowest complexity, motivating the adaptation.

Safety and content flags:

• 0.65% (301/46,593) of LM suggestions were flagged by users; none contained suicide/self-harm references, suggesting filters were effective. Some flagged items echoed negative self-labels (risk of reinforcing beliefs). Users typically had alternative suggestions; dropout did not increase among flaggers (slightly lower).

Findings show that human–LM interaction can effectively support cognitively and emotionally challenging components of self-guided cognitive restructuring. The intervention reduced negative emotion intensity for most users and produced reframes many found believable, helpful, and memorable, addressing RQ2a. RCTs clarified design choices (RQ2b): personalization through situations improves helpfulness; interactivity enabling iterative LM support improves emotional outcomes, with actionable and empathic guidance particularly valuable; psychoeducation was appreciated but did not improve short-term quantitative outcomes; soliciting emotions without incorporating them can reduce perceived fit. Equity analyses (RQ3) revealed heterogeneous effects across issues and demographics; adapting language complexity for adolescents measurably improved outcomes, illustrating a pathway to greater equity. The work underscores the importance of calibrated interactivity (avoiding over-reliance while enabling productive struggle), personalization aligned with available model inputs, structured interfaces aligned with CBT practices (vs free-form chat), and robust safety measures in high-stakes mental health settings. The approach can complement traditional care by providing on-demand, scalable skill practice and support.

The authors present a human–language model interaction system that guides users through cognitive restructuring (identifying thinking traps and reframing thoughts). In a large, ecologically valid deployment with 15,531 participants, the system reduced negative emotion intensity for most users and supported helpful reframing. Randomized trials validated design choices: contextualizing via situations and enabling iterative interactivity improved outcomes; targeted adaptations (simpler, more casual reframes) improved adolescent experiences. The study contributes design guidance for self-guided, LM-supported mental health tools, highlights equity considerations across subpopulations, and demonstrates safety-aware deployment. Future work should investigate longer-term skill acquisition and real-world application, enhanced personalization (e.g., incorporating emotions), and subgroup-specific adaptations to improve fairness and effectiveness.

• Generalizability: Single-platform deployment (MHA) limits external validity; some demographic groups (e.g., 65+, AIAN, NHPI) were underrepresented.
• Short-term evaluation: Outcomes reflect single-use, immediate effects; longer-term skill acquisition and behavior change were not assessed.
• Model dependence: Results depend on the quality of LM-generated thinking traps and reframes; improvements in models may change outcomes.
• Emotion incorporation: The system did not explicitly incorporate self-reported emotions into generation, leading to lower relatability when emotions were solicited.
• Complexity of issues: For nuanced issues (e.g., hopelessness, loneliness), suggestions were sometimes perceived as superficial; more sophisticated modeling may be needed.