This paper details the development and deployment of Crisis Message Detector-1 (CMD-1), a machine learning-enabled natural language processing (NLP) system designed to expedite the triage of mental health crisis chat messages within a large telehealth network. The two-stage NLP system, incorporating keyword filtering and logistic regression, was trained on 721 chat messages (32% representing potential crises). Retrospective (N=481) and prospective (N=102,471) test sets demonstrated high sensitivity (0.99 and 0.98, respectively) and improved AUC (0.82 and 0.98, respectively) and PPV (0.35 and 0.66, respectively). CMD-1 significantly reduced median response times from 9 hours to 8-13 minutes. The study concludes that NLP-based models can effectively identify crisis messages, aiding human triage within existing clinical workflows.

Akshay Swaminathan, Iván López, Rafael Antonio Garcia Mar, Tyler Heist, Tom McClintock, Kaitlin Caoili, Madeline Grace, Matthew Rubashkin, Michael N. Boggs, Jonathan H. Chen, Olivier Gevaert, David Mou, Matthew K. Nock