Applying speech technologies to assess verbal memory in patients with serious mental illness

Verbal memory deficits are some of the most profound neurocognitive deficits associated with schizophrenia and serious mental illness (SMI). Traditional assessments are resource-intensive and limited in frequency. The authors developed a digital, ambulatory verbal memory test with automated scoring and repeatable self-administration on smart devices. In 104 adults (25 patients with SMI, 79 healthy volunteers), 92% of 1035 speech responses were of sufficient quality (patients: 86%; healthy: 96%). Human-rated recall scores were approximated using NLP on transcripts (R = 0.83, within human-to-human correlations R = 0.73–0.89). A fully automated pipeline using ASR transcripts also performed well (R = 0.82) and correlated highly with scores based on human transcripts (R = 0.99). Findings support leveraging speech technologies for frequent assessment of verbal memory for clinical monitoring in psychiatry.

Terje B. Holmlund, Chelsea Chandler, Peter W. Foltz, Alex S. Cohen, Jian Cheng, Jared C. Bernstein, Elizabeth P. Rosenfeld, Brita Elvevåg