Detection of acute 3,4-methylenedioxymethamphetamine (MDMA) effects across protocols using automated natural language processing

Objective evaluations in psychiatry are needed to complement subjective clinical assessments. Analyzing free speech is a promising approach due to its cost-effectiveness and ubiquity. Advances in natural language processing (NLP) allow for quantitative, automated speech characterization. This research focuses on acute drug effects, where mental state changes are central. Traditional assessments using self-report measures have limitations in sensitivity and objectivity, potentially failing to capture the full spectrum of effects from emerging drugs or individual variability. Computerized speech analysis may overcome these limitations. Prior work by the authors indicated that NLP could distinguish between different drugs (MDMA vs. methamphetamine vs. placebo) using semantic content analysis, showing high accuracy. Another study used a bag-of-words approach to analyze speech differences between MDMA and placebo, revealing words linked to social and emotional content. These prior studies, however, were limited by small sample sizes and lack of independent validation. This current study aims to address these limitations by analyzing a larger dataset with a broader range of speech features (semantic, acoustic, psycholinguistic) and using multiple independent datasets for validation. The hypotheses were: (i) unique speech signatures for each drug condition; (ii) dose-dependent changes with MDMA; (iii) greater emotional expression during monologue vs. description tasks; (iv) model generalizability across datasets.

The study builds upon previous research on objective evaluations in psychiatry, specifically utilizing NLP to analyze speech. Prior research has shown the potential for quantifying incoherence in schizophrenic speech and other clinical applications. Traditional manual coding methods have limitations in objectivity. In the context of drug effects, previous work by the authors demonstrated the ability of NLP to differentiate MDMA effects from those of other substances, using semantic analysis. These studies, however, were limited by small sample size and a lack of independent validation. This study aims to address these limitations by analyzing a wider range of acoustic, semantic, and psycholinguistic features, including a larger sample size and multiple validation datasets. The study design also considers the interaction of different speech elicitation tasks, which has implications for interpreting the mental states underlying speech variability.

The study used a randomized, double-blind, within-participants design. 35 healthy adults (31 in training/validation, 4 excluded due to unusable recordings) with prior MDMA use underwent screening and completed the study. Participants received placebo, two doses of MDMA (0.75 and 1.5 mg/kg), and oxytocin (20 IU) across four sessions, spaced at least 5 days apart. A double-dummy approach ensured blinding. Before each session, participants abstained from food, alcohol, and other substances. Two 5-min speech tasks were administered during peak drug effects: a Description task (about an important person, with a researcher present) and a Monologue task (on any topic, alone). Speech was recorded, transcribed, and analyzed. Feature extraction included 88 acoustic features (using Praat and Python), semantic features (Latent Semantic Analysis – LSA, using NLTK and a large corpus), and psycholinguistic features (CPIDR, parts of speech, lexical content). Univariate analysis (Wilcoxon sign rank tests, FDR correction) compared drug conditions for each task. Partial correlations analyzed feature relationships within conditions and tasks. Classification analysis (linear SVM, nearest neighbors, random forest, nested leave-one-participant-out cross-validation) assessed the predictive power of features to distinguish drug conditions. The models were validated on two independent datasets with similar conditions (Description task only). Feature selection was conducted using two-sample t-tests. Multivariate analysis explored the weights assigned by the linear SVM models.

The analysis revealed that different features were most informative depending on the drug and task. Acoustic features were more prominent in detecting oxytocin effects, particularly related to prosody and emotional expression, with variations in formant frequencies. The Description task with a researcher present showed a different pattern compared to the Monologue task, with pause duration being a prominent feature. Classification analysis using combined features achieved cross-validated accuracies up to 87% in the training/validation dataset, and validation datasets yielded accuracies up to 92% and 66% (chance = 50%). The most accurate classifications were obtained using linear SVMs, followed by Random Forest and Nearest Neighbors. The highest accuracies were observed for the low dose MDMA compared to placebo. Multivariate analysis of the weights of the most informative features within the optimal models highlighted the different contribution of features across tasks and drug conditions. Higher doses of MDMA didn't necessarily correlate with larger differences from placebo, with the lower dose sometimes outperforming the higher dose in classification accuracy. Partial correlations showed stronger relationships between speech variables under active drug conditions, with distinctive patterns across tasks. A multidimensional scaling analysis revealed two key dimensions: one related to the task type (monologue vs. description) and another related to MDMA dose. Differences in race between datasets are noted as a potential confounder.

The findings demonstrate the potential of automated speech analysis as a non-invasive tool for assessing acute drug effects. The study successfully demonstrated that multiple types of speech features contribute to detecting the effects of MDMA and oxytocin. The variations in significant features and classification accuracy across different tasks (monologue vs. description), doses and datasets, highlight the influence of experimental details. The task's impact suggests that how speech is elicited is critical for capturing meaningful drug effects. The higher accuracy in independent datasets (especially ID1) strengthens the potential generalizability of this method. The findings could provide objective and readily available data complementing traditional clinical assessments, particularly useful for monitoring changes over time or addressing limitations in access to trained professionals.

This study provides strong evidence supporting the use of automated speech analysis to detect acute mental state changes due to MDMA and oxytocin. The method offers a promising, objective, and cost-effective approach for assessing intoxication. Future work should focus on refining the methods, utilizing larger and more diverse datasets, and systematically exploring the contribution of specific speech markers for various drugs and populations, potentially considering factors such as race and ethnicity. The findings advance the potential of digital phenotyping for understanding human mental states.

The study's use of datasets from previous research introduces limitations due to methodological differences in tasks and timing of speech collection. The relatively limited number of drug conditions investigated restricts the generalizability of the findings. The potential influence of race/ethnicity on the results should be investigated further. The study focused primarily on cross-validated analysis, making generalization to other datasets a point of ongoing research.