Language or rating scales based classifications of emotions: computational analysis of language and alexithymia

Quantitative assessments in mental health predominantly rely on rating scales, despite language being the natural medium for expressing emotions. This study addresses the question of whether language-based approaches, particularly when analyzed computationally, might offer superior accuracy in classifying emotional states compared to traditional rating scales. The study also explores the influence of alexithymia, a condition characterized by difficulty identifying and describing emotions, on the accuracy of both language-based and rating scale assessments. The increasing prevalence of major depressive disorder (MDD) and generalized anxiety disorder (GAD) underscores the critical need for improved diagnostic tools. More accurate and efficient assessments are linked to better treatment outcomes and long-term prognosis. Alexithymia, prevalent in approximately 10% of the general population, is strongly associated with both MDD and GAD, potentially influencing the validity of self-report measures. Rating scales offer quantifiable data, ease of use, and efficiency, but they may oversimplify the complexity of individual emotional experiences. Language, in contrast, allows for richer and more nuanced expression of mental states. Computational methods, such as Latent Semantic Analysis (LSA), offer a means of quantitatively analyzing linguistic data, creating multi-dimensional representations of emotional expression. This study combines these approaches to investigate whether language-based responses, analyzed computationally, provide a more valid assessment of emotional states than rating scales, especially in the context of varying levels of alexithymia.

Existing research highlights the limitations of rating scales in capturing the complexity of emotional experience. Psychometric properties of rating scales, including those assessing alexithymia, are often challenged. Unidimensional scales may lack the nuance needed to fully represent the individual's emotional landscape. Conversely, language-based assessments are considered more intuitive, as language is the natural way to communicate mental states. Recent studies have shown success in using computational models to analyze textual data, offering a potential pathway for improving assessments of depression and anxiety. However, the interaction between alexithymia and the accuracy of language-based assessments requires further investigation. Previous research using computational language assessment showed high correlations with rating scales, but studies have not adequately controlled for alexithymia. There is a need to understand whether alexithymia influences the ability to both generate and interpret emotional language. The Multiple Code Theory suggests that emotional information is processed using a specific code, and disruption of this code in alexithymia might lead to difficulties in recognizing and labeling emotions, potentially impacting the accuracy of language-based assessments. While some research indicates that basic emotion naming may be intact in individuals with alexithymia, difficulties in emotional empathy and nuanced expression are often observed. Previous research has established connections between alexithymia and reduced verbal expressiveness, emotional openness, and the perception of emotional intensity.

This study employed a two-phase design with separate participant groups. Phase 1 (N=304) involved participants generating narratives describing events related to depression, anxiety, satisfaction, and harmony. They also provided five descriptive words for each emotional state and completed rating scales (PHQ-9, GAD-7, SWLS, HILS, and PAQ). Phase 2 (N=232) involved a separate group of participants who read the narratives from Phase 1 and provided their own five-word summaries and completed the same rating scales. The descriptive words were quantified using Latent Semantic Analysis (LSA), creating high-dimensional vector representations of words. Multinomial logistic regression was used to classify emotional states based on word responses and rating scale scores, both individually and in combination. A 10% nested cross-validation leave-out procedure was used to evaluate the models. To increase training data size, data from a related study was also included. Alexithymia levels were determined based on PAQ scores using a median split. Statistical analyses included t-tests, Pearson correlations, and multiple linear regression. Test-retest reliability was also examined by comparing correlations between Phase 1 and Phase 2 ratings and word-based estimations. Word clouds were generated to visually represent the words most associated with each emotional state. The study used data from a large sample of US adult participants recruited via Prolific, with appropriate exclusions made for those who did not meet inclusion criteria or failed control questions.

The study found that language-based responses, as analyzed by LSA and classified using multinomial logistic regression, were significantly more accurate (62%) in classifying emotional states than rating scales (33%). The baseline accuracy for guessing was 25%. There was no significant difference between using only word responses versus combining word responses and rating scales. Analysis of the confusion matrices showed that rating scales had more classification errors than the word-based measures. Further analyses revealed that the test-retest correlation between Phase 1 and Phase 2 estimates was substantially higher for language-based responses compared to rating scales. While narratives from participants with high alexithymia scores were more difficult to classify, the alexithymia level of the evaluators did not significantly affect classification accuracy. The mean rating scale scores differed significantly between low and high alexithymia groups, but this difference was not observed for the word-based estimates. Word clouds provided visual representations of words associated with each emotional state, offering further insights into the language used to describe these states. The correlation between estimated PAQ from words and the PAQ rating scale was r = 0.18 (P<0.001). The correlation between estimated PHQ-9 scores from word responses and PHQ-9 scores was r = 0.62 (P<0.001).

The results demonstrate the superiority of language-based computational methods over rating scales in classifying emotional states, with a substantial effect size. This finding advances previous research, which showed reasonably high correlations between language-based and rating scale methods, but did not demonstrate superior accuracy of language-based classifications. The higher accuracy and reliability of language-based measures offer considerable advantages over traditional methods. Language is the natural mode of emotional expression, making language-based assessments more ecologically valid and potentially improving patient rapport. Further strengthening this advantage is the potential for language-based methods to be employed simultaneously for both assessment and treatment, such as in expressive writing therapy. The word cloud visualizations offer a novel way to represent and understand the semantic space of different emotional states. The finding that alexithymia did not influence classification accuracy suggests that the computational approach is robust to this individual difference factor. The difficulty in classifying narratives from high-alexithymia individuals likely reflects challenges in generating, rather than evaluating, emotional language.

This study demonstrated the superior accuracy of computationally analyzed language-based responses compared to rating scales for classifying emotional states, even considering alexithymia. While alexithymia did impact the ease of classification, it did not influence the accuracy of the language-based assessment. Future research should investigate the applicability of this method in clinical settings, its generalizability across diverse populations, and its potential use in assessing other psychological constructs and disorders. Investigating the use of more sophisticated natural language processing models, such as BERT, could further enhance the accuracy and scope of language-based assessments.

The study used a convenience sample of participants recruited online, limiting the generalizability of findings. The reliance on self-reported alexithymia scores introduces potential biases. The use of a single rater for the rating scales in Phase 2 prevented the assessment of inter-rater reliability. The study did not involve clinicians, limiting generalizability to clinical settings and the inclusion of a broader range of emotional experiences. The methodology might not capture the full range of symptoms included in clinical diagnoses like MDD and GAD. The study was conducted online, leaving less possibility to monitor and control how the participants conducted the task.