Multimodal assessment of communicative-pragmatic features in schizophrenia: a machine learning approach

Pragmatics concerns using language (and other expressive means) to convey intended meanings in context. Schizophrenia is associated with significant language and communication impairments that persist even with preserved syntax and semantics. Patients show pervasive pragmatic difficulties, especially with non-literal language (e.g., indirect speech acts, irony, metaphors, idioms, proverbs), recognizing and repairing communicative failures, detecting deceit, and recognizing violations of Gricean maxims. Deficits extend to non-verbal/extralinguistic behaviors (gestures, facial expression) and paralinguistic cues (prosody), with atypical voice patterns and impaired recognition of emotional and linguistic prosody. Despite this, few studies have comprehensively assessed multiple communicative phenomena across modalities within the same sample to determine which features best discriminate schizophrenia from health. This study aims to identify the most informative pragmatic phenomena and modalities (linguistic, extralinguistic, paralinguistic) distinguishing patients with schizophrenia from healthy controls using a multimodal battery and a machine learning Decision Tree approach.

Prior work shows broad pragmatic impairments in schizophrenia across non-literal language, conversation, and narrative abilities. Multimodal studies report deficits across verbal, non-verbal, and paralinguistic domains, with patients performing worse than controls on conversational abilities and engagement-related non-verbal aspects. Comprehensive assessments have revealed poorer performance in comprehension/production of sincere, deceitful, and ironic acts in both linguistic and gestural modalities. Meta-analyses highlight large patient-control differences in indirect speech act comprehension, deceit, irony, and detection of Gricean maxim violations. Emotional prosody recognition shows large effects, with more modest but notable differences in quantitative acoustic voice measures. Gesture perception/production is less meta-analytically established but empirical studies indicate impairments. While the communicative disorder is recognized as core in schizophrenia, it remains unclear which specific phenomena/modalities most effectively discriminate patients from controls. Based on prior evidence, the authors hypothesized irony, Gricean maxim violations, and prosodic recognition as prime discriminators, with exploratory evaluation of gestural phenomena given limited prior meta-analytic data.

Design: Cross-sectional comparison of patients with schizophrenia and healthy controls using a multimodal pragmatic assessment and Decision Tree (DT) classification to identify discriminative features. Participants: 32 individuals with schizophrenia (7 females; mean age 40.17 years, SD 10.19; mean education 10.59 years, SD 2.45) and 35 healthy controls (6 females; mean age 39.46, SD 10.95; mean education 10.57, SD 2.46), matched on gender, age, and education. All patients met DSM criteria for schizophrenia, were clinically stable (chronic stage), Italian native speakers, and met cognitive/linguistic cutoffs (MMSE ≥24/30; Token Test ≥5/6; AAT denomination: no deficit). Exclusions: neurological disorders, head injury, substance abuse, impaired hearing/vision. Ethics approval obtained; informed consent provided. Assessment: The Assessment Battery for Communication (ABaCo) was administered. It comprises five scales assessing comprehension and production of pragmatic phenomena across modalities: (1) Linguistic scale (basic speech acts; sincere direct/indirect acts; deceit; irony via verbal modality); (2) Extralinguistic scale (same acts via gestures/facial expressions); (3) Paralinguistic scale (basic acts via paralinguistic indicators; basic emotions; paralinguistic contradiction); (4) Context scale (Gricean maxims comprehension; social appropriateness); (5) Conversational scale (topic maintenance; turn-taking). The battery includes 72 live interaction items and 100 brief video clips (~20–25 s each); administration ~90 minutes. Items scored 1 (correct) or 0 (incorrect) per ABaCo manual rules. Machine learning analysis: A Decision Tree classifier (J48, implementation of C4.5) in Weka v3.8.3 was used. The tree selects features at each node via information gain to best separate classes (schizophrenia vs controls). Model evaluation used 10-fold cross-validation with metrics: Accuracy, Sensitivity, Precision, Specificity, and AUC. Additional analyses on symptom/medication prediction in patients were conducted (reported in Supplementary Information). Classifier parameters: confidence factor 0.5; minNumObj 2.

- The DT identified four main discriminators: linguistic irony, Gricean maxims of linguistic communication, extralinguistic deceit, and extralinguistic sincere (direct and indirect) communicative acts. - Tree structure and thresholds: • Root node: Linguistic irony. If score ≤0.5, classify as schizophrenia with probability 94.1%; if >0.5, proceed to Gricean maxims. • Gricean maxims (linguistic): if ≤0.5, evaluate extralinguistic deceit; if >0.5, evaluate extralinguistic sincere acts. • Extralinguistic deceit: if ≤0.8, classify as schizophrenia (92.1%); if >0.8, classify as control (75.1%). • Extralinguistic sincere acts: if ≤0.6, classify as schizophrenia (100%); if >0.6, classify as control (85.1%). - Overall model performance (10-fold cross-validation): Accuracy 0.821 (SD 0.118); Sensitivity 0.758 (SD 0.285); Precision 0.910 (SD 0.151); Specificity 0.900 (SD 0.175); AUC 0.894 (SD 0.143). - In the test folds summarized in discussion: Sensitivity of 76% corresponded to 24 patients correctly classified; Precision of 91% implied 4 false positives.

Applying a multimodal pragmatic assessment with a DT classifier showed good discriminatory performance between schizophrenia and healthy controls. The most informative feature was linguistic irony, aligning with robust literature indicating severe irony comprehension deficits in schizophrenia and the high cognitive demands of irony (theory of mind, executive function, inferential processing). The second feature was recognition of violations of Gricean maxims, consistent with documented difficulties adhering to conversational norms (quantity, quality, relation, manner) and their links to symptomatology and ToM. Beyond linguistic tasks, extralinguistic (gestural) comprehension/production of deceit and sincere acts also contributed significantly, underscoring that non-verbal communication deficits are a salient aspect of schizophrenia and relevant for classification. These findings support prioritizing assessment and rehabilitation that target both high-level linguistic pragmatics (e.g., irony, conversational norms) and extralinguistic communicative skills, reflecting the multimodal nature of pragmatic impairment.

This study provides an integrated, multimodal profile of communicative-pragmatic impairments in schizophrenia and identifies the features that best distinguish patients from controls using a Decision Tree approach. Linguistic irony is the strongest discriminator, followed by violations of Gricean maxims, with extralinguistic deceit and sincere acts also key contributors. These insights can guide clinicians in diagnostic assessment and the design of focused rehabilitative interventions, and they highlight pragmatic features that may be relevant for early identification of psychosis risk. Future research should validate these discriminative features in larger, diverse samples, across different pragmatic tasks and contexts, and explore longitudinal and neurocognitive correlates to refine targeted interventions.

- Exploratory analysis with a relatively small sample (n=67), limiting generalizability and out-of-sample validation typical for ML approaches. - Clinical heterogeneity in schizophrenia; results need replication across broader and varied clinical profiles. - Pragmatic abilities can be measured by diverse tasks with varying cognitive/inferential loads; discriminative features identified here require confirmation across alternative batteries and ecological contexts.