Gaze behaviors during free viewing revealed differences in visual salience processing across four major psychiatric disorders: a mega-analysis study of 1012 individuals

The aberrant salience hypothesis of psychosis proposes that aberrant assignment of salience to elements of experience leads to delusions and hallucinations. Salience encompasses both emotional/motivational and perceptual components derived from novelty and sensory features. Imaging studies in schizophrenia show reduced signaling in midbrain regions with additional involvement depending on salience type (novelty, negative emotion, targetness, rarity/deviance). Different domains of salience likely share a computational framework despite distinct neural underpinnings. Prior gaze behavior research shows schizophrenia-associated abnormalities during free viewing of images or drawings: gaze trajectories are constrained to a narrower area than in healthy controls and scan paths are altered. These findings suggest altered visual salience may underlie gaze abnormalities in schizophrenia and possibly other psychiatric disorders, motivating the present cross-disorder investigation.

Design and participants: Mega-analysis across 7 COCORO sites including 1012 participants: 550 healthy controls (HC), 238 schizophrenia (SZ), 41 bipolar disorder (BD), 50 major depressive disorder (MDD), and 133 autism spectrum disorder (ASD). Some participants overlapped with prior eye-movement studies. Psychiatric assessments are detailed in supplementary materials. Task and stimuli: Participants viewed a 19-inch LCD monitor (1280×1024 pixels) at 70 cm. Each trial began with a fixation spot (random duration), followed by an 8 s presentation of a test image; 20 trials per participant with 20 different images in random order. Free-viewing instruction. Image categories: buildings, everyday scenes, food items, portrait images, and grayscale comic patterns (4 images each). Images without color contrast were excluded from color analyses. Eye movement recording and preprocessing: Left eye position and pupil area recorded at 1 kHz using Eyelink 1000 Plus. Eye position (in degrees) smoothed with a FIR filter (1–3 dB at 30 Hz); velocity and acceleration computed via two-point difference algorithm. Eye movements segmented into blinks, saccades, and microsaccades; saccades with amplitude >1° classified as regular saccades. Computational saliency modeling: Saliency maps computed for each image using the Itti–Koch model (via GBVS toolbox for MATLAB), a neurobiologically inspired model using low-level features. A full saliency map was defined as the sum (equal weights) of three low-level feature maps: luminance, color, and orientation. For each image, saliency maps were downsampled to 80×64 pixels and represented as density maps. For each participant and model (full and each feature), mean saliency values at gaze locations were computed across images to obtain a salience score, reflecting the mean saliency at the eyes during free viewing. Analyses conducted in MATLAB; code available on request. Statistical analysis: Within-institute linear regression models estimated effect sizes (unbiased d) for differences between HCs and SZ for full and feature-specific salience scores while controlling for covariates (e.g., age, sex). Effect sizes were meta-analyzed across institutes. General linear models assessed group (diagnosis) effects on salience scores across disorders (controlling for age, sex, task), with Bonferroni-adjusted post hoc pairwise comparisons. Power analysis described in Supplementary Methods. Correlations with clinical symptom scales (e.g., PANSS) and medication dosages (chlorpromazine equivalents for SZ; lithium for BD; antidepressant equivalents for MDD) were tested with Bonferroni correction for multiple comparisons.

- SZ patients exhibited the largest abnormalities in salience-guided gaze among diagnostic groups; salience at gaze was higher in SZ than in HCs, indicating stronger guidance by low-level image salience. - Full-model salience: Estimated marginal means differed significantly between SZ and HC (p = 1.94×10^(-12)), SZ and MDD (p = 2.38×10^(-7)), and SZ and ASD (p = 1.74×10^(-9)). - Feature-level effects: Significant group effects for color (p = 1.24×10^(-7)) and orientation (p = 2.51×10^(-32)) salience; luminance showed no significant group effect (p = 0.10). - Pairwise comparisons: • Color: SZ vs HC (p = 2.85×10^(-8)); SZ vs ASD (p = 5.73×10^(-10)). • Orientation: SZ differed from all other groups: HC (p = 6.21×10^(-3)), BD (p = 1.50×10^(-7)), MDD (p = 4.43×10^(-15)), ASD (p = 3.25×10^(-12)). - No significant associations (after Bonferroni correction) between salience scores and symptom scales or medication dosages. - Cross-disorder pattern: Highest salience scores in SZ, followed by BD, then MDD, and lowest alterations in ASD, paralleling the empirical gradient of psychosis frequency across these disorders. - Sample and paradigm details: 1012 participants; 20 images per participant; 8 s free viewing per image; salience modeled via Itti–Koch with luminance, color, orientation features.

Findings support the utility of free-viewing salience analysis for detecting psychiatric and neurological disorders. Using a biologically plausible saliency model (Itti–Koch), the study replicated and extended prior single-site results: SZ participants preferentially fixated locations with higher bottom-up salience than HCs, with the strongest effects in orientation-based salience and additional, smaller effects in color (luminance largely unaffected). These results are consistent with prior evidence of altered orientation processing and contextual modulation in early visual cortex in SZ, aligning with the aberrant salience hypothesis of psychosis. Cross-disorder comparisons showed a gradient of alteration (SZ > BD > MDD > ASD) that mirrors known gradients in psychosis prevalence and some neuroimaging findings across disorders. Discrepancies with prior ASD studies may reflect differences in participant selection (e.g., high-functioning ASD) and computational models (simple bottom-up versus models incorporating semantic/object-based salience). The orientation salience differences may aid in discriminating SZ from other psychiatric conditions, though further validation is needed.

The study provides strong, robust evidence for elevated visual salience during free viewing in schizophrenia, particularly for orientation features, with additional smaller effects for color and minimal effects for luminance. Cross-disorder analyses indicate a graded alteration pattern (SZ > BD > MDD > ASD), supporting the aberrant salience framework in psychosis. These results suggest that salience-guided gaze metrics could aid differential characterization of psychiatric disorders. Future work should validate diagnostic utility, incorporate longitudinal and interventional designs, and integrate cross-diagnostic symptom measures and more comprehensive salience models.

- Generalizability: Predominantly adult sample; applicability to children and older adults is unknown. - Medication effects: Many patients were medicated; disease versus treatment contributions cannot be fully disentangled despite no significant correlations with dosage. Drug-free or drug-naive samples and longitudinal/interventional studies are needed. - Symptom measures: Symptom data were diagnosis-specific (e.g., PANSS for SZ, HAMD for MDD); cross-diagnostic rating approaches were not applied and should be employed in future studies. - Model scope: Salience modeling focused on bottom-up features; models incorporating semantic/object-based salience may reveal additional disorder-specific effects.