Manipulating image luminance to improve eye gaze and verbal behavior in autistic children

The study addresses how sensory-level visual processing traits in autism—particularly a bias toward local details (local interference)—affect global scene understanding and social communication. Building on theories such as weak central coherence, enhanced perceptual functioning, and differences in spatial frequency use, the authors propose that manipulating low-level image characteristics may support global processing. They introduce a think-see-say paradigm to measure both visual attention (eye fixations) and semantic representation (verbal descriptions). The key research questions are whether manipulating luminance and spatial frequency increases fixations in global Areas of Interest (hot spots) and whether these manipulations increase the likelihood of global verbal responses to the prompt “What is this picture about?”. The overarching purpose is to evaluate a digital filtering approach as an assistive technology to augment access to global information in natural scenes for autistic children, potentially improving social communication.

Prior work has documented atypical global processing in autism and proposed mechanisms including local interference, weak central coherence, enhanced perceptual functioning, and differences in spatial frequency processing. Studies indicate that autistic individuals may attend more to low-level salience and pixel-level features, while neurotypical viewers focus on semantic regions. Visual attention differences are linked to social processing challenges, including atypical scanning of faces and social scenes. Spatial frequency content of natural images differs from artificial stimuli, and global scene gist is formed rapidly in early viewing. Earlier research has also shown that luminance and spatial frequency interact in visual perception and that initial fixations can provide insight into cognitive processes. Despite this extensive literature, few interventions directly target sensory-level features to support global processing, motivating the present assistive filtering approach guided by neurotypical gaze heatmaps from an open-source dataset (OSIE).

Design: A 2×2 within-subject factorial design (condition: baseline vs filtered; session: two sessions on consecutive days) with counterbalanced presentation of 50 images (25 per session). The think-see-say paradigm presented each image for 3 s, followed by a 7 s prompt screen reading “What is this picture about?”. Participants completed three practice trials before testing. Participants: Eleven school-aged participants (9–19 years) receiving specialized speech and language services; 10 had an autism diagnosis (P3 did not). All had documented pragmatic/semantic language challenges (CELF-4 Pragmatics Profile ratings from mild to severe). Consent and assent procedures were followed. Stimuli and filtering intervention: Images and corresponding neurotypical eye-gaze heatmaps were sourced from an open repository (OSIE; heatmaps from 20 NT adults over 3 s viewing). The assistive filter desaturated and blurred non-relevant regions based on NT heatmaps to lower contrast and spatial frequency in cold spots, effectively highlighting global Areas of Interest (AOIs). Luminance (lightness) was computed by converting RGB to HLS and using the lightness channel. Spatial frequency was quantified as overall image activity; filtered images had lower spatial frequency than baseline. Apparatus and eye tracking: A head-mounted eye tracker (positivescience.com) was used in naturalistic settings. Calibration used five points and took ~2 minutes per session. Children viewed a 27-inch monitor at ~28 inches distance. An SLP sat beside each participant to maintain engagement. Measures:

• Eye gaze: AOI hits (hot spots) were scored from 3 s clips by trained assistants reviewing videos at 0.25× speed; a hit occurred if the gaze path passed through any predefined hot spot. Interobserver agreement for gaze scoring was 82%.
• Verbal responses: Responses to “What is this picture about?” were scored on a 0–2 rubric: 0 = incorrect/unrelated; 1 = local/irrelevant detail; 2 = plausible global description. The rubric was developed by SLPs and achieved 87% inter-rater reliability. Of 550 potential baseline/filtered pairs (11 participants × 50 images), 75 pairs were unusable due to missing responses, leaving 475 scored pairs. Image characteristics coding: Semantic content was categorized (initially multiple categories, then reliably reduced to living vs nonliving). AOI size was quantified as pixel proportion of hot spots from grayscale heatmaps. Luminance statistics were computed separately for hot and cold spots. Spatial frequency measures were computed overall and for AOIs. Human factors included age, gender, and CELF-4 pragmatic severity. Analysis: Generalized linear regression models assessed predictors of two outcomes: (1) fixation in hot spots (AOI hit) and (2) global verbal score. Predictor sets included human factors (age, pragmatic severity), study factors (condition, session order, image order), and image factors (semantic content, AOI size, luminance in cold/hot spots, spatial frequency). Interaction analyses tested luminance × spatial frequency effects on fixation rates.

• Participant factors: Age significantly increased the likelihood of fixations in hot spots (p = 4.12e-08) and global verbal responses (p = 0.01). Greater pragmatic language ability (lower severity) strongly predicted higher global verbal scores (p ≈ 9.91e-11).
• Study factors: Baseline (unfiltered) condition was associated with a higher likelihood of fixations in hot spots (p = 0.02). Filtered condition improved the likelihood of global verbal responses (p = 0.005). Trials in the second session were more likely to yield fixations in hot spots regardless of condition (p < .001); no significant session effect on global verbal behavior (p ≈ 0.19).
• Semantic content: Images containing living entities increased the likelihood of global verbal responses (reported p = 6.10e-07), with no similar effect on fixations reported in narrative results.
• Luminance (cold spots/background): Lower average luminance (darker) in cold spots increased AOI fixations (p = 0.007). Higher average luminance (lighter) in cold spots increased the likelihood of global verbal responses (p = 0.003), indicating opposite directions for gaze vs verbal outcomes.
• Spatial frequency: Main effects were not significant for fixations or verbal responses; however, luminance × spatial frequency interactions were significant for fixation likelihood, with the most effective condition being light hot spots with lower spatial frequency. Overall, the filter shifted semantic responding toward global descriptions, while fixations were more frequent in baseline but were modulated by luminance contrasts, particularly darker backgrounds enhancing AOI hits.

Findings support the premise that manipulating low-level visual properties can influence both attention and semantic processing in autistic children. Age and pragmatic language ability relate to better global processing, aligning with developmental expectations. The filter, designed to de-emphasize nonrelevant regions by lowering luminance/contrast and spatial frequency, facilitated global verbal descriptions, suggesting that offloading sensory complexity can free cognitive resources for gist extraction and communication. However, baseline images elicited more AOI fixations, and darker cold spots increased AOI hits, whereas lighter cold spots favored global verbal responses—highlighting a dissociation between gaze and verbal outcomes. Session order effects (higher fixation rates in session two) suggest time- or familiarity-related changes in gaze behavior, emphasizing the importance of multi-session designs. Semantic content influenced verbal but not gaze outcomes, reinforcing the rationale to target sensory-level features rather than only semantic content to promote attention guidance. The luminance × spatial frequency interaction indicates that contrast and texture complexity shape fixation behavior; designing filters that balance background dimming with maintaining low-frequency, light AOIs may optimize both gaze allocation and semantic output. Implications include integrating gaze-prediction and dynamic filtering into assistive technologies to scaffold global processing in real time.

This study demonstrates the feasibility of an assistive digital filter that manipulates luminance and spatial frequency to support global processing in autistic children. Empirical results show that sensory-level adjustments can increase the likelihood of global verbal responses and modulate fixation patterns toward Areas of Interest, especially under conditions of darker backgrounds and optimized luminance–frequency contrasts. Contributions include (1) quantifying how luminance in non-salient regions differentially affects gaze and semantic responses, (2) showing that participant age and pragmatic ability are robust predictors of global processing outcomes, and (3) presenting a practical, ecologically valid pipeline leveraging neurotypical gaze heatmaps to guide filtering of natural scenes. Future research should refine filter parameters to reconcile the opposing luminance effects on gaze versus verbal responses, test dynamic and personalized filtering in real time and 3D contexts, examine longer-term stability of effects across sessions, and explore applicability to other neurodiverse populations with global processing challenges.

• Language prompt ambiguity: The question “What is this picture about?” may have been challenging for some autistic participants, potentially affecting verbal scores, especially in younger children.
• Session/order effects: Significant differences across sessions suggest time- or exposure-related influences; the two-session design avoided within-session repetition but introduced order effects.
• Eye-tracker constraints: Head-mounted tracking improved ecological validity but required manual scoring due to alignment limitations; eye-gaze scoring interobserver agreement was 82%.
• Missing/variable data: Some participants did not wear the eye tracker in certain sessions, leading to missing gaze data; 75 response pairs were unusable due to no verbal response.
• Small, heterogeneous sample: Eleven participants with varying language profiles; no standardized IQ measures; generalizability is limited.
• Stimulus scope: Only 50 images from one dataset; chroma was excluded due to high correlation with luminance; spatial frequency measures may be influenced by luminance, complicating interpretation of main effects.
• Coding inconsistencies risk: Differences between narrative and tabulated statistics for some variables (e.g., semantic content) indicate potential coding or model specification nuances that warrant replication.