Developing cognitive workload and performance evaluation models using functional brain network analysis

The study addresses the need for objective, evidence-based evaluation of cognitive workload and performance. Cognitive workload reflects the mental effort required by tasks and is influenced by working memory and long-term memory interactions. Older adults often exhibit higher cognitive workload than younger adults when performing the same tasks, potentially indicating inefficient brain resource use and early cognitive impairment. Current assessments are largely subjective, and EEG-based objective methods have been hindered by issues like volume conduction. This work aims to identify brain areas whose function is responsive to cognitive workload and performance using high-density EEG with spatial filtering to mitigate volume conduction, and to develop models that objectively evaluate cognitive workload and performance during two simulator-based cognitive tasks. The overarching goal is to lay groundwork for tools that could help detect cognitive decline by monitoring cognitive workload changes.

Prior work shows mixed evidence on whether cognitive activities (e.g., video games) improve executive functions in aging. Several studies report benefits of mental exercises in older adults, including increased hippocampal gray matter after 3-D gaming and improvements in attention, memory, and multitasking. Conversely, large-scale studies found no generalized cognitive benefits from brain-training programs, even with long-term engagement. Given these contradictions and the reliance on subjective assessments in practice, objective measures—particularly using EEG—are sought. However, EEG studies face challenges like volume conduction affecting connectivity measures, motivating high-density recordings and spatial filtering. The study builds on limited literature linking network flexibility with cognitive state, learning, mood, and mental workload, and explores integrating functional brain network metrics with spectral features for workload/performance evaluation.

Design and participants: 26 participants (18 males, 8 females; mean age 35.5 ± 11.7 years) including premedical students, scientists, surgical residents/fellows, and surgeons. IRB-approved with verbal consent. Tasks: Two levels (2 and 3) of the da Vinci Skills Simulator Matchboard task, engaging attention, memory, executive functions, and visuospatial abilities. Level 2 involved retracting panel doors with one instrument and placing characters with another. Level 3 required coordinating three instruments to manipulate sliding and swinging doors and place characters, constituting a more challenging task. Outcomes: Performance score (0–100) generated by simulator from weighted metrics (time, economy of motion, collisions, excessive force, instruments out of view, workspace range, drops). Cognitive workload assessed post-task with SURG-TLX (six domains rated 1–20; summed total). EEG acquisition and preprocessing: 124-channel AntNeuro headset (Cz reference), 500 Hz sampling. Poor-quality channels (F8, POz, AF4, AF8, F6, FC3, M1, M2) removed, leaving 116 channels. Processing steps: common average reference; 60 Hz notch; band-pass 0.2–250 Hz (24 dB/octave); artifact correction via ASA (blind source separation, topographical PCA) with visual inspection; spatial Laplacian to mitigate volume conduction. Feature extraction: For each of 21 Brodmann areas (BAs) and four frequency bands (theta 4–8 Hz, alpha 8–12 Hz, beta 13–35 Hz, gamma 35–65 Hz), computed:

• Functional brain network features from coherence-based adjacency matrices: search information (bits required to traverse shortest paths), strength (average within-BA connection weight), temporal network flexibility (frequency of community assignment changes across 1-s windows via Louvain modularity and consensus partition), integration (probability of being in same community with nodes from other systems), recruitment (probability of being in same community with nodes from same BA). Module allegiance matrices derived from partition matrices used for integration/recruitment. Each feature averaged within BA to yield 84 features per type (21 BAs × 4 bands).
• Power Spectral Density (PSD): Short FFT with 1-s Kaiser window and 50% overlap, band powers averaged within BA (84 features). Statistical analysis: Pearson correlations among age, performance, and workload. Three modeling approaches:
• Approach A: Forward feature selection with seven-fold cross-validation; predictors selected at least twice entered final linear random-intercept models. Šidák correction applied. Reported Efron’s pseudo-R², MAE, RMSE. Initially excluded PSD from selection; also built PSD-only models (Supplementary).
• Approach B: Aggregated features by cortex (frontal, parietal, occipital, temporal) across feature types and bands to form 96 features. Outliers removed via Local Outlier Factor. Used GLMM-LASSO for variable selection and estimation; tuning lambda via BIC and cross-validation.
• Approach C: Reduced-density analysis using 32 channels per 10–20 system. Extracted same feature set (96 cortical features), applied LOF outlier removal and GLMM-LASSO. All tests two-sided at alpha=0.05; analyses conducted in SAS 9.4.

Approach A (random-intercept models at BA level):

• Matchboard level 2 performance: Average temporal network flexibility in BA 45 (beta) positively associated (Estimate 0.16, SE 0.06, p=0.007); subject random effect p=0.002; pseudo R²=0.72; MAE=7.36; RMSE=9.36.
• Matchboard level 2 cognitive workload: Average search information in BA 47 (theta) negatively associated (Estimate -0.39, SE 0.11, p=0.001); random effect p=0.001; pseudo R²=0.95; MAE=2.34; RMSE=3.31.
• Matchboard level 3 performance: Positive association with average temporal flexibility in BA 9 (theta) (Estimate 0.41, SE 0.15, p<0.001); negative associations with recruitment in BA 47 (theta) (Estimate -0.28, SE 0.10, p<0.001) and search information in BA 37 (gamma) (Estimate -1.29, SE 0.34, p<0.001). Random effect p=0.01; pseudo R²=0.55; MAE=9.43; RMSE=11.61.
• Matchboard level 3 cognitive workload: Positive association with average search information in BA 45 (gamma) (Estimate 0.51, SE 0.16, p=0.002) and temporal flexibility in BA 7 (beta) (Estimate 0.29, SE 0.09, p=0.002); negative association with temporal flexibility in BA 44 (gamma) (Estimate -0.36, SE 0.11, p=0.001). Random effect p=0.001; pseudo R²=0.88; MAE=4.23; RMSE=6.18.
• PSD-only (Approach A; Supplementary): Level 2: BA 45 beta PSD associated with performance; BA 40 alpha PSD with workload. Level 3: BA 20 beta PSD with performance; BA 45 beta PSD with workload. Overall R²: Level 2 performance 0.74, workload 0.96; Level 3 performance 0.46, workload 0.85. Correlations:
• Performance vs workload: non-significant at Level 2 (r=-0.13, p=0.56) and Level 3 (r=-0.17, p=0.44).
• Age vs performance: Level 2 r=-0.40, p=0.06; Level 3 r=-0.14, p=0.5.
• Age vs cognitive workload: Level 2 r=0.38, p=0.08; Level 3 r=0.54, p=0.01. Age not selected as a significant predictor in models. Approach B (GLMM-LASSO, cortical features):
• Level 2 performance: Average strength in parietal cortex (theta) negatively associated (Estimate -34.49, p=0.014); N=82; R²=0.67; MAE=5.61; RMSE=7.23.
• Level 2 cognitive workload: Significant predictors included temporal network flexibility in occipital (alpha) (1.44, p=0.034) and parietal (theta) (-2.19, p=0.004), and frontal beta PSD (-3.33, p<0.001). N=62; R²=0.97; MAE=1.81; RMSE=2.29.
• Level 3 performance: Search information in frontal (beta -17.93, p=0.031; gamma -19.23, p=0.021; alpha 29.09, p=0.035) and temporal (beta 22.7, p=0.022) cortices; PSD in occipital gamma (9.24, p=0.029) and parietal gamma (-11.81, p=0.049). N=111; R²=0.54; MAE=8.57; RMSE=10.48.
• Level 3 cognitive workload: Temporal network flexibility in frontal (theta) (1.99, p=0.024), strength in temporal (beta) (-10.13, p=0.001), frontal beta PSD (8.44, p=0.002). N=83; R²=0.95; MAE=3.44; RMSE=4.18. Approach C (32-channel subset, GLMM-LASSO):
• Level 2 performance: Flexibility occipital (gamma) -4.14 (p=0.021), strength occipital (beta) 19.4 (p=0.027), strength temporal (gamma) -16.13 (p=0.016), PSD temporal (theta) 4.13 (p=0.048); N=84; R²=0.69; MAE=5.86; RMSE=7.30.
• Level 2 cognitive workload: Significant predictors across multiple cortices and bands including flexibility occipital (beta) 2.36 (p=0.008), search information parietal (beta) 6.41 (p=0.003), search information frontal (gamma) -5.06 (p=0.012), search information occipital (gamma) 3.68 (p=0.045), multiple strength terms (parietal alpha 25.73, p=0.029; occipital alpha -13.87, p=0.021; parietal theta -9.02, p=0.013; frontal theta -39.66, p=0.001; occipital theta 14.67, p=0.006), and PSD occipital alpha 23.71 (p<0.001). N=63; R²=0.94; MAE=1.82; RMSE=2.41.
• Level 3 performance: Flexibility frontal (beta) 5.57 (p=0.033), parietal (gamma) 8.31 (p=0.026), occipital (gamma) -6.79 (p=0.002); search information parietal (theta) 8.61 (p=0.025); strength temporal (gamma) -8.69 (p=0.038). N=118; R²=0.57; MAE=7.52; RMSE=9.34.
• Level 3 cognitive workload: Flexibility parietal (alpha) 2.67 (p=0.024); search information frontal (beta) 4.11 (p=0.001); strength occipital (alpha) -3.88 (p=0.025); PSD features: parietal alpha -12.27 (p=0.027), frontal alpha 32.17 (p=0.001), temporal alpha -13.13 (p=0.013), occipital beta -5.71 (p=0.014), frontal theta -22.65 (p=0.005). N=89; R²=0.96; MAE=3.05; RMSE=4.07. Integration of PSD and network features improved cognitive workload model performance compared to using either alone: workload R² increased from 0.95 to 0.97 (Level 2) and from 0.88 to 0.95 (Level 3) relative to network-only models, and exceeded PSD-only models (0.96 and 0.85). Reduced-density (32-channel) EEG still yielded robust models with relatively high R², though significant features differed from high-density analyses. Age correlated with workload only in the more challenging task (Level 3, r=0.54, p=0.01).

Findings show that EEG-derived features from functional brain networks and spectral power can objectively evaluate cognitive workload and, to a lesser extent, performance during simulator-based cognitive tasks. Network features such as temporal flexibility, search information, strength, integration, and recruitment captured dynamic inter-regional communication and were particularly informative for complex tasks (Matchboard level 3). PSD features complemented network metrics by indexing band-limited power changes associated with cognitive states. Combining both feature classes (Approach B) yielded the most accurate cognitive workload models, indicating complementary information about overall activity (PSD) and network dynamics (connectivity/topology). The lack of correlation between performance and workload suggests these constructs are distinct; however, the positive correlation between age and workload for the challenging task indicates that task complexity can reveal age-related increases in cognitive effort. Reduced EEG density (32 channels) still enabled accurate modeling, supporting feasibility in practical settings where high-density systems are impractical, though models required more features and differed in selected predictors. Overall, results support the utility of integrated EEG feature modeling for objective, task-level assessment of cognitive workload with potential relevance to monitoring cognitive change.

The study demonstrates that cognitive workload and performance can be evaluated using EEG-derived functional brain network features and PSD, with integrated models providing superior workload prediction, especially in complex tasks. Specific BA- and cortex-level features (e.g., temporal flexibility and search information) were robustly associated with outcomes, and age-related increases in workload emerged only for the more challenging task. Reduced-density EEG systems can still produce informative models, indicating translational potential. Future work should validate these models in larger, more diverse and older populations, across tasks of varying complexity, and explore longitudinal applications for early detection and monitoring of cognitive decline.

Small sample size (N=26) with a predominantly young, healthy, and well-educated cohort; only three participants were over 60 years old. Generalizability is limited; models need validation in older adults, individuals with cognitive impairments, and across different education levels and task complexities. SURG-TLX workload assessment relies on self-report, introducing subjectivity.