A comprehensive review of deep learning in EEG-based emotion recognition: classifications, trends, and practical implications

Emotion recognition based on EEG signals is increasingly important for human−computer interaction and broader applications such as mental health (PTSD, depression) and safety (fatigue-related driving). Compared with facial and speech cues that can be consciously manipulated, EEG offers greater objectivity and authenticity for emotion inference, driving research attention. The paper addresses the need for a precise, comprehensive classification of deep learning approaches in EEG-based emotion recognition and clarifies why distinct research directions require different modeling strategies. It proposes organizing the field into subject-independent (cross-individual) and subject-dependent (within-individual) approaches to provide clearer guidance on goals, methods, and applications.

The review synthesizes prior surveys covering EEG-based emotion recognition and related modalities: Roy et al. (2019) analyzed 154 deep learning EEG works (2010–2018), highlighting CNN dominance and use of raw/preprocessed time series; Suhaimi et al. (2020) reviewed 2016–2019 progress with emphasis on stimuli delivery (including VR), sample sizes, hardware, and ML techniques; Rahman et al. (2021) summarized links between EEG and emotions, extraction/selection/classification techniques, and future challenges; Li et al. (2022) provided a tutorial-style overview of psychological/physiological foundations, frameworks, and evaluation; Dadebayev et al. (2022) contrasted consumer vs research-grade EEG devices and algorithmic challenges; Jafari et al. (2023) focused on DL methods for EEG emotion recognition and potential SoC/FPGA/ASIC implementations; Khare et al. (2023) surveyed emotion recognition across questionnaires, physical cues, and physiological signals, noting trust and real-time issues; Prabowo et al. (2023) and Vempati & Sharma (2023) analyzed datasets, preprocessing, features, selection, classifiers, and AI/ML/DL methods. Collectively, prior reviews offered valuable perspectives but lacked a unified, precise classification tied to modeling rationales and practical application pathways.

The survey spans works published from 2018 to 2023 in English journals, conferences, and preprints sourced from Web of Science, IEEE Xplore, PubMed, and arXiv using keywords EEG, deep learning, emotion recognition, emotional classification, and emotional computation. Initial retrieval yielded 2,539 articles (Web of Science 1,891; IEEE Xplore 445; PubMed 78; arXiv 125). After screening titles, abstracts, and keywords, 305 articles remained; evaluation by validation datasets, innovativeness, research value, and performance led to 101 included articles. Stage 1 screening ensured use of mainstream public datasets (e.g., DEAP, SEED, SEED-IV; excluding self-collected/non-mainstream). Evaluation criteria: contribution to the field (filling gaps or proposing novel solutions), originality (novel methods or meaningful enhancements), and model efficacy (outperforming contemporaries or improved variants). Stage 2 categorized literature according to the proposed framework: subject-dependent vs subject-independent, including cross-session and cross-individual experiments. Final counts: subject-dependent 45, subject-independent 48, traditional machine learning 8.

- The paper introduces a classification framework separating subject-dependent (personalized, within-individual) and subject-independent (cross-individual) approaches, each with distinct goals: feature richness and accuracy vs generalization and robustness. - Datasets overview and comparison: DEAP (32 subjects, 40×1-min videos, 32 channels, 128 Hz), SEED (15 subjects, 3 sessions, 62 channels, 200 Hz, 4-min clips), SEED-IV (15 subjects, 24×2-min clips, 62 channels, 200 Hz), DENS (40 subjects, 11×1-min videos, 128 channels, 250 Hz). - Preprocessing techniques commonly used: regression, wavelet transforms, filtering (adaptive, Wiener, high-pass), PCA, ICA; feature extraction across time (HOC, Hjorth), space (CSP, HDCA), and frequency (FFT, PSD, DE in δ, θ, α, β, γ bands). - Evolution of deep learning methods in subject-dependent research: RNN/LSTM → CNN → Transformer, with recent gains from hybrid/fusion models. Reported accuracies on DEAP arousal/valence often exceed 95%, with transformer-based models reaching ≈99% (e.g., 99.10–99.40% valence/arousal), and strong SEED/SEED-IV results (often >90%). - Subject-independent research emphasizes domain adversarial and domain adaptation to bridge source-target discrepancies. Performance varies by dataset and method, with notable advances: e.g., cross-subject SEED accuracies frequently in 85–97% range; DEAP arousal/valence performance typically lower due to variability (often 56–80%, with some methods higher). Multi-source and attention-based strategies, graph models, and transformer hybrids show promise. - Trend insights: Transformer architectures and model fusion (CNN+Transformer, CNN+LSTM) are becoming predominant for capturing multi-domain EEG features (time, space, frequency) and long-range dependencies. - Practical implications: Established models can be adapted for downstream tasks such as psychological assessment and intervention (e.g., VR-assisted therapy), objective recommendation systems, and large-scale applications where subject-independent generalization is essential.

The classification framework clarifies modeling choices aligned with the intrinsic variability of EEG signals. Subject-dependent approaches benefit applications needing personalized emotion profiles and higher per-user accuracy, driven by comprehensive feature extraction across time, space, and frequency. Subject-independent approaches are essential for large-scale deployment but face domain shift challenges; domain adaptation/adversarial strategies, graph attention, and transformer-based hybrids improve generalization and robustness. Dataset constraints (coarse emotion categories, limited diversity) hinder fine-grained emotion modeling; leveraging VR to elicit richer emotions and collecting nuanced labels can improve downstream performance. Applying these models beyond pure recognition—to mental health diagnostics and therapy, recommendation systems, and HCI—will translate methodological advances into practical impact, provided explainability, trust, and real-time support are addressed.

The review maps recent EEG-based emotion recognition advances and proposes a systematic classification into subject-dependent and subject-independent streams, highlighting distinct objectives, methods, and applications. It surveys mainstream and emerging datasets, preprocessing and feature extraction practices, and the progression from RNN/LSTM through CNN to transformer architectures, with increasing emphasis on hybrid/fusion models. By delineating principles and priorities across directions, the framework aids researchers in selecting appropriate methodologies and datasets. The paper also discusses applications and outstanding challenges to guide future work toward more robust generalization, richer emotion taxonomies, and practical deployment in healthcare, recommendation, and HCI.

The review focuses on English-language publications from 2018–2023 and excludes self-collected and non-mainstream datasets, which may limit coverage of some recent or niche studies. Public datasets used for validation often provide relatively coarse emotion categories, constraining fine-grained analyses. Performance comparisons may be influenced by differences in preprocessing, experimental protocols, and evaluation splits across studies. Inclusion of preprints introduces works that may evolve post peer review.