Detecting Users’ Emotional States during Passive Social Media Use

The widespread use of social media significantly impacts users' emotions, with negative affect often prevalent and harmful to mental health. Detecting emotions during passive social media use—the dominant engagement mode—is challenging due to sparse behavioral traces. This work introduces the first predictive model that estimates user emotions during passive social media consumption alone. In a study with 29 participants using a controlled, Instagram-like feed built from standardized affective image databases, the apparatus passively captured smartphone interaction and physiological signals while participants provided self-reports using two validated emotion models (Russell’s Circumplex Model of Affect and Mikel’s Wheel). Using supervised training, the emotion classifier robustly detected up to eight emotional states, achieving a peak accuracy of 83% for affect classification. Analyses show behavioral features alone were sufficient to robustly recognize emotions and that objective features reveal a participant’s new emotional state within eight seconds following a change in content. Grounding labels in a componential emotion model (Mikel’s Wheel) outperformed dimensional models in higher-resolution state detection. Additionally, using deep-learning-predicted emotional properties of images further improved emotion recognition.