Automatic Detection of Reactions to Music via Earable Sensing

This paper introduces GrooveMeter, a system for automatically detecting vocal and motion reactions to music using earable sensing. It leverages smart earbuds' inertial measurement unit (IMU) and microphone to detect reactions like singing along, humming, whistling, and head movements. A new dataset, MusicReactionSet (926 minutes of data from 30 participants), was collected to train and evaluate GrooveMeter. The system employs sophisticated processing pipelines to filter out irrelevant data, improve robustness, and reduce energy overhead. Evaluation shows high accuracy (macro F1 scores of 0.89 for vocal and 0.81 for motion reactions) and robustness compared to alternative methods. Potential applications include automatic music rating, reaction-based recommendations, and tools for music therapy.

CHULHONG MINEuihyeok Lee, JAESEUNG LEE, JIN YU, SEUNGWOO KANG