Decoding speech perception from non-invasive brain recordings

This paper introduces a contrastive learning model to decode speech perception from non-invasive MEG and EEG recordings. Using four public datasets (175 volunteers), the model achieves up to 41% accuracy in identifying the corresponding speech segment from 3 seconds of MEG signals, reaching 80% in the best participants. This performance enables decoding words and phrases unseen during training, highlighting the importance of contrastive learning, pretrained speech representations, and a multi-participant convolutional architecture. Analysis suggests the decoder relies on lexical and contextual semantic representations, offering a promising non-invasive approach for language decoding.

Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, Jean-Rémi King