This paper introduces a contrastive learning model to decode speech perception from non-invasive MEG and EEG recordings. Using four public datasets (175 volunteers), the model achieves up to 41% accuracy in identifying the corresponding speech segment from 3 seconds of MEG signals, reaching 80% in the best participants. This performance enables decoding words and phrases unseen during training, highlighting the importance of contrastive learning, pretrained speech representations, and a multi-participant convolutional architecture. Analysis suggests the decoder relies on lexical and contextual semantic representations, offering a promising non-invasive approach for language decoding.
Publisher
Nature Machine Intelligence
Published On
Oct 05, 2023
Authors
Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, Jean-Rémi King
Tags
contrastive learning
speech perception
MEG
EEG
language decoding
non-invasive
Related Publications
Explore these studies to deepen your understanding of the subject.