Neural dynamics of phoneme sequences reveal position-invariant code for content and order

Speech comprehension involves mapping variable acoustic signals onto discrete linguistic representations. While seemingly effortless, the underlying computational processes remain a significant challenge. Existing models primarily explain word recognition in isolation, with empirical support from neural encoding of phonetic features and interactions between phonetic and (sub)lexical units. However, how sequences of acoustic-phonetic signals are assembled during naturalistic speech comprehension to retrieve lexical items is poorly understood. Parsing auditory input into phoneme sequences is computationally difficult due to the lack of reliable cues for unit boundaries, acoustic blending from co-articulation, and the possibility of the same phonemes forming different words. This study aims to address this gap by investigating how the brain represents and processes sequences of phonemes in continuous speech, bridging the gap between sensory input and sub-lexical units.

The existing literature supports neural encoding of phonetic features and interactions between phonetic and (sub)lexical units of representation. However, there's a lack of understanding regarding how sequences of acoustic-phonetic signals are assembled during the comprehension of continuous speech to retrieve lexical items. Previous research has examined the recognition of words in isolation, with some success in predicting neural encoding patterns of phonetic features. However, these models don’t adequately address the challenges of processing sequences of phonemes in continuous speech, where co-articulation and the lack of clear boundaries between units complicate the process. This study fills this knowledge gap by examining how the brain handles these challenges in the context of natural speech comprehension.

The researchers recorded two-hour magnetoencephalograms (MEGs) from 21 native English speakers while they listened to four fictional narratives. The speech stimuli were synthesized using Mac OSX text-to-speech, employing three different voices to introduce variability. Participants answered comprehension questions to maintain attentiveness. MEG data were preprocessed using the Continuously Adjusted Least Squares Method (CALM) for noise reduction. A temporal receptive field (TRF) model was used to regress out responses sensitive to pitch and envelope fluctuations in the acoustic speech signal. The data was then bandpass-filtered, downsampled, and segmented into epochs time-locked to phoneme onset. Auditory signals were processed to create mel spectrograms. The researchers employed a back-to-back (B2B) ridge regression model to decode fourteen binary phonetic features (voicing, manner, and place of articulation), along with nuisance variables (stress, frequency, location, etc.). The B2B model consisted of a decoding step and an encoding step to address the issue of correlated features, providing a more accurate measure of decoding performance. Temporal generalization (TG) analysis assessed the stability and dynamics of neural representations over time. The proportion of variance explained was calculated to quantify effect sizes. Statistical significance was determined using permutation-based cluster tests. Finally, simulations were used to reconstruct phoneme sequences from the MEG data, helping to understand the capacity of the brain to process multiple phonemes simultaneously.

The study revealed that the brain simultaneously processes at least three phonemes, maintaining their phonetic representations for approximately 300 ms, far exceeding the duration of sensory input. Each phoneme's neural representation evolves systematically over time, encoding both its phonetic features and the time elapsed since its onset. This dynamic encoding prevents interference between consecutive sounds. The representations are position-invariant, meaning that the same neural pattern encodes a specific phoneme regardless of its position within a word. The speed of this evolution adapts to phoneme duration, ensuring efficient processing even with varying speech rates. Furthermore, processing initiation is earlier for more predictable phonemes, and the representation is sustained longer when lexical identity is uncertain. MEG topographies showed that phonetic features remain localized within auditory regions, while phoneme position information exhibits a posterior-anterior gradient across time. The dynamic encoding scheme is not a trivial reflection of the acoustic input but results from active brain processes.

The findings address the research question by demonstrating a dynamic, position-invariant encoding scheme for phoneme sequences. The brain's ability to simultaneously represent multiple phonemes, while preventing representational overlap, is a crucial computational mechanism. This joint content-temporal coding supports the existence of a 'sliding triphone' representation as an intermediary stage between phonetic features and lexical access. The flexible timing of phonetic processing, modulated by predictability and lexical uncertainty, highlights the continuous interaction between different levels of linguistic processing. These findings are consistent with predictive coding and analysis-by-synthesis models. The spatial dynamics observed suggest local changes within auditory cortices rather than strict anatomical transpositions to higher-level areas.

This study provides novel insights into the neural mechanisms of speech processing, demonstrating a dynamic encoding scheme that simultaneously represents the content and order of phoneme sequences. This position-invariant representation, modulated by predictability and lexical uncertainty, efficiently handles the challenges of continuous speech comprehension. Future research could investigate the link between representational overlap and comprehension errors, explore the specific role of working memory, and use higher-resolution neuroimaging techniques to refine the spatial understanding of these processes.

The study's limitations include the relatively low signal-to-noise ratio of single-trial MEG data, which limited the ability to analyze specific speech features within specific contexts and made the spatial claims less specific. The passive listening paradigm prevented direct association of decoding performance with behavioral measures. Future studies could address these by increasing the number of trials, using higher-resolution techniques (e.g., electrocorticography), and incorporating behavioral tasks.