Recent advancements in natural language processing (NLP) using deep learning have yielded algorithms capable of generating, summarizing, translating, and classifying texts. However, these models still lag behind human capabilities. The study posits that this discrepancy stems from a fundamental difference in how language is processed: while current NLP models are optimized for predicting the next word in a sequence, the human brain might employ a more sophisticated predictive coding framework that operates across multiple timescales and levels of representation. Predictive coding theory suggests that the brain continuously predicts sensory inputs, compares these predictions to reality, and updates its internal model based on the prediction errors. This process is hypothesized to be hierarchical, with higher-level brain regions generating more abstract and long-range predictions based on information from lower-level regions. This research directly investigates this theory, aiming to establish a concrete link between the computational mechanisms of the brain and the limitations of current NLP approaches. The importance of understanding this discrepancy lies in creating more human-like AI systems and further elucidating the cognitive processes underlying human language comprehension.

Prior research has shown a correlation between the activations of deep language models and human brain responses to speech and text, particularly emphasizing the models' ability to predict future words. However, limitations remain: current models struggle with long-form text generation, summarization, coherent dialogue, and accurate semantic interpretation. They often fail to capture complex syntactic structures and demonstrate a superficial understanding of linguistic nuances. While previous studies have demonstrated evidence of predictive processing in the brain through correlations with measures like word or phoneme surprisal (the unexpectedness of a word or sound), these measures are typically derived from models that only predict the immediately next element, thus neglecting the potential for multi-timescale, hierarchical predictions. This study builds on this existing work by investigating the brain's predictive representations more thoroughly, considering both the temporal scope and hierarchical levels of these predictions.

The study employed fMRI to record the brain activity of 304 participants while they listened to short stories from the Narratives dataset. The researchers compared these brain responses with the activations of several state-of-the-art deep language models, primarily focusing on GPT-2, a causal language model. To quantify the similarity between brain activity and model activations, they used a linear ridge regression to predict fMRI signals from the model activations and calculated a 'brain score' based on the correlation between predicted and actual fMRI signals. A key innovation was the introduction of 'forecast windows,' which involved concatenating the model's activations of the current word with the activations of future words at varying distances. This allowed them to assess the impact of long-range predictions on the brain score. The 'forecast score' quantifies the improvement in brain score when incorporating these forecast representations. Further analyses explored the hierarchical organization of predictions by varying the layer of GPT-2 used to generate activations and examining the optimal depth for predicting brain activity at various distances. To disentangle syntactic and semantic components of prediction, the authors employed a method to generate synthetic future words with the same syntax but varied semantics, enabling the calculation of distinct syntactic and semantic forecast scores. Finally, the study involved fine-tuning GPT-2 with a combined language modeling and high-level long-range prediction objective to determine whether this enhancement would improve its correspondence to brain activity.

The study yielded several significant findings: First, it confirmed the existing observation that deep language models linearly map onto brain activity, with activations peaking in areas associated with language processing, including the auditory cortex and superior temporal areas. Second, incorporating long-range predictions (specifically, a forecast window of approximately 3 seconds) significantly enhanced the brain-model alignment. The improvement was particularly pronounced in regions associated with language processing and was, on average, approximately 23% higher compared to models that only predicted the next word. Third, the study revealed a hierarchical organization of predictions across the cortex: frontoparietal cortices demonstrated longer forecast distances than temporal cortices, suggesting that higher-level brain regions make more distant and abstract predictions. Fourth, the optimal depth of GPT-2 layers used for prediction also varied hierarchically, with higher-level brain areas better modeled by deeper, more contextualized representations. Fifth, decomposing predictions into syntactic and semantic components showed that semantic predictions drive long-range forecasts (about 3 seconds), while syntactic predictions showed a shorter range (approximately 2.5 seconds). Finally, fine-tuning GPT-2 with both language modeling and a long-range, high-level objective resulted in improved brain mapping, particularly in frontoparietal regions.

These findings strongly support the hypothesis of a hierarchical predictive coding architecture for language processing in the human brain. The results show that the brain's processing of language involves predicting multiple levels of representations over different temporal scales. The observed hierarchical organization, with frontoparietal areas making longer-range, higher-level predictions, corroborates the predictive coding framework. The difference between semantic and syntactic predictions further enhances this understanding. The enhanced brain mapping achieved by fine-tuning GPT-2 with a multi-timescale objective highlights the importance of this computationally sophisticated approach. This study provides a crucial bridge between neuroscience and artificial intelligence, suggesting a need for developing AI models that predict multiple levels of representations over various timescales to better emulate human cognitive capabilities.

This research offers compelling evidence for a hierarchical predictive coding model of human language processing, showing that the brain predicts language at multiple levels and timescales, unlike current NLP models. The study’s findings emphasize the need for future research to develop AI models incorporating long-range, high-level predictions to better mimic human-like language processing. Future studies might explore more sophisticated predictive coding architectures, investigate the precise nature of the predicted representations in different brain regions, and evaluate the efficacy of these models on various NLP benchmarks.

The study primarily relies on fMRI, which has a limited temporal resolution (around 1.5 seconds), potentially hindering the investigation of sublexical predictions. The interpretation of neural representations remains a challenge, and further research is needed to precisely characterize the predictions made by each region in the cortical hierarchy. The predictive coding architecture tested is rudimentary and requires further generalization, scaling, and evaluation to fully demonstrate its practical utility.