Stimulus representation in human frontal cortex supports flexible control in working memory

The study addresses how stimulus information is represented and maintained across cortical regions during working memory and how these representations adapt to varying control demands. Prior work shows robust stimulus-specific encoding in early visual cortex (EVC) during simple maintenance, with weaker and less stable representations in frontoparietal regions; however, frontal stimulus representations can strengthen under attentional prioritization, categorization, or extensive training. The authors propose that differences in stimulus representation reflect flexible coding strategies that adapt to task goals and control demands, with a tradeoff between representing the original stimulus and newly generated information (e.g., categories) under limited cognitive resources. They predict that frontal stimulus representations will increase with greater demands for active control (e.g., rule-based categorization), while visual cortical stimulus representations will be enhanced for precise maintenance. They further predict a dynamic tradeoff between stimulus and category representations within regions involved in encoding the newly generated information.

Human neuroimaging has demonstrated stimulus-specific WM representations across sensory, parietal, and frontal cortices. EVC robustly encodes simple visual features during maintenance (supporting sensorimotor recruitment), but is vulnerable to distraction and task demands that de-emphasize visual details. Frontal cortex stimulus representations have been emphasized in non-human primate neurophysiology and can be robust under attentional prioritization, categorization, or after training. The field has debated the functions of stimulus representations across regions, with evidence suggesting task-dependent variability and susceptibility to interference in visual cortex. Theoretical frameworks from cognitive flexibility propose complementary coding strategies: low-dimensional abstraction for generalization and high-dimensional stimulus representations for flexible readout. The present work situates WM representations within this flexibility framework, predicting region-specific roles and tradeoffs between stimulus and category codes under varying control demands.

Participants: Two fMRI experiments each included n=24 healthy adults (Experiment 1: flexible rule switching; Experiment 2: fixed single rule). All participants completed behavioral training to learn novel orientation-based categorization rules prior to scanning. Stimuli and Tasks: Trials presented two oriented bars (0.75 s each) followed by a retro-cue, then a task cue indicating either maintenance (maintain cued orientation precisely) or categorization (categorize the cued orientation per current rule) during an 8 s delay, then a response. In Experiment 1, blocks randomly alternated between two orthogonal categorization rules; maintenance and categorization trials were interleaved. In Experiment 2, a single fixed categorization rule was used; maintenance and categorization trials were still interleaved. Response mapping was randomized across trials to minimize motor-planning confounds. MRI Acquisition: 3T Siemens scanners (Tim Trio; later participants in Exp 2 on Prisma) with multiband EPI (TR=1000 ms, TE=30 ms, FA=40°, voxel 3×3×3 mm, multiband 4, 60 slices) and T1 anatomical scans. Preprocessing included motion correction, registration to anatomy, and detrending (AFNI). ROIs: Anatomical masks from probabilistic atlas (Wang et al., 2015) for EVC (V1–V3), IPS (IPS0–5), and sPCS (FEF), refined by functional GLM selection (top 500 voxels during sample for EVC; during delay for IPS and sPCS). Additional frontal ROIs (iPCS, IFS, MFG) from HCP atlas were explored; M1 used for control analyses. Analyses: Population-level stimulus reconstructions with inverted encoding models (IEM) across timepoints using leave-one-run-out cross-validation; reconstruction fidelity quantified via vector projections of channel responses. Multivariate pattern analysis (MVPA) used linear SVMs to decode stimulus bins and category labels, including an opposite-rule decoder to compute an abstract category index (true minus opposite). Behavioral relevance assessed via correlations between representational fidelity and trial-wise accuracy across time. Representational similarity analysis (RSA) and linear mixed-effects modeling (LMEM) on neural representational dissimilarity matrices (RDMs) to disentangle graded stimulus, discrete stimulus, and abstract category contributions. Statistics used sign-flip permutation tests with FDR correction; defined early (5–10 s) and late (11–16 s) epochs for comparisons. Computational Modeling: Multi-module recurrent neural networks (RNNs) with three hierarchical modules (posterior/EVC-equivalent; middle/IPS-equivalent; anterior/sPCS-equivalent) and short-term synaptic plasticity, trained to perform both maintenance and categorization tasks. Two output designs: RNN1 with choice-only outputs; RNN2 with additional orientation-tuned outputs enforcing stimulus preservation. Networks trained to 90% accuracy; population decoding and RSA compared model modules to human ROIs; flexible-rule and fixed-rule variants simulated.

Behavior: Participants performed maintenance and categorization with comparable accuracy (Maintenance: 0.81±0.07; Categorization: 0.82±0.05; t(23)=1.51, p=0.144) and showed classic boundary effects in categorization (better performance farther from category boundaries). Stimulus Representation (IEM): In EVC, orientation representation was significant in both tasks from the sample period, with stronger fidelity for maintenance during the delay; late-epoch difference significant (Figure 2D; EVC: p<0.00001). In IPS, both tasks showed significant orientation representation with minimal differences (IPS: p=0.063). In sPCS, categorization showed persistent orientation representation through delay and response, exceeding maintenance in late delay (sPCS: p=0.007). Control analyses (condition-specific IEM, SVM decoding, mean activation removal, voxel selection parity, M1 control) supported robustness. Behavioral Correlation: EVC fidelity in early delay predicted accuracy in both tasks; IPS fidelity predicted accuracy in early delay for maintenance and across delay for categorization; sPCS fidelity predicted accuracy exclusively in categorization across the delay, not in maintenance. Reduced Control Demand (Experiment 2): With a fixed rule, EVC and IPS patterns mirrored Experiment 1. sPCS showed enhanced stimulus representation in categorization emerging earlier (early epoch 5–10 s: p=0.015) but not in late epoch (11–16 s: p=0.372). Mixed ANOVA revealed an Experiment×Epoch interaction (F(1,46)=7.43, p=0.009), indicating temporal shift of frontal enhancement under reduced control demands. Category and Abstract Category: Category decoding during late epoch was significant across ROIs in both experiments (ps<0.044). Abstract category index (true minus opposite-rule decoding) was significant in Experiment 2 across ROIs (ps<0.017) but not in Experiment 1 (ps>0.14), with higher abstract category in sPCS for Experiment 2 vs. 1 (p=0.034), indicating a tradeoff between stimulus and category representations tied to control demands. Additional frontal ROIs (iPCS, IFS, MFG) generally showed weaker effects; MFG exhibited partial patterns in Experiment 2. RNN Modeling: RNN1 (choice-only output) showed stronger stimulus decoding in maintenance than categorization in middle and anterior modules (p_middle=0.011; p_anterior=0.007), opposite to human sPCS/IPS trends. RNN2 (stimulus-preserving output) reproduced human-like frontal enhancement with increased stimulus decoding in categorization in the anterior module (p_anterior=0.026). Fixed-rule RNNs showed increased abstract category decoding relative to flexible-rule RNNs in both architectures (RNN2: p_posterior=0.045, p_middle=0.003, p_anterior<0.001; RNN1: all ps<0.001). RSA indicated higher similarity of RNN2 to human data, especially in the anterior/sPCS-equivalent module, for both stimulus- and category-aligned RDMs.

Findings demonstrate that stimulus representations in human WM are flexibly distributed and reconfigured across cortical regions according to task demands: EVC supports precise maintenance of sensory details, while sPCS in frontal cortex enhances stimulus representation under active control demands such as rule-based categorization. The behavioral relevance of sPCS fidelity selectively in categorization highlights its role in control-related operations rather than passive maintenance. A dynamic tradeoff was observed between high-dimensional stimulus codes and low-dimensional category codes, modulated by control demand (flexible vs. fixed rules), consistent with theories of cognitive flexibility and resource allocation. Computational RNN models replicate key neural patterns only when the network is required to preserve stimulus information for readout, suggesting that frontal stimulus representations may arise from global coding strategies that facilitate flexible behavior. Together, results reconcile debates on the roles of visual vs. frontal cortices in WM by delineating their complementary functions: precision maintenance in EVC and flexible control in frontal cortex.

The study reveals that stimulus representations in the visual and frontal cortices serve distinct but complementary functions in WM: EVC supports precise maintenance of visual features, whereas frontal sPCS supports flexible control demands, with stimulus representation increasing during categorization and predicting behavior. Varying control demands modulate a tradeoff between stimulus and category codes, and multi-module RNNs reproduce human neural patterns when stimulus information is preserved at the output. These findings offer a unified framework for WM in dynamic environments and suggest future directions to test generalization across stimuli and frontal regions, probe meta-control mechanisms selecting coding strategies, and refine models to capture finer-grained visual representations and broader control tasks.

- The alignment between RNN and fMRI was not complete: all RNN modules showed stronger abstract category decoding than fMRI, and the EVC-like module in RNN2 did not reproduce the reduced stimulus representation during categorization seen in human EVC (IEM analyses suggested only a weak negative trend). - Effects in additional frontal ROIs (iPCS, IFS, MFG) were weaker than sPCS, limiting generalization across frontal cortex. - Tasks focused on orientation stimuli; generalization to other feature domains (e.g., color) or other control demands (e.g., mental rotation) remains to be tested. - Sample sizes, while adequate and typical for fMRI, were not determined a priori. - The study relies on fMRI BOLD signals and decoding models; causal mechanisms and single-neuron dynamics were not directly measured.