Feature-specific neural reactivation during episodic memory

The study addresses why episodic memories vary in vividness and how neural reactivation of specific visual feature levels underlies this variability. Prior work suggests memories are reconstructed from perceptual representations (neural reactivation) across the dorsal and ventral visual streams, and that executive/frontal systems modulate visual memory via top-down control. There is debate about whether frontal regions encode abstract task-level information or also contain stimulus-specific, feature-level representations. Deep CNNs capture hierarchical visual features corresponding to cortical hierarchies but inter-layer correlations complicate feature-specific attribution. The authors introduce FSIC to isolate feature-level reactivation during recall while accounting for correlations across feature levels. Hypotheses: (1) feature-specific reactivation (low-, mid-, high-level visual and semantic) will be widespread across posterior visual regions and frontal cortex and synchronized across regions; (2) vividness will correlate with reactivation of all feature levels, with a predicted stronger role for lower-level features; (3) recognition accuracy in an old/new task with semantically similar lures will depend primarily on low-level feature reactivation during recall.

Research on imagery, episodic and working memory shows overlap between perception and memory representations, including low-level (edges, orientation, luminance) and high-level semantic content, with reactivation linked to vividness. Visual memory depends on capacity limits and executive processes (selective attention, working memory), supported by top-down frontal influences on posterior visual cortex. The representational nature of frontal cortex remains debated: some evidence favors abstract/task-level coding; other work shows stimulus-specific codes and mixed selectivity in prefrontal regions. Early feature mapping approaches emphasized either low-level or categorical features, limiting coverage of the hierarchy. Hierarchical CNN features map broadly onto visual cortex and mirror cortical hierarchies, but strong inter-layer correlations can produce false positives for feature presence if not controlled. Previous ROI-layer assignment methods mitigate this but may miss weakly represented features by assuming one feature level per voxel.

Participants: 37 recruited; 27 right-handed adults (15 male, 12 female; age 20–32, mean 25) included after exclusions (motion, sleep, incomplete). Ethics approved; informed consent obtained. Stimuli: 111 natural images paired into 111 semantically similar pairs (222 total); 21 pairs for practice, 90 pairs in-scan. Each image had an audio title used as text cue during retrieval. Two short-clip videos (~10.5 min each) served as training stimuli; an additional Indiana Jones clip was shown but not used for modeling due to aspect ratio mismatch. Design/Procedure: In-scanner runs: two short-clip video runs (plus a third Indiana Jones video not used for modeling), then three encoding-retrieval sets. Encoding: 1-back task while memorizing 30 images per run (each image repeated 4 times; 120 trials/run). Retrieval: for each cued image title, participants performed 6 s imagery/recall within a bounding box, rated vividness (1–4), then viewed a probe (same or lure) for old/new judgment and rated confidence. MRI acquisition: 3T Siemens Trio, 32-channel head coil; multiband EPI (2×2×2 mm, 63 slices, TR=1.77 s, TE=30 ms, FA 62°); T1-weighted MPRAGE (1 mm isotropic). Visual presentation via projector and mirror; E-Prime 2.0 used. Preprocessing: AFNI for motion correction and coregistration; data projected to subject-specific cortical surfaces (Freesurfer 5.3), resampled to 32k vertices using MapIcosahedron. Retrieval runs segmented into trial-wise beta estimates for imagery (recall) and old/new (recognition) using HRF-convolved regressors per trial; encoding runs modeled with least-squares-sum (3dLSS) to obtain trial-wise betas; all mapped to surface. CNN features: VGG16 pretrained (TensorFlow). Feature levels: layer 2 (low-level visual), layer 7 (mid-level), layer 13 (high-level), layer 16 softmax (semantic probabilities). Convolutional feature maps downsampled to 3×3 to mitigate eye-movement-related spatial variability; log-transformed activations. Feature vector sizes: low 576, mid 2304, high 4608, semantic 1000. Encoding models: Vertex-wise models per subject and feature level using non-negative lasso (nnlasso), selecting top-100 features per vertex (by correlation) from training data (movie + encoding; 3-fold CV across images). Regularization tuned via path of lambda values; best by SSE on held-out encoding folds retained for decoding. Image decoding during recall: For each ROI (148 bilateral Destrieux ROIs reported; decoding summarized over 74 bilateral ROIs), trial, and feature level, predicted activation patterns for the 90 encoded images were correlated with observed recall activity; ranks of correct-image correlations transformed to centered rank (chance=0; positive>chance). Seed ROI selection: For FSIC seeds, ROIs showing relatively maximal decoding for a given feature level during perception (recognition probe) were weighted based on z-scored accuracy across ROIs, thresholded and normalized to emphasize feature-selective peaks. FSIC analysis: For each seed ROI and target ROI, trial-wise reactivation of a target feature level in the seed ROI was related to target ROI reactivation using linear mixed-effects partial regression, controlling for non-target feature levels in the target ROI. Subject and image were random intercepts. Significance assessed via bootstrap with FDR correction. FSIC was validated with simulations (200 simulated subjects) using VGG16-driven synthetic vertices across ROIs with noise and variable memory fidelity to show specificity to true feature levels and to estimate on- vs off-diagonal partial correlations. Statistics: LME models for decoding reactivation (ROI-wise), vividness and accuracy correlations (within-subject and between-subject), bootstrap CIs and p-values (FDR-corrected).

- Decoding of recalled images was significantly above chance across dorsal and ventral visual streams for all feature levels; significant decoding also appeared in lateral prefrontal cortex (notably inferior frontal sulcus). - Preserving coarse spatial structure in convolutional features improved decoding in early visual cortex during perception: calcarine sulcus decoding for low-level features was higher with 3×3 vs 1×1 features (mean rank 11.7 vs 10.1), t(26)=5.51, p<0.001. - FSIC isolated feature-specific reactivation across cortex, revealing widespread reactivation for low-, mid-, high-level visual and semantic features during recall. Mid-level features were primarily occipital; low-, high-, and semantic features extended into higher-order dorsal/ventral streams and frontal cortex. - Simulation validated FSIC specificity: on-diagonal partial regression coefficients (same feature level across regions) exceeded off-diagonal (different levels) [on-diagonal mean=0.075; off-diagonal mean=0.013; difference mean=0.063; 90% CI lower bound=0.062; p<0.001], indicating largely independent trial-by-trial variation across feature levels. - Frontal cortex exhibited significant FSIC for low-level features (e.g., middle frontal sulcus, superior/inferior precentral sulci, superior frontal gyrus, anterior cingulate and midcingulate, inferior frontal gyrus pars opercularis; FDR-corrected p-values ≤ 0.023; β values ~0.048–0.083). - Vividness: Within-subject, both lower-level (low+mid) and higher-level (high+semantic) reactivation within corresponding ROIs positively correlated with subjective vividness; the lower-level effect was not significantly greater than the higher-level effect [difference=0.005, p=0.423]. Higher-level reactivation within lower-level ROI negatively correlated with vividness (consistent with predictive coding constraints). - Recognition accuracy: No significant within-subject correlations across all participants. Between-subjects, recognition accuracy correlated positively with low-level reactivation in lower-level ROI and exceeded the correlation for higher-level features in higher-level ROI [difference=1.199, p=0.032]. - In the higher-performing half (n=13) on lure trials, within-subject low-level reactivation within corresponding ROI predicted accuracy [β=0.081, p=0.028], whereas higher-level did not [β=0.031, p=0.320], suggesting individual differences in reliance on low-level feature reinstatement. - Overall, results show widespread, feature-specific reactivation beyond early visual cortex, including frontal regions, challenging a strictly feed-forward hierarchical interpretation and supporting coordinated, multi-level reinstatement during recall.

Findings demonstrate that episodic recall reinstates multi-level visual features across extensive cortical territories, including higher-order regions and frontal cortex. FSIC disambiguated contributions of correlated CNN-derived feature levels, showing strong within-level synchrony and weak cross-level dependency of trial-by-trial reactivation. The presence of low-level feature representations in higher-order and frontal areas suggests pervasive feedback and cross-hierarchical integration, inconsistent with a strictly serial, feed-forward visual hierarchy. Inferior frontal and broader frontoparietal involvement likely reflects top-down guidance and maintenance of feature-specific representations in early visual regions during recall. Subjective vividness depends on both lower- and higher-level feature reinstatement, aligning with predictive coding accounts whereby higher-level predictions and lower-level details jointly shape the richness of memory; negative associations for higher-level reactivation in lower-level ROIs may reflect top-down inference of non-episode-specific details. Objective recognition with semantically similar lures benefited from low-level feature reinstatement, particularly among individuals who effectively leveraged such details, highlighting meaningful individual differences in strategy or capacity to reactivate fine-grained visual information.

The study introduces FSIC, a feature-level informational connectivity approach that controls inter-layer correlations without sacrificing sensitivity, enabling accurate mapping of feature-specific reactivation during episodic recall. Using FSIC and CNN-derived features, the authors show that low-, high-level visual and semantic features are reinstated throughout the cortical hierarchy, including frontal cortex, with mid-level features more confined to occipital regions. Lower- and higher-level feature reinstatement contribute equally to subjective vividness, supporting predictive coding mechanisms in recall. Low-level feature reactivation in early visual cortex is linked to recognition accuracy when fine-grained discrimination is required, especially among higher-performing individuals. Future research should probe the sources of individual differences in leveraging low-level reactivation, further test causal top-down influences (e.g., perturbation studies), and refine feature models to disentangle semantic and visual contributions across tasks and contexts.

Potential noise correlations across regions could mimic feature-specific FSIC effects; the authors addressed this by showing specificity to feature-congruent seeds (e.g., low-level seeds did not yield mid-level FSIC in frontal cortex). CNN semantic outputs may conflate high-level visual similarity with category semantics; FSIC controls for cross-level correlations but imperfect semantic mapping remains possible. Mid-level feature detection appeared weaker and more posterior, potentially reflecting sensitivity limits. Within-subject recognition effects were absent overall, indicating substantial individual differences in reliance on low-level reactivation; larger samples and targeted training may clarify these effects.