Forming cognitive maps for abstract spaces: the roles of the human hippocampus and orbitofrontal cortex

The study addresses how the human brain constructs and uses cognitive maps to support flexible behavior in non-physical, multidimensional abstract spaces. Building on Tolman’s cognitive map concept, prior work implicates the medial temporal lobe—particularly the hippocampal–entorhinal (HIP–EC) system—and the orbitofrontal cortex (OFC) in representing task and state spaces for navigation and inference. The HIP–EC system is linked to spatial and relational memory, mental representations of spatial layouts, object representations, and replay for consolidation, with distinct activity patterns across learning stages. The OFC has been proposed to encode task-state cognitive maps to support planning and inference when states are partially observable, integrating inputs from sensory cortices, hippocampus, and striatum. However, how HIP/EC and OFC collaborate during learning, exploration, and exploitation in abstract spaces remains unclear. The authors aim to determine whether HIP, EC, and OFC form internal representations during exploration and exploitation while navigating 1D, 2D, and 3D abstract spaces, and whether brain activation and representational patterns vary with learning level and behavioral performance.

Prior studies suggest the HIP–EC system provides core mechanisms for cognitive map construction, supporting spatial reasoning, relational memory, and updating object representations; hippocampal place cells encode locations and replay spatiotemporal sequences. Distinct hippocampal activation patterns emerge with familiarity and task structure learning. The OFC is implicated in encoding cognitive maps of task/state space for planning and inference, with its diverse functions (value prediction, credit assignment, response inhibition, emotion) potentially arising from such representations. OFC receives multimodal sensory, hippocampal, and striatal inputs, supporting integrative associative representations. Evidence indicates HIP and OFC encode different aspects of cognitive maps, but their collaboration in flexible behavior in abstract spaces is not fully characterized.

Participants: 27 healthy adults (14 women; mean age 21.78 years, range 18–29) with normal/corrected vision participated; two were excluded for poor/insufficient task performance, yielding N=25 for analysis. IRB approval: SCNU (#2019-3-062); written informed consent obtained. Design and stimuli: Two base symbols (hat, dog) each with four features (hat: tilt angle, brim width, pompom size, color lightness; dog: tail direction, body length, leg length, color tone). Dimensions corresponded to manipulated features. For each subject, features were randomly selected to build five abstract spaces: S1P (1D primary symbol), S1C (1D control symbol), S2P (2D primary), S2C (2D control), S3P (3D primary). Locations in spaces are discrete positions in 1–3 dimensions. Half the subjects used hat as primary; half used dog. Tasks ordered fixed: S1P, S2C, S2P, S1C, S3P. Set 1 denotes first per dimension (S1P, S2C, S3P); Set 2 the second per dimension (S1C, S2P). Procedure: Day 1 included demographics and cognitive assessments (not analyzed) and training. Day 2 included refresher training and MRI scanning. Within each space, subjects completed 10 navigation paths (start to destination). Trial structure: display current location (2 s), then goal (4 s), then option array appears; responses via 4-button box with spatial mapping to options. If the chosen option equals the goal, feedback (“Achieved!” plus coin) ends the path; otherwise, the chosen option becomes the new current location; inter-trial interval 3–9 s. Paths could be probabilistically terminated (20% chance) if optimal options were not chosen; terminated paths were included in analyses; inter-path fixation 6–13 s. For each step, recorded response accuracy (RA; 1 for optimal option, else 0) and response time (RT). Each task scan had a maximum duration (1D: 10 min/400 volumes; 2D/3D: 15 min/600 volumes); two subjects failed to complete some tasks and were excluded from analyses. MRI acquisition: Siemens Prisma 3T, 64-channel head/neck coil. Task-fMRI: single-shot SMS/MB gradient-echo EPI: TR=1500 ms, TE=31.0 ms, flip angle 70°, MB factor 3, FOV=211×211 mm², matrix 88×88, 60 interleaved slices, voxel size 2.4 mm isotropic, A>P phase encoding, bandwidth 2186 Hz/px. Field map: double-echo GRE (TR=620 ms, TE1/TE2=4.92/7.38 ms, flip=60°, 2.4 mm isotropic). Structural: T1-weighted 3D MP-RAGE (TR=1800 ms, TE=2.07 ms, flip=9°, voxel 0.8 mm isotropic), plus T2-SPACE and HARDI (not analyzed). Session included localizer, rs-fMRI, field map, five task-fMRI runs, structural scans, HARDI. Preprocessing: fMRIPrep 21.0.0 (Nipype 1.6.1). Steps: reference volume and skull-strip; motion estimation (FSL mcflirt) with six motion parameters retained as nuisance; field map registration to functional reference; slice-time correction to 50% of acquisition range (AFNI 3dTshift); BBR co-registration to T1; framewise displacement (FD) computed; exclusion threshold mean FD>0.25 (no exclusions); normalization, resampling to MNI space 2.0 mm isotropic; high-pass filtering (cutoff 1/100 Hz). Smoothing 6 mm FWHM for univariate GLMs; no smoothing for RSA. Behavioral analyses: Linear mixed-effects models (lmerTest) assessed performance improvements and transfer: LMM1 compared early (first 3 paths) vs late (last 3) learning phases on RA_path and RT_path; LMM2 compared Set 1 vs Set 2 to assess transfer across spaces of same dimensionality; LMM3 compared exploration vs exploitation stages determined via DNN+k-means (see below). Fixed effects: phase or set or stage; random effects: subject-by-dimension. Significance via paired t-tests on fixed effects. Learning level estimation and stage categorization: For each path, constructed B matrix of stepwise RA and RT (N_step×2). A subject-specific DNN (TensorFlow 2.8) with input layer, two hidden layers (64 and 32 units, ReLU), and a 2-unit softmax output estimated probability that a path was early vs late phase. Training used labeled early and late phase paths; per dimensionality, two paths reserved for testing; optimization via Adam (lr=0.001), categorical cross-entropy loss, 100 epochs; prediction accuracy averaged across epochs. The softmax probability of “early” served as the path’s learning level (P(early)). K-means (k=2; optimized initialization) clustered paths by learning level into exploration (higher P(early)) and exploitation (lower P(early)) per subject and dimension. Univariate fMRI GLMs: GLM1 modeled exploration paths, exploitation paths, and feedback (plus six motion regressors); contrasts exploration vs baseline and exploitation vs baseline; within-subject fixed effects combined across runs, group random-effects compared exploration vs exploitation. GLM2 estimated path-wise activation (10 path regressors + feedback + motion); path COPEs were regressed against learning level to identify learning-associated activation; also tested modulation by dimensionality. GLM3 included navigation, feedback, and a parametric modulator for step-level RA during navigation; effects averaged across runs; group random-effects tested association with RA. Multiple comparisons controlled via GRF correction (cluster-forming p<0.001, cluster-level p<0.05). Representational similarity analysis (RSA): Whole-brain voxel-wise searchlight RSA per stage. Theoretical RDMs computed from Euclidean distances between destinations across paths. Neural RDMs computed as 1−r (Pearson correlation) between path-specific parameter estimate maps (from unsmoothed GLM2). Spearman’s rho between theoretical and neural RDMs yielded a ρ-map per stage; Fisher-transformed ρ-maps compared between stages (exploitation > exploration) using paired t-test with TFCE, 10,000 permutations, FWE-corrected.

Behavioral: - Performance improved with learning. LMM1: RA_path was significantly lower in early than late learning (t = −3.89, p < 0.001), indicating higher accuracy later; RT_path difference not significant (t = 1.76, p = 0.079). - Transfer learning across spaces of the same dimensionality. LMM2: RA_path was significantly lower in Set 1 than Set 2 (t = −4.86, p < 0.001), indicating better accuracy in later-presented spaces; RT_path not significantly different (t = −0.37, p = 0.711). - DNN learning-level estimation was above chance (t = 41.74, p < 0.001), with no significant differences across dimensionalities (F(2,48) = 8.68, p = 0.134). K-means assigned ~50% of paths to exploration (50.32%) and ~50% to exploitation (49.68%), with no stage-count differences by dimensionality. - Exploration vs exploitation validation (LMM3): RA_path was lower in exploration than exploitation (t = −5.63, p < 0.001); RT_path was longer in exploration (t = 2.06, p = 0.040). Univariate fMRI (GLM1–3): - Exploration > exploitation: stronger activation in bilateral hippocampus (largest cluster; peak MNI ~ (28, −66, −16)), bilateral inferior frontal gyrus, bilateral insula, bilateral medial frontal gyrus, bilateral inferior parietal lobule, left middle frontal gyrus, right ventral ACC, right thalamus, and bilateral cerebellum. - Exploitation > exploration: increased activation in right posterior cingulate cortex and right superior frontal gyrus; broader discussion and convergent analyses highlight medial OFC and retrosplenial cortex engagement during exploitation. - Learning level (GLM2): Positive associations in right hippocampus, bilateral middle/superior/medial frontal gyri, bilateral insula, left postcentral gyrus, right IPL, bilateral cerebellum; negative associations in left medial frontal gyrus and right PCC. - Response accuracy (GLM3): Positive associations in left hippocampus, bilateral middle frontal gyrus, bilateral inferior frontal gyrus, right lingual gyrus, right thalamus. Negative associations in left ACC, bilateral IPL, right PCC, left retrosplenial cortex, bilateral MTG, right STG, right parahippocampal gyrus. Representational similarity (RSA): - Exploitation showed more accurate destination representations than exploration in bilateral entorhinal cortex, bilateral orbitofrontal cortex, left hippocampus, right ACC, left frontal pole, bilateral SFG, left IFG, bilateral cuneus/visual cortex, left ITG, and right cerebellum (TFCE, FWE-corrected). Overall: Hippocampus and lateral PFC are more engaged during exploration and positively track accuracy and learning level. OFC and retrosplenial cortex are more engaged during exploitation and negatively track accuracy, with OFC/EC/HIP encoding more precise destination representations after learning.

The findings demonstrate that constructing cognitive maps in multidimensional abstract spaces involves distinct neural systems during exploration and exploitation. Exploration engaged a network including hippocampus, lateral prefrontal cortices, insula, thalamus, parietal, and visual cortices, consistent with roles in collecting information, relational/spatial memory, and updating representations. Exploitation engaged medial prefrontal/orbitofrontal and retrosplenial regions, consistent with integrating sensory evidence with learned task-state representations, evaluating outcomes, anchoring internal maps to current context, and guiding decisions. Learning-level associations and accuracy correlations converged with these stage effects: hippocampal and visual/PFC activations were positively associated with accuracy and lower learning levels (exploration), while OFC and retrosplenial activations were negatively associated with accuracy and higher learning levels (exploitation). RSA showed that entorhinal cortex, hippocampus, and OFC encoded destinations more precisely in exploitation, suggesting that once an internal map is established, these regions represent goal locations in a more structured manner to support inference and efficient navigation. Collectively, results support complementary roles: hippocampus/entorhinal cortex in building and updating map-like representations, and OFC (with mPFC/RSC) in utilizing these maps for decision-making in abstract spaces.

This study shows that humans refine internal representations of abstract spaces through experience, with distinct neural systems supporting exploration versus exploitation. The hippocampus, lateral PFC, and visual cortex exhibit stronger activation during exploration and positively relate to accuracy and learning level, while OFC and retrosplenial cortex are more active during exploitation and negatively relate to accuracy. RSA reveals more accurate destination representations in entorhinal cortex, hippocampus, and OFC during exploitation, indicating consolidated cognitive maps guiding decisions. These findings elucidate how medial temporal and prefrontal regions collaborate to form and use cognitive maps for multidimensional abstract tasks. Future work could test more naturalistic or complex abstract spaces, manipulate reward/uncertainty to probe exploration–exploitation trade-offs, examine causal roles via neuromodulation or patient studies, and optimize fMRI protocols for MTL/OFC signal to enhance measurement sensitivity.

- Ecological validity: The abstract spaces were artificially constructed and may not capture real-world conceptual complexity; participants might consider dimensions separately rather than integrating them. - Imaging sensitivity: BOLD measurements in hippocampus, entorhinal cortex, and orbitofrontal cortex are sensitive to acquisition parameters (hardware, multiband acceleration, slice orientation, susceptibility, and resolution), potentially affecting detection and localization of effects.