Temporally organized representations of reward and risk in the human brain

Adaptive behavior requires evaluating both expected reward and uncertainty. While reward expectations guide actions, real-world unpredictability necessitates tracking higher-order variables such as expected risk. The temporal dynamics of how the brain computes and represents variables including expected value (EV), reward prediction error (RePE), expected risk (E.Risk), and risk prediction error (RIPE) remain poorly characterized due to limitations of noninvasive neuroimaging. Building on normative formulations in which RIPE is a second-order uncertainty term derived as a function of RePE, the study tests two hypotheses: (1) neural representations of RePE should precede those of RIPE, and (2) representations of RePE should directly relate to, and potentially inform, RIPE computations. The authors also hypothesize specific regional involvement: vmPFC and amygdala for reward value, anterior/ventromedial PFC and insula/OFC for risk-related computations, and potential shared or distinct representational codes within regions such as OFC and anterior insula. Using iEEG during a within-trial card-guessing task designed to temporally separate computations, the study aims to map the spatiotemporal organization of reward and risk representations across widespread cortical and subcortical areas and to test interactions between computations.

Prior work shows that humans and animals guide actions by reward expectations and track higher-order uncertainty. RPEs quantify discrepancies between outcomes and expectations and can inform measures of outcome variability (risk). It has been proposed that RIPEs are computed when predictive associations change, serving as second-order uncertainty terms derived from RPE. Scalp EEG and MEG studies suggest mixtures of parallel and sequential computations, including posterior-to-anterior flows for reward-related signals, but are limited in spatial resolution and depth access. fMRI meta-analyses implicate vmPFC, OFC, insula, amygdala, and parietal regions in value and risk. iEEG studies have identified reward- and risk-related signals in anterior insula and OFC, but direct temporal comparisons and interactions between RePE and RIPE, particularly in anterior insula and OFC, remain unclear. There is mixed evidence on whether OFC encodes expected value and risk simultaneously or sequentially. This study leverages iEEG to address these gaps and directly test temporal ordering and cross-variable representational relationships (e.g., whether RePE representations precede and predict RIPE).

Participants: Ten adult patients (7 female; 22–56 years) undergoing clinical intracranial monitoring for refractory epilepsy completed 16 sessions total (1–2 sessions/participant; 90 trials/session; 1440 trials). Electrode implantation (depth and/or subdural grids) was based on clinical needs. Recordings were acquired at 2 kHz (0.1–500 Hz bandpass). Data collection occurred at least 1 h from any seizure activity. IRB approval obtained; written consent provided. Task: On each trial, participants guessed whether the second of two sequentially drawn cards (from 10-card deck, ace=1, no replacement within trial; reshuffled between trials) would be higher/lower than the first. Timeline per trial: Guess display (1000 ms), fixation (1500 ms), Card 1 on (2000 ms), inter-stimulus period with only guess shown (1500 ms), Card 2 on (2000 ms), then report whether they had won (correct guess) or lost. Points: ±10 for correct/incorrect guess; ±5 for correct/incorrect report (outcomes only shown during instructions/training, not during analyzed task). Left/right response mappings randomized per trial. Computational variables: The normative model defined EV after Card 1 onset as the expected probability of a correct outcome given Card 1 and the guess; Outcome (OUT) after Card 2; Expected Risk (E.Risk) as expected variance of (OUT−EV)² computed after Card 1 offset (before Card 2); RePE = OUT − EV; RIPE = RePE² − E.Risk (at Card 2). Variables were temporally aligned to specific trial events to support within-trial separation. iEEG preprocessing: Data trimmed to task periods; downsampled to 500 Hz; notch filtered at 60 Hz; high-pass (1 Hz) then ICA-based denoising using KL-divergence criteria to remove global/local noise components; low-pass at 250 Hz. Bipolar re-referencing of adjacent contacts to enhance spatial specificity; white-matter pairs excluded. Data epoched −200 to +500 ms around event onsets (guess, Card1-on, Card1-off, Card2-on, report) with baseline correction. ROIs and coverage: A priori ROIs: Frontal Pole, OFC (Frontal Orbital Cortex), vmPFC (Subcallosal Cortex), anterior insula, amygdala (Harvard–Oxford atlas). Exploratory ROIs: posterior insula, putamen, cingulate gyrus, hippocampus, angular gyrus, supramarginal gyrus. Contacts localized via CT–MRI co-registration to MNI space; bilateral contacts pooled for main analyses. Decoding approach: A pseudo-population design pooled contacts across participants within ROIs. For each contact, a GLM regressed out covariates and other computational variables (including orthogonalization to handle collinearity: RePE w.r.t OUT; RIPE w.r.t observed risk O.Risk), plus per-trial report accuracy; residuals used as features. Event-locked epochs for five events (251 timepoints/event) concatenated; logistic regression with L2 regularization used to classify binarized target variables at each timepoint. Ten-fold cross-validation on event dimension; ROC AUC as accuracy metric. Nonparametric max-cluster permutation tests (1000 label permutations; 95th percentile thresholds) with FWE control determined significant temporal clusters. Exploratory ROIs FDR-corrected across 11 ROIs. Bootstrapped 95% CIs for AUC via resampling test sets (n=500 per fold). Generalization analyses: Temporal cross-variable decoding tested whether models trained on RePE could decode RIPE at later timepoints (and in OFC, whether outcome-trained models could decode RePE). Conducted within 0.200–0.500 s after Card 2 (and across full epoch for robustness), using 2D cluster-based permutation tests with FWE control. Feature-importance via leave-one-feature-out quantified contact contributions; cross-variable correlations of OFC feature-weight vectors assessed representational independence. Behavioral analyses: Mixed-effects logistic regressions confirmed high report accuracy (~91.5%) with no learning-like strategies (no win-stay/lose-shift, guess/keypress stickiness) and no influence of prior-trial EV/RePE/E.Risk/RIPE on current guess, indicating trials were treated independently.

- Behavior: Participants reported outcomes accurately (mean 91.50%, s.e.m. 2.84%; β=0.415, 95% CI [0.358, 0.471], t(9)=14.398, p=1.184e-7). No effects of trial count on accuracy (β=−1.272, p=0.075). No evidence for win-stay/lose-shift or other heuristic strategies; next-trial guesses were not predicted by prior-trial EV/RePE/E.Risk/RIPE (all p>0.2). - Parallel EV encoding: EV was decodable in vmPFC (AUCpeak≈0.576; significant cluster 0.040–0.142 s after Card 1) and amygdala (AUCpeak≈0.564; 0.044–0.102 s), with overlapping timings. Exploratory: hippocampus showed two clusters (early 0.082–0.138 s; late cluster; both significant), with early overlapping vmPFC/amygdala. - Sequential outcome cascade: Outcome (after Card 2) was decodable across a posterior-to-anterior spatiotemporal gradient. Significant clusters with peak latencies increased along y-plane centroid: angular gyrus [0.008–0.064 s; y=−56], hippocampus [0.086–0.150 s; y=−17] (vs angular gyrus: U=957, p=2.44e−18), amygdala [0.188–0.236 s; y=−3] (vs hippocampus: U=825, p=5.73e−17), vmPFC [0.194–0.299 s; y=15] (vs amygdala: U=1108, p=8.33e−7), OFC [0.257–0.289 s; y=23] (later than vmPFC: U=688.5, p=8.14e−4), frontal pole [0.236–0.345 s; y=51] (later than OFC: U=620.5, p=0.030). Decoding also significant in hippocampus (q=0.007) and angular gyrus (q=0.007). OFC’s window overlapped neighbors despite respecting gradient in peak times. - Expected risk: E.Risk was decodable in OFC (0.287–0.317 s after Card1-off; AUCpeak≈0.551). Not significant in anterior insula. Exploratory: posterior insula [0.102–0.154 s; q=0.103] and supramarginal gyrus [0.449–0.499 s; q=0.103] showed E.Risk decoding. - Reward prediction error (RePE): Significant decoding in OFC [0.273–0.327 s; AUCpeak≈0.554] and anterior insula (early [0.277–0.307 s], late [0.339–0.405 s]; AUCpeak≈0.562–0.560). Exploratory: cingulate gyrus showed very early RePE [0.002–0.0785 s; q=0.011], suggesting an early error signal followed by simultaneous representations in multiple regions. Posterior insula showed a near-threshold overlapping cluster [0.269–0.299 s] (q=0.091). - Risk prediction error (RIPE): Significant decoding in OFC [0.206–0.293 s; AUCpeak≈0.543], anterior insula [0.431–0.499 s; AUCpeak≈0.540], and putamen [0.429–0.499 s] (exploratory). RIPE in anterior insula occurred later than RePE (U=1620, p=7.68e−24), consistent with RIPE depending on RePE computationally. No timing difference between putamen and anterior insula RIPE (U=647.5, p=0.425). - Cross-variable generalization: In anterior insula, a model trained on RePE generalized to RIPE at later times, indicating compositional coding: training [0.200–0.391 s] RePE predicted [0.383–0.499 s] RIPE (CAUC=62.504, Cthresh=28.641, p=0.014). Under a more stringent threshold, training [0.200–0.283 s] predicted [0.439–0.489 s] (CAUC=6.554, Cthresh=0.460, p<0.001). In OFC, no cross-variable generalization was found (Outcome→RePE; RePE→RIPE not significant), and feature-weight vectors for Outcome vs RePE (r=−0.206, p=0.214) and RePE vs RIPE (r=0.126, p=0.455) were uncorrelated, indicating separable representations. Sub-regional analyses implicated distinct OFC subregions (area 47/10, area 11, area 13) in parallel decoding of outcome, RePE, and RIPE, respectively. Overall: EV is represented in parallel across regions; outcome representations cascade from posterior to anterior; error signals (RePE, RIPE) exhibit mixed sequential and parallel organization. Anterior insula supports temporal compositionality from RePE to RIPE; OFC supports temporally parallel but representationally independent codes.

The findings delineate distinct temporal organizations for value-based computations across the human brain. EV, a top-down integrative computation after Card 1, was represented simultaneously across vmPFC, amygdala, and hippocampus, consistent with parallel coding within a distributed valuation network. In contrast, outcome representations, reliant on exogenous information at Card 2, unfolded as a posterior-to-anterior cascade from parietal (angular gyrus) through hippocampus and amygdala to vmPFC/OFC and frontal pole, indicating a temporally ordered flow consistent with hierarchical cortical gradients. Error computations shared mixed temporal configurations across domains. RePE appeared earliest in cingulate cortex, aligning with its role in general error monitoring, and then in OFC and anterior insula, revealing simultaneous distributed coding. RIPE emerged earlier in OFC and later in anterior insula and putamen. Critically, in anterior insula, RePE preceded RIPE and RePE-trained decoders generalized to RIPE at later times, directly supporting the hypothesis that RIPE computations build on RePE representations within the same region. This provides evidence for compositional neural codes where earlier computations serve as inputs to later ones. In OFC, despite parallel temporal decoding of outcome, RePE, and RIPE, lack of cross-variable generalization and uncorrelated feature-weight patterns indicate independent, subregion-specific codes rather than a shared representational transformation. These results advance understanding of how the human brain temporally organizes reward and risk computations, emphasizing that different computational demands (integration vs exogenous outcome resolution vs error signaling) map onto distinct temporal architectures (parallel vs sequential vs mixed) and highlighting functional roles of anterior insula and OFC in uncertainty processing.

This study provides a spatiotemporally resolved account of value and uncertainty computations in the human brain using iEEG and a within-trial task that temporally separates EV, outcome, E.Risk, RePE, and RIPE. Key contributions include: (1) demonstration of parallel EV encoding across vmPFC, amygdala, and hippocampus; (2) discovery of a posterior-to-anterior cascade for outcome representations; (3) identification of mixed sequential–parallel organization for error signals; and (4) evidence that anterior insula implements compositional coding wherein RePE representations support subsequent RIPE decoding, while OFC encodes multiple variables in parallel but with separable subregional representations. These findings underscore the importance of temporal dynamics in value-based decision-making under uncertainty. Future directions include: expanding sampling to increase coverage and power across ROIs; integrating causal electrophysiological perturbations to map directed information flow; examining learning-dependent and subjective computational variables; reconciling modality-specific discrepancies (iEEG vs fMRI) in risk-related codes; and characterizing tonic versus phasic properties and potential multi-epoch representations across tasks with greater complexity.

- Sample and coverage: Small patient sample with clinically determined electrode placements limits generalizability and may miss regions critical for specific computations. The pseudo-population approach assumes cross-subject pooling approximates a representative population and yields unequal ROI sampling, potentially reducing power in under-sampled regions. - Modality constraints: iEEG’s high temporal precision comes with limited coverage and potential discrepancies with fMRI findings (e.g., lack of anterior insula E.Risk decoding here), possibly due to contact locations (inferior plane of anterior insula) or differences in neural signals measured (electrophysiology vs BOLD). - Temporal windowing: Analyses targeted hypothesized event-locked windows for each variable; variables may also be represented outside these windows, which were not tested exhaustively. - Functional connectivity: Cross-correlation analyses did not reveal differences tied to computational variables; more sensitive or causal methods (e.g., cortico-cortical evoked potentials) are needed to test directed information flow. - Task design: The within-trial, learning-independent paradigm isolates computations but may limit conclusions about learning-related or belief-updating processes and ecological generalization.