Dynamic emotional states shape the episodic structure of memory

Everyday experience unfolds continuously, yet episodic memories are organized into discrete events via event segmentation. External context changes (e.g., spatial/perceptual shifts) are known to segment memory, but it is less clear whether internal context changes—namely dynamic emotional states—play a similar role in structuring temporal memory. Prior work suggests that stability versus change in arousal relates to integration vs separation of temporal information. Valence, a core affective dimension, may also differentially influence episodic organization: negative states can narrow attention and promote item-focused processing, whereas positive states broaden attention and may support associative binding. Most emotion-memory research has emphasized recognition or non-temporal features using static stimuli, overlooking how continuously fluctuating emotions shape memory for event structure within extended sequences. This study tests whether moment-to-moment changes in music-induced emotion (valence and arousal) act as internal contextual boundaries that segment or integrate episodic memories, and whether these dynamics enhance later memory for items and their temporal contexts.

Event segmentation research shows that items within stable contexts are linked, whereas abrupt context changes create event boundaries that separate memories and alter temporal judgments. Pupil-linked arousal dynamics relate to temporal order encoding, with auditory boundaries eliciting arousal spikes that impair temporal order and expand subjective time. Valence is a key affective dimension: negative affect often increases item-focused processing and can disrupt temporal binding; positive affect can broaden attention and enhance associative memory. Traditional emotional memory studies often use discrete emotional stimuli and emphasize item recognition or non-temporal features, providing limited insight into temporal structure across continuous experiences. Music can induce a wide range of emotions with reduced semantic confounds relative to emotional images or narrative films, enabling control over perceptual features while eliciting dynamic affect. This makes music a suitable tool to dissociate emotional dynamics from perceptual change and to examine how internal affective fluctuations organize episodic memories.

Design and preregistration: The design, stimuli, Emotion Compass tool, power analyses, and analyses were preregistered (OSF: https://doi.org/10.17605/osf.io/s8g5n). Participants: Ninety-six young adults (mean age = 27 years, SD = 4.9; 61 females) were recruited via Prolific with inclusion criteria (18–35 years; US/Canada; native English; normal/corrected vision; no serious head injury; ≥50 prior Prolific studies; ≥90% approval). Informed consent and IRB approval (UCLA) were obtained. Compensation was $9/h. Day 1 temporal memory analyses included 81 participants (9 did not complete Day 1; 6 excluded for <50% temporal order accuracy). Day 2 delayed memory analyses included 72 returnees. Emotion Compass profile analyses included 65 after quality control. Stimuli: Neutral object images (N=360; 300×300 dpi) from DinoLab Object Database, excluding highly arousing/extreme valence categories. Music: Three film-score composers (NYU program) wrote original instrumental segments (violin, guitar, cello, piano) designed to convey four emotions spanning arousal/valence extremes: anxious (high arousal, low valence), sad (low arousal, low valence), joyous (high arousal, high valence), calm (low arousal, high valence). Tempo and key were held constant across segments to partially dissociate perceptual change from felt emotion. Ten 120 s songs were created, each comprising three 30–40 s emotional segments joined by 6–9 s transitions, with no more than two segments sharing the same valence or arousal level. Procedure Day 1: Sequence encoding—Participants studied 11 sequences (first was practice; excluded) of 24 neutral objects while a song played. Each image displayed for 3 s with 2 s ISI (fixation). Participants formed mental narratives to promote deep and temporal encoding and pressed spacebar when incorporating images into narratives. Each song started 5 s before the first image. After each sequence, a 45 s arrow-detection distractor task (rapid left/right arrows, 0.5 s presentation, 0.5 s ISI) was performed. Temporal memory tests—After each sequence, 10 pairs of objects from that sequence were probed. For each pair: (1) temporal order judgment (which appeared first), then (2) subjective temporal distance rating (very close/close/far/very far). Critically, all probed pairs had exactly three intervening items during encoding (same objective distance). An 8 s response limit applied to each judgment. Day 2 (≈24 h delay; mean ≈22 h 36 m): Surprise recognition and temporal source memory. All studied objects plus 120 lures (50% lure rate) were presented in random order. Participants made old/new judgments with confidence. For items endorsed old, a temporal source judgment was made using a 24-position slider corresponding to serial position within the original list. Emotion Compass task: After the Day 1 tasks, participants re-listened to each song (same order) and continuously rated felt valence (x-axis; negative to positive) and arousal (y-axis; low to high) using a cursor-controlled 2D circumplex interface sampled at 6 Hz. A colored planchette provided feedback (valence color gradient; arousal saturation). Participants practiced on a non-tested clip and were instructed to rate how the music made them feel, not how it sounded. Compliance was surveyed; included participants reported tracking felt emotion. Emotion Compass preprocessing: Ratings were mean-filtered across every 3 frames (6 Hz to 2 Hz), then a Savitzky–Golay filter (window=50; order=5) was applied to capture low-frequency trends while preserving sudden changes. Inclusion criteria for usable Compass data: engagement (>50% of song with changes), no rapid axis-crossing (>2 within 3 s), and absence of frenetic/artifactual movements. To retain participants with noisy cursor data but good memory performance, song-level normative valence/arousal profiles were created by concatenating cleaned participant ratings for each song. These song-level time series were mapped back onto encoding trials. Each trial window spanned the 3 s image plus the subsequent 2 s ISI (total 5 s) to assign average valence and arousal scores. Discrete boundary detection: Perceptual (musical) boundaries were annotated by a separate in-lab sample (n=6) who pressed a key at meaningful musical changes (pitch, tempo, tone) constrained to ~30–70 s spacing and 2–3 changes per song. Inter-annotator similarity (Dice coefficient) was 0.823; average 2.04 boundaries/song. Consensus boundary times were averaged across annotators. Emotional boundaries (valence, arousal) were identified via change-point analysis on each song’s normative valence and arousal profiles using ruptures (binary segmentation; penalization factor=15), detecting significant changes in mean and slope. On average, 2.1 valence and 2.4 arousal change-points were detected per song. Boundary timestamps were aligned to encoding: for paired temporal memory, the window was from the start of the first item to the end of the second item’s ISI; for Day 2 items, from item onset to end of its ISI. Trials with a change-point within the window were labeled boundary trials. Continuous emotional change metrics: For each probed pair, absolute difference and signed change (second minus first) in valence and arousal were computed from the song-level profiles. Modeling and statistics: Linear mixed-effects models (LMMs) and generalized LMMs (GLMMs; lme4 in R) tested relations between boundary labels or continuous emotion metrics and outcomes: subjective temporal distance (LMM), temporal order accuracy (GLMM), recognition (GLMM), and temporal source displacement (LMM). Fixed effects included boundary indicators (perceptual/valence/arousal), absolute/signed emotion differences, and for Day 2, item-level valence and arousal, plus their interaction. Random intercepts for participant and song were included. Significance was assessed via likelihood ratio tests; post-hoc contrasts used FDR-corrected P-values. Model assumptions were checked with DHARMa; residuals did not deviate from normality and variances were homogeneous.

Event boundaries and immediate temporal memory: • Subjective time dilation: Boundary-spanning pairs were judged farther apart than non-boundary pairs for perceptual boundaries (β=0.02, SE=0.008, χ²(1)=7.28, p=0.007; post-hoc distance difference=0.045, z=2.7, p=0.007) and valence boundaries (β=0.034, SE=0.008, χ²(1)=16.8, p<0.001; post-hoc difference=0.066, z=3.938, p<0.001), but not arousal boundaries (β=−0.002, p=0.772). • Temporal order: Accuracy was impaired for pairs spanning perceptual (β=−0.09, SE=0.027, χ²(1)=13.86, p<0.001; post-hoc difference=0.201, z=3.733, p<0.001) and valence boundaries (β=−0.15, SE=0.027, χ²(1)=32.78, p<0.001; post-hoc difference=0.31, z=5.739, p<0.001), but not arousal boundaries (β≈0, p=0.992). • Above-and-beyond perceptual change: Valence boundary effects remained significant when controlling for perceptual boundaries for both subjective dilation (B=0.033, χ²(1)=15.5, p<0.001) and temporal order impairment (B=−0.15, χ²(1)=30.24, p<0.001). Continuous emotion dynamics and immediate temporal memory: • Absolute valence change between items predicted greater subjective time dilation (β=0.019, SE=0.009, χ²(1)=5, p=0.025) and greater temporal order impairment (β=−0.08, SE=0.028, χ²(1)=7.23, p=0.007). • Signed valence change (more positive over time) predicted subjective temporal compression (β=−0.022, SE=0.009, χ²(1)=6.61, p=0.013) and better temporal order (β=0.076, SE=0.037, χ²(1)=4.19, p=0.041). • Exploratory: Integration effects were driven by shifts away from highly negative states to less negative states (compression: β=−0.047, p=0.006; order: β=0.12, p=0.008), whereas shifts within the positive range were not significant. • Arousal (absolute or signed) did not predict immediate temporal distance or order (all ps>0.25). Long-term item and temporal source memory: • Items at boundaries: Recognition was higher for items at perceptual (z=4.11, PFDR<0.001) and valence (z=2.996, PFDR=0.008) boundaries vs non-boundary; arousal boundaries showed no significant recognition benefit (PFDR=0.261). Temporal source displacement was lower (better) for items at perceptual (z=−5.83, PFDR<0.001), valence (z=−10.94, PFDR<0.001), and arousal (z=−9.08, PFDR<0.001) boundaries vs non-boundary; valence and arousal boundaries outperformed perceptual boundaries (valence vs perceptual: z=−3.81, PFDR<0.001; arousal vs perceptual: z=−2.2, PFDR=0.031). • Valence × arousal interactions at encoding: Recognition—A significant interaction (χ²(3)=8.73, p=0.033); under high valence, recognition increased with arousal (β≈0.08, p<0.001), whereas under low valence, arousal did not predict recognition (p=0.721). Temporal source—A significant interaction (p<0.001); under high valence, increasing arousal reduced displacement (β≈−0.33, p<0.001; better source), whereas under low valence, increasing arousal increased displacement (β≈0.13, p<0.001; worse source). Overall pattern: • Discrete and continuous changes in emotional valence segment memory (dilation, impaired order), beyond perceptual changes. • Positive shifts (especially reductions in negativity) promote temporal integration (compression, improved order). • High-arousal positive states enhance delayed item and temporal source memory, while high-arousal negative states impair temporal source memory.

Dynamic changes in felt emotion function as potent internal contextual cues that shape the temporal organization of episodic memory. Large shifts in emotional valence, like external perceptual changes, serve as event boundaries that separate adjacent experiences, expanding subjective time and disrupting temporal order memory. Crucially, valence dynamics explained segmentation effects beyond lower-level musical changes, indicating that emotion provides unique scaffolding for organizing novel experiences. At the same time, the direction of valence change matters: shifts toward more positive states, largely driven by reductions in negativity, facilitated temporal integration (compressed subjective time and better order), consistent with broaden-and-build accounts linking positive affect to relational processing and associative binding. Music-induced states of high positive valence with high arousal strengthened both item representations and their temporal contexts over 24 h, suggesting that such states enhance both encoding and consolidation of neutral information encountered concurrently. Emotional and perceptual boundaries also acted as temporal landmarks, improving long-term memory for when items occurred, with emotional boundaries conferring greater benefits than perceptual boundaries alone. The absence of arousal effects on immediate temporal metrics, but clear effects on delayed item and source memory, suggests that arousal-related benefits may emerge more strongly after consolidation. Collectively, these findings extend event segmentation theory by demonstrating that internal, dynamic affective contexts—particularly valence trajectories—organize episodic memory, with positive affect supporting binding across time and negative affect promoting segmentation.

Music-evoked emotion dynamics sculpt continuous experiences into discrete, memorable events. Discrete and continuous shifts in valence segment temporal memory, whereas positive shifts—especially relief from negative states—promote integration. Emotional and perceptual boundaries enhance long-term recognition and temporal source memory, with high-arousal positive states yielding the strongest benefits. These results identify dynamic emotional states as internal contextual signals that structure episodic memory across time, advancing theories of event segmentation and affect-cognition interactions. Future work should test generalization to richer, naturalistic narratives and stimuli, examine individual-level emotion traces (rather than song-level aggregates), probe neural mechanisms linking affective dynamics to hippocampal-cortical event modeling, and determine how manipulating affect can therapeutically improve temporal coherence in populations with memory disorganization.

Arousal dynamics did not predict immediate temporal memory, potentially reflecting the need for consolidation, measurement insensitivity, or strategic reliance on valence during encoding. Emotion Compass profiles were aggregated at the song level to retain participants, which may have reduced sensitivity to individual differences in arousal trajectories. Music, though relatively low in semantic content, can elicit narratives and imagery that might influence encoding strategies; instructions to form narratives may have introduced uncontrolled semantic factors. Although perceptual change was controlled (tempo/key) and annotated, emotional and perceptual changes cannot be fully disentangled. Boundary positions tended to occur around one-third and two-thirds of lists, though analyses addressed list-position confounds. Generalizability to real-world emotional events with rich semantic and causal structure remains to be established, particularly under intense aversive arousal.