Serotonin and psilocybin activate 5-HT1B receptors to suppress cortical signaling through the claustrum

The study investigates how serotonin (5-HT) and the psychedelic psilocybin modulate cortico-claustral signaling, focusing on the anterior cingulate cortex (ACC) input to the claustrum and its downstream projections to parietal association cortex (PtA). Neuromodulators, including serotonin, broadly influence brain states and cognitive control, yet their roles in subcortical hubs like the claustrum are underexplored. The claustrum supports cortical network states via cortico-claustro-cortical loops and receives dense serotonergic input from the dorsal raphe. Human and animal data suggest psilocybin affects claustrum functional connectivity and BOLD signal fluctuations. The hypothesis tested is that 5-HT, and psilocybin (via psilocin), suppress ACC-claustrum synaptic transmission through presynaptic 5-HT1B receptors, thereby modulating claustrum-dependent cortical signaling.

Prior work shows the claustrum’s integral role in coordinating cortical networks and slow-wave sleep, with dense ACC input that is behaviorally significant for cognitive control. The dorsal raphe provides serotonergic innervation to the claustrum and exhibits state-dependent activity (reduced during sleep), while the claustrum generates synchronous cortical events during slow-wave sleep. Human studies report psilocybin disrupts claustrum functional connectivity and reduces low-frequency BOLD amplitudes. Serotonin receptor expression (including 5-HT1B/1D/2A/2C/1A) is enriched in the claustrum across species. Elsewhere, 5-HT1B presynaptic receptors depress excitatory synapses (e.g., corticostriatal, lateral habenula). Psilocin has high affinity for 5-HT1B, and many psychedelic effects are attributed to 5-HT2A, though broader receptor involvement has been recognized. These findings motivate testing 5-HT1B-mediated modulation at the ACC-claustrum synapse and its engagement by psilocybin.

Animals: Male and female C57BL/6 mice (10–20 weeks), group-housed, standard light/dark cycle. Equal sex distribution unless noted. All procedures followed University of Maryland IACUC guidelines. Viral strategies: For ACC input stimulation, AAV5-hSyn-ChR2(H134R)-EYFP was injected into ACC (AP +1.00 mm, ML ±0.30 mm, DV −1.10 mm). For projection-defined recordings, rgAAV-CAG-tdTomato was injected into PtA (AP −1.90 mm, ML ±1.40 mm, DV −0.40 mm) to retrogradely label PtA-projecting claustrum neurons. Expression allowed for 4–6 weeks before recordings. Slice preparation: Coronal brain slices (250 µm) prepared in ice-cold sucrose-based cutting solution, then incubated in oxygenated aCSF (2 mM CaCl2). For asynchronous release experiments, Sr2+ replaced Ca2+ during recording. Electrophysiology: Whole-cell recordings (−60 mV for voltage clamp; standard K-gluconate internal for current clamp and CsMeSO3 internal for voltage clamp) at 28–30 °C. Optogenetic stimulation (470 nm, 5 ms pulses) of ACC terminals to evoke oEPSCs (voltage clamp) or oEPSPs/APs (current clamp). Paired-pulse protocol (50 ms ISI) every 20 s; drug applications for 10 min followed by wash. For projection-defined assays, stimulation intensities spanned 0–100% (0–3.2 mW) with three pulses (150 ms interpulse) per intensity; nine stimulations per intensity; outcomes averaged as AUC of voltage response and APs per light pulse (APs/LP). Pharmacology: Bath-applied agents (Tocris): 5-HT (10 µM), CP-93129 (5-HT1B agonist, 10 µM), SB-216641 (5-HT1B antagonist, 20–40 µM), GR-127935 (5-HT1B/D antagonist, 20 µM), PNU-142633 (5-HT1D agonist, 10 µM), LY-310762 (5-HT1D antagonist, 10 µM), TTX (500 nM), 4-AP (100 µM), picrotoxin (50 µM). sEPSCs recorded in PTX; mEPSCs in PTX+TTX. Asynchronous oEPSCs in Sr2+. In vivo psilocybin experiments: Mice received i.p. psilocybin (1 mg/kg) or saline. Head twitch responses (HTR) were recorded for 30 min as a pharmacodynamic readout (SLEAP-based pose estimation plus human confirmation). Brains were then sliced for ex vivo recordings of ACC-evoked responses in PtA-projecting claustrum neurons. For receptor dependence, GR-127935 (10 mg/kg i.p.) or vehicle was administered 15 min before psilocybin. To assess persistence, recordings were conducted 24 h after psilocybin or saline. Data analysis: Electrophysiological data analyzed with Minhee Analysis Package, Python, pyABF; statistics in GraphPad Prism. Paired or unpaired two-tailed Student’s t-tests as appropriate. Significance thresholds: *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. Controls: Monitored series resistance; assessed intrinsic properties (capacitance, resting potential, input/membrane resistance, rheobase, AP threshold) to rule out postsynaptic excitability changes. Sex comparisons performed for select measures.

• 5-HT depresses ACC→claustrum synaptic transmission via a presynaptic mechanism:
  • Bath 5-HT (10 µM, 10 min) reduced first oEPSC amplitude vs baseline (t10=3.43, p=0.0064, n=13 neurons) and vs no-drug control (t16=2.527, p=0.0224; control n=6). Increased paired-pulse ratio (PPR; t12=3.18, p=0.007). Coefficient of variation (COV) of first EPSC decreased (t12=3.80, p=0.002). Effect persisted in TTX+4-AP, indicating monosynaptic mechanism (difference vs 5-HT alone: t14=0.017, p=0.99).
• 5-HT1B receptor mediates the depression:
  • 5-HT1B agonist CP-93129 (10 µM) depressed oEPSCs (t10=4.81, p<0.001; n=11), increased PPR (t12=2.43, p=0.046; n=8), and increased COV (t8=3.80, p=0.005). SB-216641 (5-HT1B antagonist) blocked CP-93129 depression (group comparison t15=2.90, p=0.011; SB n=6). GR-127935 (5-HT1B/D antagonist) prevented CP-93129 effect (paired t4=0.58, p=0.59).
  • SB-216641 (40 µM) significantly attenuated 5-HT-induced depression (t10=2.83, p=0.018), indicating substantial 5-HT1B contribution.
  • Specificity: LY-310762 (5-HT1D antagonist) did not block CP-93129-induced depression (paired t4=5.89, p=0.004); 5-HT1D agonist PNU-142633 (10 µM) had no effect (paired t5=0.33, p=0.76).
• Spontaneous and asynchronous release analyses support presynaptic locus:
  • 5-HT increased sEPSC interevent interval (IEI) (paired t7=2.55, p=0.038) with no amplitude change (t7=2.18, p=0.066).
  • CP-93129 increased sEPSC IEI (paired t7=2.56, p=0.037) with no amplitude change (t7=0.21, p=0.84).
  • At ACC inputs (a-oEPSCs in Sr2+), CP-93129 increased IEI (paired t11=2.30, p=0.042) without amplitude change (t11=0.057, p=0.96).
  • mEPSCs unaffected by 5-HT or CP-93129 in TTX (IEI and amplitude ns), consistent with effects on presynaptic excitability/VGCC-dependent release.
• Projection-defined recordings: suppression of ACC drive onto PtA-projecting claustrum neurons:
  • After 5-HT (10 µM, 10 min), AUC and APs/LP were lower vs pre-exposure cells from same slices (both t22=2.10, p=0.048; n=12 vs 12). Intrinsic properties unchanged.
  • After CP-93129 (10 µM, 10 min), AUC (t27=2.30, p=0.030) and APs/LP (t29=2.36, p=0.025) were reduced vs pre-exposure; intrinsic properties unchanged.
• Psilocybin engages 5-HT1B to depress ACC→claustrum→PtA signaling:
  • In vivo psilocybin (1 mg/kg i.p.) increased head twitch responses (t18=12.45, p<0.0001) and reduced AUC (t18=2.67, p=0.016) and APs/LP (t18=2.26, p=0.036) ex vivo; intrinsic properties unchanged.
  • Pre-injection with GR-127935 (10 mg/kg i.p.) did not alter head twitches (t22=0.91, p=0.37) but prevented the synaptic depression: higher AUC (t22=2.55, p=0.018) and APs/LP (t22=2.73, p=0.012) vs saline pre-injected group.
  • Lasting effect: 24 h after psilocybin (1 mg/kg), AUC (t25=2.16, p=0.041) and APs/LP (t25=2.32, p=0.029) remained reduced vs saline, despite psilocin being cleared within 24 h. Overall, serotonin and psilocybin suppress ACC-driven excitation of PtA-projecting claustrum neurons predominantly via presynaptic 5-HT1B receptors.

The findings directly address the hypothesis that serotonergic modulation suppresses cortico-claustral transmission via presynaptic 5-HT1B receptors. Optogenetic and slice electrophysiology demonstrate that 5-HT and a selective 5-HT1B agonist reduce glutamate release probability at ACC terminals in the claustrum, evidenced by decreased oEPSCs, increased PPR, altered event frequency without amplitude changes, and persistence in TTX+4-AP conditions. Projection-specific recordings show that this synaptic depression functionally weakens ACC drive onto PtA-projecting claustrum neurons without altering postsynaptic intrinsic excitability, indicating a presynaptic locus. In vivo relevance is supported by psilocybin administration, which elevates head twitch responses and yields a persistent reduction of ACC-evoked responses in identified claustrum projection neurons. Antagonism with GR-127935 mitigates this depression without affecting 5-HT2A-mediated head twitching, implicating 5-HT1B mechanisms specifically in the circuit-level suppression. The persistence up to 24 h suggests longer-lived synaptic modulation downstream of transient drug exposure. These results position 5-HT1B as a key regulator of ACC input gain to the claustrum, offering a mechanistic substrate for how serotonergic state and psilocybin could modulate broader cortical network dynamics via the claustrum. Given known psilocybin-induced alterations in claustrum functional connectivity, 5-HT1B-dependent presynaptic inhibition at cortico-claustral inputs likely contributes to network desynchronization and altered information flow, alongside additional receptor- and circuit-specific effects elsewhere in cortex.

This work identifies a presynaptic 5-HT1B-dependent mechanism whereby serotonin, and psilocybin via psilocin, suppress glutamatergic ACC input to claustrum neurons, including those projecting to PtA. The mechanism reduces synaptic efficacy and functional output (AUC, APs/LP) without altering intrinsic excitability, persists at least 24 h after psilocybin, and is prevented by 5-HT1B/D antagonism. These findings provide a circuit-level explanation for serotonergic and psychedelic modulation of claustrum-mediated cortical networks and suggest 5-HT1B as a potential therapeutic target for modulating cortical network states. Future research should determine when and how this 5-HT1B mechanism is naturally engaged in vivo across behavioral states, its prevalence across diverse cortico-claustral pathways, contributions at the claustrum→cortex synapse, interactions with other 5-HT receptor subtypes, and its relationship to long-term structural plasticity and therapeutic outcomes of psychedelics.

• Generalizability: Findings are specific to the ACC→claustrum→PtA pathway and may not extend to all cortico-claustro-cortical circuits.
• Receptor specificity: Although pharmacology implicates 5-HT1B, contributions from other serotonin receptors (e.g., 5-HT2A, 5-HT2C, 5-HT1A) cannot be excluded, especially in vivo.
• Synaptic locus: Effects at the claustrum→cortex synapse were not examined, leaving downstream modulation uncharacterized.
• Slice pharmacology: Acute slice conditions and pharmacological manipulations limit direct inference to intact in vivo network states.
• Temporal scope: Some psilocybin-induced structural changes (e.g., dendritic spine growth) may emerge beyond 24 h and were not captured.
• Analytical notes: Reported variability measures (e.g., COV changes) showed condition-specific directions; mechanistic underpinning (e.g., VGCC involvement) is inferred from established literature rather than directly tested here.