Dynamic brain networks in spontaneous gestural communication

The study investigates how different types of spontaneous gestures shape collaborative communication and associated neural dynamics between interacting individuals. Building on distinctions between mirroring (action-perception) and mentalizing systems in social cognition, the authors ask: Do distinct gesture types (interactive, fluid, object-related, emotional) differentially relate to dynamic inter-brain (ISFC) and intra-brain (FC) states, and do these neural states predict collaborative performance? The work leverages naturalistic, turn-taking problem-solving to overcome limitations of prior single-brain, static analyses and explicitly tests three hypotheses: (I) more efficient inter-brain states (higher global efficiency and clustering, shorter path length) occur more often when both partners can gesture and relate positively to performance; (II) gesture type modulates inter-brain states, with interactive gestures recruiting superior inter-brain states and stronger ISFC within mirroring and mentalizing networks; (III) at the intra-brain level, superior states are more frequent when both can gesture, correlate with task performance, and fluid gestures preferentially relate to efficient intra-brain states and cognitive fluency.

Prior work shows gestures enhance comprehension, regulate dialogue, and support creative and flexible thinking. Interactive gestures facilitate turn-taking and social exchange; fluid gestures (repetitive, non-referential movements) enhance creative fluency and flexibility; object-related gestures aid idea generation tied to objects; emotional gestures bias attention to affective content. Neuroscientific accounts implicate complementary mirroring (IFG, STG, frontoparietal circuits) and mentalizing (TPJ, precuneus, PFC) systems in gesture use and understanding. Inter-subject functional connectivity (ISFC) indexes neural alignment during interaction and predicts comprehension and cooperation; stronger ISFC across mirroring and mentalizing networks aligns with better perspective-taking and conversational understanding. However, most studies rely on static, averaged measures or observer-only paradigms and lack time-resolved links between specific spontaneous gestures and instantaneous brain states. Dynamic approaches (sliding windows, clustering) can reveal fluctuating inter- and intra-brain states and their behavioural relevance.

Design: Within-subject fNIRS hyperscanning study of 27 dyads (N=54; mean age 20.6±2.2; right-handed), completing three conditions in counterbalanced order during collaborative realistic presented problems (RPPs): BOTH (both can gesture), ONE (only one assigned gesturer), NO (neither can gesture). Turn-taking conversation; one idea per turn; unlimited turn duration; passes allowed. Baseline: 2-min eyes-closed rest; three 5-min task sessions interleaved with rest. Pre/post ratings: valence/arousal (SAM), depletion, difficulty, enjoyment (1–5).

Gesture coding: Video-recorded spontaneous gestures classified into interactive, fluid, object, emotional based on functional criteria. Two trained raters coded counts; inter-rater reliability: interactive ICC=0.82, object=0.75, fluid=0.78, emotional=0.71. Behavioural metrics: fluency (idea count), flexibility (categories/fluency; ICC=0.83), originality and feasibility (3 raters; 5-point Likert; ICC originality=0.81, feasibility=0.70). Dyadic index of cooperation (IOC)=converge/(group fluency−converge).

fNIRS acquisition: Hitachi ETG-7100 measured HbO/HbR over PFC (3×5 optode set; 22 channels) and right temporal-parietal-occipital (r-TPO; 4×4; 24 channels). Probe registration via 10–20 system and 3D digitizer; channels mapped to AAL; channels grouped into 13 ROIs: FPC, lDLPFC, rDLPFC, lIFG, rIFG, rMTG, rSTG, rPSC, rAG, rSMG, rMotor, rSAC, V3. Mirroring system: lIFG, rIFG, rSTG, rPSC, rMotor, rSAC, V3; Mentalizing: FPC, lDLPFC, rDLPFC, rMTG, rAG, rSMG.

Preprocessing: Visual inspection; noisy channels replaced by neighbouring within-ROI (avg 2.3 channels/participant). Principal component spatial filter (PCA) to remove systemic components; correlation-based signal improvement (CBSI) for motion artefacts. Trimming first/last 30 s of each task block (240 s analysed). Primary analyses on HbO.

Connectivity computation: Wavelet transform coherence (WTC) to compute intra-subject FC (13×13 per participant) and inter-subject ISFC (13×13 per dyad; same ROI pairs averaged across participants). Frequency-of-interest (FOI) determined by comparing task vs baseline across full range (0.01–0.7 Hz) with FDR correction; significant band 0.034–0.045 Hz (period 22.2–29.7 s) used; FC/ISFC averaged within FOI.

Dynamic analysis: Sliding windows of 26 s (per FOI), step 1 s, to generate time-resolved FC/ISFC matrices. Group-averaged matrices clustered via k-means (Manhattan distance; elbow criterion; k=3; 1000 iterations) to derive centroids (State 1–3). These centroids used to label dyad-level windows for dFC/dISFC.

Graph metrics: Thresholded matrices over sparsity 0.20–0.50 (step 0.01) to weighted networks; computed clustering coefficient (Cp), shortest path length (Lp), global efficiency (globE) using GRETNA; ROI strength defined as sum of connections to a ROI. For FC: under BOTH and NO, participant 1 and 2 intra-brain matrices averaged; under ONE, analysed separately for user (gesturer) and observer.

Behaviour locking: Time stamps of coded gestures used to extract concurrent dISFC/dFC states and ROI strengths for each gesture type and for non-gesture periods.

Statistics: Repeated-measures ANOVAs with Condition or State factors on behavioural outcomes, gesture counts, network metrics, occurrence rates, and ROI strengths; post-hoc Bonferroni and FDR corrections applied. Pearson correlations tested links between state occurrence rates and behaviour (IOC, fluency, originality, flexibility, gesture counts). Validation/permutation: ISFC reshuffled dyads (400 iterations) and static sISFC (1000 iterations); FC circular time-shifts (400 iterations) to verify that observed increments and correlations were in top 5% vs null distributions. Static FC/ISFC also computed by averaging across the entire task window for comparison.

Behavioural outcomes:

• Condition effects: sqrt(IOC) F(2,52)=7.96, p=0.001, ηp²=0.23; originality F(2,52)=25.29, p<0.001, ηp²=0.49; fluency F(2,52)=3.58, p=0.035, ηp²=0.12. Post-hoc: BOTH > ONE and NO for sqrt(IOC) (MBOTH=0.64, MONE=0.53, MNO=0.49) and originality (MBOTH=3.10, MONE=2.63, MNO=2.48). Fluency: ONE > NO (MONE=20.52, MNO=17.67).
• Gesture usage: Compared to BOTH/2, ONE used significantly more fluid gestures and fewer interactive gestures, suggesting compensatory use of isolated, fluid movements when only one partner can gesture.
• Correlations: Number of interactive gestures correlated with originality (BOTH: r=0.513, p=0.006; ONE: r=0.678, p<0.001) and with fluency in BOTH (r=0.439, p=0.022). Number of fluid gestures correlated with fluency (BOTH: r=0.534, p=0.004; ONE: r=0.687, p<0.001). No condition differences in depletion, difficulty, enjoyment, valence, arousal.

Dynamic inter-brain connectivity (dISFC):

• Three states identified per condition. State 1 exhibited higher globE and Cp and lower Lp than States 2/3 (BOTH and ONE); in NO, States 1/2 > State 3 on globE and Lp. ROI strength: State 1 > States 2/3 across conditions.
• ISFC increments vs baseline: Under BOTH and ONE, only State 1 showed significant ISFC increases concentrated in mirroring and mentalizing areas (e.g., rSAC, rSTG, rSMG, rPSC). States 2/3 showed no significant increases.
• Occurrence patterns: In NO, State 1 occurred less than States 2/3; in ONE, State 1 and State 2 occurred less than State 3. Under BOTH, State 1 occurrence positively correlated with originality (r=0.43, p=0.041) and sqrt(IOC) (r=0.44, p=0.020).
• Behaviour locking (inter-brain): In BOTH, interactive gestures increased State 1 occurrence (S1>S2/S3; F(2,52)=40.06, p<0.001, ηp²=0.61; M1=0.56, M2=0.26, M3=0.18). In ONE, during no-gesture periods, State 3 > States 1/2. ROI strength: In BOTH, interactive gestures elevated FPC, rDLPFC, rPSC vs fluid and none; visual cortex strength higher during fluid and interactive vs none. In ONE, rMTG strength higher during interactive/fluid vs emotional (limited emotional gesture counts; permutation confirmed robustness).

Dynamic intra-brain connectivity (dFC):

• Three states per condition (BOTH, NO, ONE-user, ONE-observer). Across conditions, State 1 had higher globE, Cp, ROI strength and lower Lp than States 2/3.
• FC increments vs baseline: BOTH and ONE-observer showed State 1 increases in mirroring and mentalizing regions (e.g., rIFG, lIFG, rSTG, rSMG, rDLPFC); ONE-user showed increases mainly in mentalizing (e.g., rSMG, rDLPFC); NO showed no significant FC increments.
• Correlations: State 1 occurrence correlated with RPP fluency in BOTH (r=0.38, p=0.048) and ONE-user (r=0.38, p=0.049); also with number of fluid gestures in ONE-user (r=0.40, p=0.037).
• Behaviour locking (intra-brain): In BOTH and ONE-user, fluid gestures increased State 1 occurrence (BOTH: F(2,52)=5.78, p=0.005, ηp²=0.18; ONE-user: F(2,52)=5.81, p=0.005, ηp²=0.18). In ONE-observer, interactive gestures tended to increase State 1 (marginal, F(2,52)=2.94, p=0.061). Visual cortex ROI strength higher when observers watched interactive vs object gestures.

Static analyses:

• sISFC: FPC–rSTG higher in BOTH and ONE than NO; rDLPFC–rSTG higher in BOTH than ONE and NO. Static intra-brain FC showed no significant differences, underscoring the sensitivity of dynamic analysis.

Overall: Interactive gestures align with superior inter-brain states and team originality/cooperation, while fluid gestures align with efficient intra-brain states and individual fluency. Absence of gestures increases inefficient inter-brain states (State 3).

Findings demonstrate that spontaneous gestures actively shape multi-brain dynamics during face-to-face collaboration. Efficient inter-brain State 1 (higher clustering and global efficiency; shorter path length; stronger ROI strength within mirroring and mentalizing systems) is associated with better cooperation and originality and is preferentially engaged during interactive gestures, consistent with gestures’ role in joint attention, perspective-taking, and information exchange. Fluid gestures preferentially relate to efficient intra-brain State 1 and correlate with individual fluency, supporting their intrapersonal facilitation of cognitive processing. When gestures are absent or restricted, dyads more often occupy inefficient inter-brain configurations (State 3), aligning with reduced collaborative performance. Dynamic analyses capture transient, behaviour-linked neural alignment that static averages miss. Together, the results indicate complementary but distinct roles: interactive gestures promote inter-brain coupling supporting idea combination and originality; fluid gestures streamline individual processing, boosting fluency. These neural signatures map onto mirroring and mentalizing networks and clarify how gestural behaviour supports communication efficacy.

This work introduces a dynamic, behaviour-locked hyperscanning framework linking specific spontaneous gesture types to time-resolved inter- and intra-brain states and to collaborative outcomes. Interactive gestures foster efficient inter-brain configurations and team originality/cooperation; fluid gestures promote efficient intra-brain configurations and individual fluency. The approach reveals fine-grained brain–behaviour coupling that static methods overlook, offering a route to optimise communication and collaborative creativity by leveraging gesture. Future research should broaden cortical coverage beyond PFC and right TPO, refine gesture taxonomies (especially emotional content and arousal), and test more naturalistic, unconstrained interactions to enhance ecological validity and generalisability.

• Neural coverage targeted PFC and right TPO; other relevant regions were not recorded, potentially omitting network contributions.
• Emotional gestures were not subdivided by emotion category or arousal, limiting interpretability of their neural effects.
• ONE and NO gesture conditions are less natural than free interaction; turn-taking and task-specific constraints may limit generalisability to everyday communication.
• Emotional gesture counts were low, reducing statistical power for that category (addressed via permutation but still a constraint).