Constructing neural network models from brain data reveals representational transformations linked to adaptive behavior

The study investigates how the human brain transforms sensory stimulus information, under task rules, into motor responses during context-dependent tasks. While prior work localized representations of sensory, motor, and rule features across cortex, it remained unclear how the brain uses and transforms those representations to implement flexible behavior. Building on the Guided Activation Theory, which posits selective mixing of task context and sensory activity to form conjunctive activations, and the Flexible Hub theory, which emphasizes flexible rule updating, the authors hypothesize that specialized association regions (conjunction hubs) integrate task rules and sensory stimuli to generate appropriate motor activations. They aim to empirically test this by constructing an empirically-estimated neural network (ENN) from human fMRI data that maps rule and sensory activations to motor responses via resting-state functional connectivity (FC), thereby providing a computational account of representational transformations supporting adaptive behavior.

Prior studies showed sensory features are represented in sensory cortices, motor action features in motor cortices, and task rules in prefrontal and association cortices. The Flexible Hub theory highlights cognitive control networks updating rule representations for flexible behavior but does not specify interactions with sensory inputs. The Guided Activation Theory proposes that rule and sensory signals interact to form conjunctive activations that guide motor outputs. Artificial neural networks (ANNs) can model such context-dependent transformations, and trained ANNs often exhibit representational geometries resembling neural data, supporting their use as blueprints for identifying brain representations. Related work has demonstrated mixed selectivity in complex tasks, compositional rule coding in control networks, and the feasibility of predicting inter-area activations via activity flow over resting-state networks. These frameworks motivated using an ANN’s hidden-layer representational similarity to identify conjunction hubs in human fMRI.

Participants: 106 recruited; 96 included for fMRI analyses (54 females; mean age 22.06±3.84), right-handed native English speakers, with informed consent under Rutgers IRB. Task: The Concrete Permuted Rule Operations (C-PRO) paradigm with 12 rules across 3 domains (logic, sensory, motor), permuted to yield 64 task contexts. Each miniblock: 3925 ms encoding, 1570–6280 ms delay, three 2355 ms stimulus trials (simultaneous audiovisual), with inter-trial intervals and post-block jitter. MRI acquisition: 3T Siemens Trio, multiband EPI (TR=785 ms, TE=34.8 ms, 2.0 mm isotropic, multiband 8), 14 min resting-state scan, eight task runs (7m36s each). High-res T1/T2 and field maps collected. Preprocessing via HCP minimal pipeline, followed by nuisance regression (24 motion regressors, aCompCor 40 physiological regressors; total 64), demeaning/detrending; analyses in CIFTI 64k grayordinates. Task GLM: Estimated activations for 12 task rules during encoding, 16 sensory stimulus pairings during stimulus periods (color, orientation, pitch, continuity), and 4 motor responses (finger presses) during response windows. Counterbalancing ensured independence among regressors. Decoding/localization: - Sensory inputs: Four-way decoding within visual and auditory network parcels (Ji et al. atlas) for each sensory dimension using minimum-distance classifier; FDR-corrected p<0.05 to select vertices. - Task rule inputs: 12-way decoding across all 360 Glasser parcels; FDR-corrected p<0.05 to select vertices (rule representations widespread, with dorsal attention network contributing strongly via FC to conjunctions). - Motor outputs: Univariate contrasts within somatomotor network to identify index vs middle finger response vertices per hand (paired t-tests; FDR p<0.05), matching somatomotor homunculus; baseline motor decoding assessed with cross-validation. ANN construction (for identifying conjunction hubs): A feedforward ANN with 28 input units (12 rules, 16 stimulus pairs), two 1280-unit hidden layers (ReLU), and 4 output units (fingers) trained with Adam on a computational analog of C-PRO to 99.5% accuracy. Hidden-layer representational similarity matrix (RSM) computed by systematically activating single input units (12 rules, 16 stimuli). Control ANN RSM obtained by shuffling learned weights/biases post-training. Conjunction hub identification: Computed RSMs for each Glasser parcel from fMRI activations (same 28 conditions) and correlated with ANN hidden-layer RSM (Spearman). Top 10 parcels with highest similarity selected as conjunction hubs (also tested top 20/30/40). Control analyses used random parcel selections. Functional connectivity estimation: Vertex-to-vertex FC between ENN layers estimated from resting-state fMRI using principal components regression with 500 components, for mappings: rule→hidden, stimulus→hidden, hidden→motor output. Overlapping vertices between source/target were excluded as predictors. FC estimated per subject and averaged to group weights. Activity flow simulations (multi-step): For each subject, simulated 960 pseudo-trials (64 contexts × 15 random stimulus combinations). Inputs: context activation pattern as mean of the three rule vectors; stimulus activation from the relevant sensory dimension specified by the context. Predicted hidden-layer activation X_hidden = ReLU(X_context W_context2hidden + X_stimulus W_stimulus2hidden). Predicted motor output X_output = X_hidden W_hidden2output. For each subject, averaged predicted motor activations over pseudo-trials to 4 prototypical responses (left index/middle, right index/middle). Evaluation: Trained a minimum-distance decoder on predicted motor patterns and tested on actual GLM-derived motor activations in held-out subjects (4-fold CV; bootstrapped training). Assessed significance with permutation tests (label shuffling; 1000 iterations), reporting mean accuracy and p-values. Control/lesion models: (a) Remove conjunction hubs (direct input→output mapping). (b) Replace hubs with random parcels (1000 samples). (c) Vary number of hubs (top 20/30/40 by ANN similarity). (d) Remove nonlinearity (no ReLU). (e) Lesion context inputs (set rule→hidden FC to zero). (f) Shuffle FC within input layers (1000 configurations).

- ENN performance predicting motor responses from rule and sensory activations was above chance: right hand decoding accuracy 62.65% (p=0.03), left hand 77.58% (p<0.001), when training on predicted outputs and testing on actual motor activations. - Baseline decodability of actual motor activations (without modeling transformations) was high: RH 82.81% (p<1e-6), LH 87.98% (p<1e-6). - Conjunction hubs were necessary: removing hubs reduced performance to chance (RH 48.98%, p=0.54; LH 50.14%, p=0.45). - Identifying hubs via ANN-brain RSM similarity mattered: random hub sets (n=1000) yielded chance mean performance (RH 50.89%, p=0.45; LH 50.85%, p=0.47), with ANN-matched hubs outperforming 83.3% (RH) and 96.4% (LH) of random selections. - Number of hubs: top 20 regions maintained above-chance performance (RH 63.90%, p<0.001; LH 76.95%, p<0.001); top 30 reduced RH accuracy (59.83%, p=0.024) and LH fell below chance (43.54%, p=0.917); top 40 not above chance. - Nonlinearity critical: removing ReLU led to chance-level performance (RH 50.72%, p=0.44; LH 45.73%, p=0.75). Representational similarity to ANN hidden layer increased with ReLU (cosine 0.60) vs without (0.44). - Context inputs required: lesioning rule→hidden connections yielded chance (RH 50.00%, p=0.46; LH 50.00%, p=0.46). - FC topography mattered: shuffling connectivity weights produced high-variance but chance-mean performance (RH mean 51.22%, p=0.44; LH mean 50.77%, p=0.47). The empirical resting-state FC performed better than 86.4% (RH) and 97.0% (LH) of shuffled configurations. - Rule representations were widespread, but dorsal attention network exerted disproportionate influence on generating conjunctive activations via FC. - Sensory feature decoding localized in visual and auditory networks; motor responses localized to somatomotor finger representations, validating GLM separability and ENN input/output selections.

The study provides evidence that flexible cognitive control relies on transforming distributed rule and sensory representations into motor outputs via conjunction hubs, consistent with and expanding the Guided Activation and Flexible Hub theories. By constructing an ENN directly from fMRI-derived activations and resting-state FC, the authors show that intrinsic network architecture can implement nontrivial, context-dependent sensorimotor transformations without supervised fitting of inter-area weights to behavior. Conjunction hubs and nonlinear combination (ReLU) of rule and sensory inputs were essential for above-chance task performance, highlighting the necessity of conjunctive representations for conditional action selection. The specific resting-state FC topography was particularly effective among many possible connectivity patterns, underscoring the computational relevance of intrinsic functional organization. The findings refine theoretical expectations about where context and conjunction representations reside (e.g., widespread context coding with strong involvement of dorsal attention and control networks) and suggest that additive activity flow with a nonlinearity can implement biased-competition-like computations to achieve flexible behavior.

An empirically-estimated neural network model assembled from human fMRI activations and resting-state functional connectivity can perform context-dependent sensorimotor transformations by integrating task rules and sensory stimuli in conjunction hubs to generate motor responses. This validates and extends the Guided Activation framework by specifying where and how its components are implemented in the brain and demonstrates that intrinsic network architecture is sufficient to support representational transformations without task-optimized learning. Future work should explore richer architectures with multiple hidden stages and recurrent dynamics, adjudicate additive versus multiplicative interaction schemes, incorporate subcortical circuits, and build individualized, temporally resolved models to link computations to behavioral variability.

- Model architecture simplicity: only one hidden layer of conjunction hubs; actual brain may involve multi-stage transformations across hierarchies. - Temporal resolution: BOLD-based analyses may miss fast emergence of conjunctive representations; recurrence and temporal dynamics not explicitly modeled. - Additive interaction assumption: activity flow used additive connectivity weights with ReLU; alternative multiplicative schemes not tested here. - No explicit recurrence: recurrent feedback known to be important was not included; persistent rule activity was approximated by holding encoding activations constant. - Group-level modeling: ENN validated at group level; individual variability and individualized models not addressed. - Potential omission of finer-grained processes: did not model early sensory feature extraction or cortico-subcortical loops (e.g., basal ganglia) that may contribute to action selection. - Large space of alternative models remains; some shuffled FC or random hub configurations can occasionally perform above chance, indicating non-uniqueness of solutions.