Debates about vaccines and climate change on social media networks: a study in contrasts

The study investigates how public debates on vaccines and anthropogenic climate change unfold on Twitter, despite strong scientific consensus in both domains. The research questions whether the distribution of sentiments and the organization of user and community networks are similar across these two topics. Motivated by evidence of polarization and echo chambers online, and by the relevance of both topics as coupled human–environment systems and common-pool resource problems, the authors hypothesized that vaccine and climate conversations would show similar sentiment distributions and network structures. They analyze tweets, users, and interactions to test this hypothesis across increasing organizational scales.

Prior work shows extensive use of Twitter to examine opinion spread, polarization, and echo chambers in topics including vaccines and climate change. Studies link Twitter sentiment to real-world behaviors such as vaccination uptake and infectious disease trends, clinical visits, and stock market movements. Echo chambers and polarization have been documented for both climate change and vaccines, with evidence that climate discussions became more polarized after major reports. Both vaccine-induced herd immunity and climate mitigation are common-pool resources prone to free-riding, embedded in coupled human–environment systems. However, few studies directly compare disparate topics like vaccines versus climate change to contextualize social interactions and test hypotheses across systems.

Data: Two GNIP (Twitter) datasets were purchased. GNIP vaccine dataset: 62.11M tweets from 2006-06-01 to 2016-10-26 using extensive vaccine-related search terms. GNIP climate dataset: 18.06M tweets from 2015-04-27 to 2016-10-26 using “climate change” OR “global warming.” Additionally, API-collected datasets (2016-10-26 to 2018-09-20) captured 4.40M vaccine and 2.71M climate tweets. Retweet/link proportions are reported for each dataset. To control for search term breadth, a subsampled vaccine set using only “vaccine OR flu OR influenza” was also analyzed.

Tweet sentiment labeling: Mechanical Turk taggers annotated randomly sampled tweets into positive, negative, or neutral (climate third set also included a news label, later merged with neutral). Agreement of at least 2 of 3 taggers was required. Final training sets: vaccine 73,268; climate set 1: 74,895; climate set 2: 75,976 (including added negatives and a news class merged into neutral for modeling). Tagger criteria: pro-vaccine if vaccines safe/effective; anti-vaccine if harmful/ineffective or diseases not dangerous; neutral otherwise. For climate: pro if human-caused climate change is real; anti if not real/not human-caused/not a problem; neutral otherwise.

Machine learning: Two approaches were used. (1) An Ensemble Averaging model combining RNN-GRU, RNN-LSTM (TensorFlow, GloVe glove.twitter.27B.200d embeddings), logistic regression (scikit-learn), and LightGBM. Climate ensemble validation accuracy 78.5% with F1: Anti 0.82, Neutral 0.62, Pro 0.83, News 0.83 (news later merged into neutral). Vaccine ensemble validation accuracy 85.1% with F1: Anti 0.76, Neutral 0.90, Pro 0.77. (2) Support Vector Machine (scikit-learn), trained on balanced subsets. Vaccine SVM: precision 0.80/0.90/0.79; recall 0.83/0.82/0.82; F1 0.82/0.86/0.81 (neg/neu/pos). Climate SVM: precision 0.72/0.75/0.58; recall 0.74/0.51/0.76; F1 0.73/0.61/0.66. Ensemble results are used primarily due to higher F1.

User sentiment classification: User sentiment was assigned by plurality of their tweets’ classified sentiments (ties labeled undecided, rare). Validation against human expert labeling of 100 randomly sampled users (up to 100 tweets per user) showed 87% agreement for climate and 84% for vaccines. A sensitivity analysis required 80% of a user’s tweets to share the same sentiment for assignment, to reduce mixed or potentially misclassified/bot accounts.

Network construction: Retweet and mention networks were built as directed and mutual graphs (mutual edges require reciprocal interaction). Edges were weighted by interaction counts. Mention networks were analyzed both with and without counting the “mentioned user” in retweets. Analyses focused on directed and mutual networks due to coverage of behaviors. Users were stratified by activity thresholds (>5, >10, >25, >50, >100 tweets). Network metrics (NetworkX) included connectivity, average shortest path length, degree metrics and entropy, and assortativity by sentiment. Community detection used the Louvain algorithm (Python-Louvain), and community measures included sizes and Shannon entropy (heterogeneity). Graphs were visualized in Gephi. Robustness checks covered dataset type (GNIP vs collected), classifier (Ensemble vs SVM), user sentiment assignment (plurality vs 80%), subsampling of vaccine tweets, and network types (retweet vs mention; directed vs mutual). A total of 260 analysis permutations were computed, with results archived in Dataverse Dataset 4.

Sentiment distributions: Across methods and datasets, climate tweets were predominantly pro-climate, while vaccine tweets were predominantly neutral. For GNIP datasets with Ensemble model: climate 53.18% positive, 9.23% negative, 37.59% neutral; vaccine 15.03% positive, 4.25% negative, 80.71% neutral. For SVM: climate 72.38% positive, 13.72% negative, 13.90% neutral; vaccine 19.90% positive, 10.82% negative, 69.28% neutral. Human-labeled training subsets: climate 80.53% positive, 11.26% negative, 8.21% neutral; vaccine 20.67% positive, 7.75% negative, 71.58% neutral.

User participation overlap and asymmetry: About 1.6M users tweeted at least once in both topics. A larger fraction of climate discussion participants also engaged in the vaccine discussion (38%) compared to the reverse (10%). Among users tweeting 100+ times on both topics, pro-vaccine users were overwhelmingly pro-climate (~98%). Conversely, a notable subset of pro-climate users were anti-vaccine (~11%) or neutral (~13%). For high-frequency users (100+), counts (GNIP): climate positive vs vaccine categories: Positive 948, Neutral 2663, Negative 279, Divided 25 (χ²=308.34, df=9, p<2×10⁻16).

Assortativity and network structure: Vaccine user networks showed substantially higher assortative mixing by sentiment than climate networks across thresholds and network types. Examples (mutual networks, retweets, ensemble/plurality): vaccine assortativity ~0.758–0.766 (thresholds 5–100) vs climate ~0.380–0.226; mentions (mutual): vaccine ~0.485–0.534 vs climate ~0.156–0.231. Directed networks: retweets vaccine ~0.503–0.571 vs climate ~0.134–0.097; mentions vaccine ~0.418–0.439 vs climate ~0.056–0.074. Differences were statistically significant (p<1e-4). Over 260 permutations, average assortativity was higher for vaccines (0.68) than climate (0.44).

Community-level patterns: Vaccine communities were larger on average and, under baseline analysis, exhibited higher Shannon entropy (heterogeneity), yet vaccine networks revealed many isolated neutral communities and clearer separation between pro- and anti-vaccine communities. Example averages (mutual networks, community size/heterogeneity): retweets, vaccine size 6.692–12.449 vs climate 4.125–8.950; heterogeneity vaccine 0.101–0.068 vs climate 0.021–0.013 across thresholds. Visualizations showed climate communities densely interconnected and largely pro-climate, with anti-climate users dispersed, while vaccine communities displayed pronounced clustering and fragmentation with limited cross-sentiment interaction.

Robustness: The higher assortativity in vaccine discussions persisted across user activity thresholds, network types (retweet/mention; directed/mutual), sentiment assignment rules (plurality vs 80%), classifiers (Ensemble vs SVM), datasets (GNIP vs collected), and subsampled vaccine keywords.

Contrary to the initial hypothesis of similar structures, vaccine and climate conversations exhibit distinct network dynamics. Vaccine discussions are more siloed, with users and communities interacting predominantly within the same sentiment, while climate discussions show more cross-sentiment interactions and integrated communities. The authors hypothesize that the spatial scale of the underlying environmental systems shapes these social structures: infectious diseases often affect local areas, leading to localized conversations and fragmented networks with isolated communities; in contrast, climate events operate at larger spatial scales, potentially compelling interaction across differing sentiments and contributing to more mixed online networks. This increased cross-sentiment interaction in climate conversations might facilitate social learning and broader acceptance of anthropogenic climate change, though this mechanism remains to be empirically tested. The predominance of neutral vaccine tweets may arise from immediate, practical, and local communications (e.g., illness reports, clinic announcements) that do not express clear sentiment, whereas climate-related discussions may more readily invoke explicit opinions about causes and solutions. Participation asymmetry suggests climate discussants are more likely to engage with vaccine topics than vice versa. Additionally, pro-vaccine users are overwhelmingly pro-climate, while a subset of pro-climate users are anti- or neutral on vaccines, potentially reflecting a nuanced political or ideological pattern. Overall, assortativity patterns may serve as indicators of social mixing and could potentially act as early warning signals of shifts in social norms.

The study demonstrates strong contrasts between Twitter debates on vaccines and climate change. Vaccine discussions show higher assortativity and fragmentation, with limited cross-sentiment interaction and many isolated neutral communities, while climate discussions are more integrated with greater cross-sentiment engagement. These findings motivated hypotheses linking social network structure to the spatial scale of the associated environmental systems. The authors advocate for comparative, cross-system analyses of social media debates to uncover insights that may be missed when studying topics in isolation and suggest future work to test proposed mechanisms and link online dynamics to real-world behavior.

Key limitations include potential misclassification by machine learning models; constraints of a three-way sentiment classification that may miss nuanced opinions; differences in search term breadth across topics despite subsampling controls; representativeness issues since Twitter users are not demographically representative of the general population; lack of user socio-demographic data; limited analysis of conversational threads or longitudinal exchanges; and reliance on observational data. Although multiple robustness checks were performed (alternative classifiers, thresholds, datasets, and network constructions), unobserved confounders and platform dynamics could still influence results.