Hierarchical machine learning models can identify stimuli of climate change misinformation on social media

The study investigates how to automatically detect and categorize climate change misinformation on social media, focusing on Twitter. Given that misinformation undermines support for climate policies, exacerbates polarization, and spreads faster than factual information, there is an urgent need for scalable detection and response tools. Prior automated systems struggle to keep pace with the rapid, platform-specific spread of misinformation and the continued influence effect once falsehoods take hold. The authors aim to improve upon the original CARDS model—which was trained on contrarian blogs and conservative think-tank texts—by developing a hierarchical approach tailored to Twitter. The research questions center on improving binary detection of contrarian versus convinced content, reliably classifying contrarian claim types, and identifying external and platform-level stimuli associated with spikes in contrarian climate claims. The work seeks to inform pre-bunking and targeted interventions by mapping the prevalence and triggers of distinct misinformation categories.

The paper reviews research on the spread and impacts of misinformation on social media, highlighting increased polarization and the role of algorithms in amplifying misleading content. Prior climate-focused analyses on Twitter have documented growing contrarian activity around COP summits, geographic and ideological patterns of denial, and the virality of uncivil or aggressive content. Automated detection efforts have included topic modeling of conservative think-tank texts, linkage of funding to polarizing discourse, and identification of climate framings in news. Broader work on automated fact-checking and logical fallacy detection is noted, including initial attempts to detect fallacies within climate misinformation. The original CARDS model used supervised learning to classify contrarian claims from blogs and think-tank sources but had not been evaluated on social media text, motivating this study’s augmentation and reassessment on Twitter data.

The authors develop Augmented CARDS, a two-step hierarchical model optimized for Twitter. Stage 1 is a binary classifier distinguishing convinced versus contrarian tweets; Stage 2 is a multilabel taxonomy classifier assigning contrarian tweets to CARDS categories. Architecture: Both stages use DeBERTa-large (24 transformer blocks, hidden size 1024, 16 attention heads) with an additional dense classification layer (~355M parameters). Training details: fine-tuned for 3 epochs at learning rate 1e-5 on a V100 GPU, batch size 6, inputs truncated/padded to 256 tokens, fixed seed for comparability. Data for training/evaluation: (1) Original CARDS dataset (contrarian blogs and conservative think-tank articles) for taxonomy training, with category 5.3 (conspiracy theories) separated from 5.2 due to its prevalence on Twitter; (2) Climate Change Twitter Dataset (University of Waterloo; approx. 90/10 verified/misleading) added to the binary classifier training set to improve detection of convinced versus contrarian tweets; (3) Expert Annotated Climate Tweets (2,607 tweets from H2 2022) labeled by climate experts for testing both tasks; (4) A large-scale analysis dataset of ~5 million climate-related tweets collected by the University of Hamburg’s Online Media Monitor (OMM) from July–December 2022 for temporal and content analysis. Analysis of spikes: The authors examined daily tweet volumes and contrarian proportions, then performed word frequency analysis comparing specific intervals to the overall dataset via log-fold change and p-values. Words with significance greater than 0.05 were filtered (retaining only significant terms) and ranked by log-fold change; top 10 terms characterized events. They categorized triggers of contrarian spikes as Natural Events, Political Events, Contrarian Influencers, and Convinced Influencers and analyzed shifts in contrarian taxonomy distributions across these triggers.

- Performance: Augmented CARDS outperformed the original CARDS on Twitter datasets. On Expert Annotated Climate Tweets, Augmented CARDS improved F1 by 16% for binary detection and 14.3% for taxonomy detection, yielding F1 ≈ 81.6 (binary) and 53.4 (taxonomy), while maintaining comparable performance on original CARDS domains. The taxonomy improvement reflects the addition and prominence of category 5.3 (conspiracy) on Twitter. - Category-level gains: Largest improvements occurred in categories most prevalent on Twitter, including 5.2 (climate actors are unreliable), 5.3 (conspiracy theories), 4.1 (policies are harmful), 2.1 (natural cycles), and 1.7 (extreme weather not linked to climate change). - Scale and temporal patterns: The 5M-tweet dataset averaged 27,464 tweets/day on climate. Peaks: late July (65,196 tweets) and mid-November (43,647 tweets). - Contrarian prevalence: Average daily proportion of contrarian tweets was 15.5%. Around President Biden’s climate emergency consideration, the contrarian share peaked at 24.7%. - Triggers of contrarian spikes: Four types identified—Political Events (e.g., Biden’s climate emergency declaration, US Senate bill, US climate ruling), Natural Events (e.g., Hurricane Ian), Contrarian Influencers (e.g., Steve Milloy, Rob Schneider, James Woods), and Convinced Influencers (e.g., Dan Rather, CBS Mornings, David Lammy, Katherine Clark). Political and Natural Events increased overall volume; influencer posts increased the proportion of contrarian tweets without necessarily increasing total volume. - Claim-type distributions: The most common contrarian category on Twitter was 5.2 (attacks on climate actors), comprising about 40% of contrarian tweets, followed by 5.3 (conspiracy theories) at ~20%. Next were 4.1 (policies harmful) and 2.1 (it’s natural cycles), with 1.7 (extremes aren’t increasing) rising during Hurricane Ian. - Trigger-specific shifts (percent changes vs. dataset baseline): Natural Events strongly favored 1.7 with a +680.15% change; Political Events favored 4.1 (+25.2%). Influencer-driven spikes tended toward 5.3 (conspiracy) and 2.1 (natural cycles); contrarian influencers showed stronger increases in conspiracy claims, while convinced influencers slightly shifted toward natural cycles alongside increased conspiracy content. - Activity concentration and automation: Average contrarian tweets per user ranged 1–2; category 5.3 averaged 1.9/user and 5.2 averaged 1.8/user. Some accounts posted hundreds of tweets per category (max examples: 5.3=921; 5.2=881; 4.1=627; 2.1=424; 1.7=108). Approx. 93.3% of contrarian detections were unique content (5.3 ≈ 91.3%, 5.2 ≈ 94.1%); ~6% of analyzed tweets were spam. Evidence of automated accounts was found, and authors caution modern AI-generated content may evade current methods.

Augmented CARDS addresses a core limitation of the original CARDS: difficulty distinguishing convinced from contrarian content when moving beyond known contrarian sources. Incorporating Twitter data and a dedicated binary stage greatly improved transfer to social media and enabled fine-grained analysis of contrarian claim types during spikes. The platform context matters: unlike think-tank articles and blogs (policy- or science-focused), Twitter contrarianism skews toward personalized attacks on climate actors (ad hominem) and conspiracy narratives, aligning with research on the personalization of politics and engagement dynamics on social media. The study delineates predictable patterns in response to different triggers: natural events prompt surges in claims that extreme events are not linked to climate change, while political events increase claims that climate policies are harmful. Influencer posts (both contrarian and convinced) elevate conspiracy and natural-cycle narratives, with contrarian influencers especially amplifying conspiracies. These insights can guide targeted pre-bunking and rapid response strategies tailored to event types and platform dynamics.

The study advances automated detection of climate misinformation on Twitter using a hierarchical, two-stage Augmented CARDS model, improving both binary detection and taxonomy classification compared to the original CARDS when applied to social media. Applying the model to 5 million tweets identified four stimuli for contrarian spikes—political events, natural events, contrarian influencers, and convinced influencers—and revealed that the most prevalent contrarian narratives on Twitter are attacks on climate actors and conspiracy theories. Practical implications include potential integration with platform moderation workflows or user-facing APIs to surface likely misinformation for fact-checking. Achieving automated debunking will additionally require robust detection of logical fallacies and explanatory counter-messaging. The findings can inform preemptive interventions keyed to expected narratives following specific event types.

- Platform scope: Analysis and model evaluation focused on Twitter/X; performance on other sources (e.g., other social platforms, legislative testimony, speeches, video transcripts, newspapers) remains untested. - Language: Training and evaluation were limited to English; generalization to other languages and cultural contexts is unknown. - Time window: The core analysis covers July–December 2022, which may not capture longer-term trends or impacts of subsequent platform changes. - Category imbalance and rarity: Some taxonomy classes are infrequent on Twitter, limiting taxonomy-level performance; larger platform-specific labeled datasets are needed. - Evasion risks: Automated and AI-generated content could circumvent current methods; approximately 6% of analyzed tweets were spam, and evidence of automation was observed.