Generative propaganda: Evidence of AI's impact from a state-backed disinformation campaign

The study asks whether generative AI, in practice, bolsters state-backed propaganda campaigns by increasing output, broadening content, and maintaining persuasiveness. Public, academic, and policy communities have expressed concern that AI could amplify disinformation and manipulate opinion. While laboratory and experimental work shows AI can produce persuasive and credible text, volatility of real-world inauthentic campaigns has hindered direct assessment. Exploiting the adoption of generative-AI tools by a Russian-affiliated outlet (DCWeekly.org), the authors assess AI’s impact on quantity, scope, and character of propaganda using observational data and a survey experiment.

Prior work indicates generative AI can produce persuasive (e.g., Goldstein et al., Simchon et al.) and credible text (e.g., Jakesch et al., Kreps et al.), and highlights threats to democracy and information ecosystems (Ferrara; Kreps & Kriner; Woolley). Research documents limitations of LLMs in personalized political microtargeting (Hackenburg & Margetts) and discusses hyperpartisan media dynamics (Rae). Additional reports and analyses describe Russian-aligned influence operations leveraging LLMs at scale (Recorded Future) and narrative laundering (Linvill & Warren). Collectively, this literature motivates examining AI’s practical effects within an ongoing state-backed campaign.

Data collection: Using the site’s WordPress REST API, the authors collected a near-complete record of DCWeekly.org posts (URL, posting date/time, full HTML, media links) from June 2021 to November 30, 2023, focusing analyses on April–November 2023 (22,889 articles), bracketing AI adoption (September 20, 2023) and prior to public exposure of the operation. Determining AI adoption: A marked shift on 2023-09-20 from copy-paste duplicates sourced from a small set of hyperpartisan outlets to seemingly original language matched to contemporaneous source stories (same facts, quotes, media) indicated AI use. A leaked prompt in an article explicitly stated generation via OpenAI’s GPT-3 and referenced Fox News. Quantity analysis: Weekly publication counts were compared before/after AI adoption, with local trends shown. To estimate the effect, the authors focused on differences between the more active pretransition period and postadoption period, finding a 2.4x increase in daily production post-adoption versus the active pre-AI period. A regression discontinuity analysis (details in Supplementary Section 3) tested changes in posts per week across periods, confirming a statistically significant increase in quantity after AI adoption. Breadth analysis—topic diversity: The authors fit Latent Dirichlet Allocation (LDA) models separately for pre- and postadoption articles. They varied k = 4, 8, …, 48 topics and selected k = 28 based on high coherence in both periods. For each document, they computed entropy of the topic probability distribution as a measure of topic diversity and averaged within period. Postadoption average entropy was 0.45, nearly double the preadoption average of 0.29. Sensitivity checks across all k confirmed consistently lower preadoption entropy. Breadth analysis—topic focus: Using a pretrained NLI model (BART-large trained on MultiNLI) as a zero-shot classifier, the authors assessed four topics: guns, crime, US domestic news, international news. Mean classification scores shifted from pre-AI (June 2023) to AI-era (October 2023) as follows: guns 0.29 → 0.69; crime 0.61 → 0.83; US domestic news 0.57 → 0.39; international news 0.70 → 0.83. To address confounding from concurrent events (Ukraine and the October 7, 2023 Israel-Hamas war), they regressed guns and crime scores on an AI-era indicator controlling for Israel and Ukraine topic scores, finding sizeable increases for guns and crime even after controls (95% CI shown in Fig. 3). Persuasion and credibility experiment: Preregistered, IRB-approved (Clemson IRB Study ID: IRB2024-0172) 1×2 between-subjects design via Prolific with a demographically balanced US adult sample. Of 892 participants, 880 remained (8 excluded for <90 seconds, 4 failing attention check). Stimuli: 20 DCWeekly articles focused on Russia’s full-scale invasion of Ukraine—10 pre-AI and 10 post-AI—randomly selected via keyword matching; presented using original HTML formatting. Outcomes: persuasiveness (agreement with author’s identified thesis; 1–5 scale, rescaled 0–1) and domain credibility (1–5, rescaled 0–1). Analytic approach included clustered SEs by respondent and 95% CIs; demographic-control robustness shown in Supplementary.

• AI adoption coincided with a 2.4x increase in daily article production compared to the more active pre-AI period; regression discontinuity confirms a significant post-adoption quantity increase. • Topic diversity broadened substantially post-adoption: LDA entropy increased from 0.29 (pre) to 0.45 (post), with consistently lower entropy pre-adoption across k = 4–48. • Topic focus shifted toward international news and issues related to guns and crime: mean NLI scores changed—guns 0.29 → 0.69; crime 0.61 → 0.83; US domestic news 0.57 → 0.39; international news 0.70 → 0.83. • Increases in guns and crime discussion persisted after controlling for Israel and Ukraine topics (linear regression; 95% CI). • Persuasiveness: reading articles from both periods increased agreement with the identified thesis relative to baseline; no significant difference between pre- and post-AI persuasiveness. • Credibility: domain credibility remained statistically unchanged across periods (β = 0.012, P = 0.513). • Operationally, AI appears to have supported multiple production steps (e.g., scoring, selection, translation, rewriting), facilitating scale and breadth without diminishing perceived credibility or persuasiveness.

Findings from a real-world, state-affiliated influence operation indicate that generative AI materially increases the scale of disinformation output while expanding topical breadth and altering focus, yet maintains comparable levels of persuasiveness and perceived domain credibility. This addresses the core question—AI’s practical impact on propaganda—by showing lower production frictions and enhanced coverage without evident trade-offs in efficacy at the article level. Prompt leaks suggest AI was used not only to generate and translate content but also to assist selection and framing via scoring, indicating AI’s value across multiple steps of the propagandist workflow. Although concurrent events (e.g., Israel-Hamas war) and unobserved internal changes could contribute to topical shifts, analyses controlling for Israel/Ukraine salience still show AI-era increases in guns and crime focus, supporting a role for AI in reshaping content mix. Overall, the evidence supports concerns that AI can amplify the size and scope of state-backed campaigns beyond laboratory conditions.

The adoption of generative-AI tools by a state-backed outlet increased volume and breadth of disinformation while preserving article-level persuasiveness and perceived credibility, enabling the construction of a more convincing, professional-seeming news façade. Given rapid improvement of AI systems, tracking and countering such operations will become more difficult as financial and temporal costs continue to fall. Future work should focus on: strengthening methods to prevent misuse of open-source models to augment disinformation; developing sustainable countermeasures against ongoing campaigns; integrating AI-aware methodologies into monitoring; and preparing the public to identify and avoid AI-augmented disinformation. These steps can help mitigate the growing practical impact of generative AI on propaganda.

• Causal attribution: Other internal operational changes may have coincided with AI adoption; the design cannot fully rule out confounding. • Intent and goals: The team’s objectives are unobserved; some campaigns may prioritize flooding over persuasion, affecting quality trade-offs. • Topic selection measurement: Desired topic set is unknown; observed shifts could be incidental to AI integration rather than strategic. • Stimuli identification: For the survey, authors inferred each article’s thesis (intended theses unknown), though baseline agreement did not differ across periods. • Data completeness: A small number of stories appear removed prior to collection; financial costs of production are unobserved. • Event confounding: External events (e.g., Israel-Hamas war) overlapped with the AI era; controls for Israel/Ukraine help, but residual confounding is possible.