Measuring exposure to misinformation from political elites on Twitter

The study addresses how to measure and understand exposure to misinformation originating from political and organizational elites on social media. Prior work has emphasized belief in and sharing of content from reliable vs unreliable news domains, but has largely overlooked what content users are exposed to based on whom they follow. Because exposure and sharing are distinct—most content seen is not shared—and mere exposure can increase perceived accuracy, the authors focus on users’ choices of which elite accounts to follow as a determinant of their information environment. They introduce a measure of exposure to elite misinformation by leveraging fact-check-based falsity scores for elites (from PolitiFact) and aggregating these scores over the elites each user follows on Twitter. The aim is to provide researchers with a tool to study exposure (not just sharing), to relate exposure to other behaviors and attributes (e.g., ideology, toxicity), and to help control for exposure when studying sharing. The approach is motivated by limitations of domain-based quality measures and the need for an exposure-focused metric that does not rely on link sharing, is updatable, and is applicable beyond Twitter.

The paper situates its contribution within research on online misinformation that often uses domain-level ratings (blacklists or continuous quality ratings by fact-checkers or crowds) to assess the quality of content people share or consume. While informative, domain lists suffer from incompleteness, rapid turnover of misinformation sites, lack of consensus update criteria, and a focus on behaviors post-exposure (sharing/clicking). Moreover, they capture only link-containing posts, missing much content. The authors build on follower-based inference of partisanship using elites and adapt it to misinformation exposure, emphasizing the need to study exposure independent of sharing and to use readily updateable fact-check databases. Methodological reviews and prior findings cited include evidence that conservatives tend to consume/share more misinformation, that exposure enhances perceived accuracy, and that elite cues shape public opinion.

Data sources and elite falsity scoring: The authors retrieved PolitiFact fact-check data for 1005 elites (politicians, bureaucrats, personalities, advocacy groups, media organizations) that had at least three fact-checks as of October 28, 2020. PolitiFact assigns six ratings: True, Mostly True, Half True, Mostly False, False, Pants on Fire. They mapped these to veracity scores (True=1, Mostly True=0.8, Half True=0.6, Mostly False=0.4, False=0.2, Pants on Fire=0), averaged per elite, then converted to a falsity score by subtracting from 1. Results are robust to alternative scoring schemes (Supplementary Table 4).

Linking elites to Twitter and user sampling: They identified 950 Twitter accounts corresponding to 816 of the 1005 elites (some elites have multiple accounts; robustness checks exclude such cases and organizations). They collected all followers of these 950 accounts (N=122,562,681 unique users). For analysis, they constructed a list of users following at least three rated elites (N=38,328,679) and drew a random sample of 5000 users. They could not retrieve followed accounts for 650/5000 due to protections or account deletion.

Misinformation-exposure score: For each sampled user, they computed the exposure score as the average falsity score of all followed rated elite accounts, weighting each elite by the account’s average number of tweets per 2-month period over the past 2 years to proxy intensity of exposure. Findings are robust without weighting (Supplementary Table 4).

Political ideology estimation: They estimated users’ ideology from the political accounts they follow using the follower-based method of Barberá et al., yielding a continuous score in [-2.5, 2.5] (negative liberal, positive conservative). They used 0 as the liberal/conservative split. They also estimated ideology from media sharing in robustness analyses (using the AP outlet as the neutral cut-off).

Quality of shared content: For each user, they retrieved up to the last 3200 tweets as of July 23, 2021 (Twitter API limit). They identified tweets containing URLs and matched domains to a list of 60 news websites rated on trustworthiness [0,1] by eight professional fact-checkers and by a politically balanced U.S. layperson crowd. Of 5,363,779 total links, 5% matched the rated list. Per-user sharing quality is the average trustworthiness of matched links. They could not retrieve timelines for 837/5000 users (protected/no tweets/deleted).

Language characteristics: They measured average language toxicity using Google Jigsaw Perspective API and expressions of moral outrage using a published estimator.

Network analyses: They constructed a co-share network where nodes are domains shared by at least 20 users and edge weights are the count of users who shared both domains. For each domain, they averaged over sharers’ estimated ideology, misinformation exposure, toxicity, and moral outrage. They also built co-follower and co-retweet networks (see Supplementary). Community detection employed standard algorithms (e.g., Louvain), and node layouts used force-directed placement.

Statistical analysis: Main analyses used linear regression with standardized coefficients; all tests two-tailed. They examined associations between misinformation exposure and (i) sharing quality (fact-checker and balanced crowd ratings), (ii) estimated ideology, (iii) toxicity and moral outrage, and (iv) network community characteristics. They also modeled misinformation exposure as a function of ideological extremity (absolute ideology) and its interaction with binary ideology (liberal vs conservative), using both follower-based and sharing-based ideology classifications. Extreme values were winsorized to the 95% quantile for visualization as noted in figures. Data collection used Twitter API and Python; ethics review waiver obtained (MIT COUHES E-3973).

• Misinformation exposure vs sharing quality: Higher elite misinformation-exposure scores are associated with sharing from lower-quality outlets.

  • Using professional fact-checker ratings: b = -0.728, 95% CI [-0.753, -0.704], SE = 0.013, t(3072) = -58.184, p < 0.001.
  • Using politically balanced layperson ratings: b = -0.540, 95% CI [-0.570, -0.510], SE = 0.015, t(3072) = -35.299, p < 0.001.
  • The association remains robust controlling for estimated ideology:
    • Fact-checker ratings: b = -0.712, 95% CI [-0.751, -0.673], SE = 0.020, t(3067) = -36.008, p < 0.001.
    • Crowd ratings: b = -0.565, 95% CI [-0.613, -0.518], SE = 0.0124, t(3067) = -23.387, p < 0.001.
  • Exposure explains substantial variance in sharing quality: 53% (fact-checkers) and 29% (crowd ratings).
  • Controlling for exposure markedly reduces and renders insignificant the effect of estimated ideology on sharing quality (e.g., coefficient decreases from -0.548 to -0.021 for fact-checkers; from -0.388 to 0.030 for crowd ratings; p-values increase to 0.265 and 0.191 respectively).

• Misinformation exposure vs ideology: Exposure is positively correlated with estimated conservative ideology: b = 0.747, 95% CI [0.727, 0.767], SE = 0.010, t(4332) = 73.855, p < 0.001.

• Language characteristics: Higher exposure is associated with more toxic language and greater moral outrage expression.

  • Toxicity: b = 0.129, 95% CI [0.098, 0.159], SE = 0.015, t(4121) = 8.323, p < 0.001; controlling for ideology: b = 0.319, 95% CI [0.274, 0.365], SE = 0.023, t(4106) = 13.747, p < 0.001.
  • Moral outrage: b = 0.107, 95% CI [0.076, 0.137], SE = 0.015, t(4143) = 14.243, p < 0.001; controlling for ideology: b = 0.329, 95% CI [0.283, 0.374], SE = 0.023, t(4128) = 14.243, p < 0.001.

• Co-share network communities (1,798 domains with ≥20 sharers): Three clusters emerged (liberal, center-left/moderate, conservative). Average exposure scores differed by cluster:

  • Cluster 1 (liberal): exposure = 0.389; average ideology = -0.470.
  • Cluster 2 (center-left/moderate): exposure = 0.404; average ideology = 0.038.
  • Cluster 3 (conservative): exposure = 0.506; average ideology = 1.22. Differences in exposure across clusters are significant with and without controlling for ideology (Tukey HSD, p < 0.001). Users in the moderate cluster show lower toxicity (0.159) and moral outrage (0.170) than liberal (toxicity 0.186; outrage 0.213) and conservative clusters (toxicity 0.199; outrage 0.226) (p < 0.001).

• Ideological extremity asymmetry: More ideologically extreme users have higher exposure, with a stronger association among conservatives than liberals.

  • Interaction (conservative × extremity):
    • Using follower-based ideology: b = 0.756, 95% CI [0.726, 0.786], SE = 0.015, t(4330) = 49.871, p < 0.001.
    • Using sharing-based ideology: b = 0.415, 95% CI [0.367, 0.462], SE = 0.024, t(3100) = 17.101, p < 0.001.
  • Decomposed slopes:
    • Conservatives: b = 0.825 (followers-based), 95% CI [0.804, 0.846], SE = 0.010, t(2852) = 77.97, p < 0.001; b = 0.567 (sharing-based), 95% CI [0.523, 0.610], SE = 0.022, t(1381) = 25.508, p < 0.001.
    • Liberals: b = 0.160 (followers-based), 95% CI [0.110, 0.211], SE = 0.025, t(1478) = 6.255, p < 0.001; b = 0.111 (sharing-based), 95% CI [0.065, 0.159], SE = 0.023, t(1719) = 4.659, p < 0.001.

• Tooling: The authors released an open-source R package and API to compute misinformation-exposure scores and provided falsity ratings and code publicly.

The findings demonstrate that users’ choices of which elites to follow are strongly linked to the quality of information they subsequently share and to linguistic markers of toxicity and moral outrage. This aligns with the notion that elite cues shape followers’ attitudes and behaviors and underscores the importance of the exposure environment in misinformation dynamics. The exposure measure isolates the effect of following inaccurate elites independent of political ideology and explains substantial variance in sharing quality, suggesting that controlling for exposure is critical in studies of sharing. Network analyses reveal communities consistent with both partisan echo chambers and “falsehood echo chambers,” where higher-misinformation users coalesce around similar elites and domains. The asymmetry whereby ideological extremity relates more strongly to exposure among conservatives mirrors prior work on misinformation sharing and indicates heterogeneity in how ideology interacts with misinformation environments. The openly available tool enables researchers to quantify exposure in observational studies and field experiments, examine predictors of following misinformation-prone elites, and evaluate interventions aimed at improving users’ information diets.

This work introduces and validates a scalable method to quantify Twitter users’ exposure to misinformation from elites by combining follower data with fact-check-derived falsity scores. The measure correlates with lower-quality sharing, conservative ideology, higher toxicity, and moral outrage, and reveals community-level structure with higher exposure among conservative-aligned domains. An open-source R package and API operationalize the approach for broad use. Future research should: (i) test causal effects of exposure to inaccurate elites on beliefs and behaviors; (ii) extend falsity ratings beyond PolitiFact (e.g., other fact-checkers, crowdsourced systems like Birdwatch) to assess robustness and mitigate potential rating biases; (iii) generalize to other platforms and incorporate exposure beyond direct follows (retweets, algorithmic recommendations); (iv) address users with few followed rated elites (e.g., precision-weighted estimates); and (v) expand sharing-quality measurement beyond a limited set of rated domains.

• Reliance on PolitiFact: Potential biases in which claims/elites are fact-checked and in evaluations may influence falsity scores and observed partisan asymmetries.
• Coverage of rated elites: Users must follow a sufficient number of rated elites to receive reliable scores, introducing selection effects; 189 elites lacked Twitter accounts; some users/accounts were inaccessible or deleted.
• Exposure scope: The measure captures exposure via followed accounts only, not indirect exposure through retweets, replies, friends, or algorithmic recommendations.
• Sharing-quality coverage: Only about 5% of shared links matched the rated domain list, limiting sharing-quality assessment; results may differ with broader link coverage or alternative metrics.
• Platform generalizability: Analyses focus on Twitter, which is not representative of the general population; external validity to other platforms or populations may be limited.
• Temporal dynamics and domain churn: While the elite-based approach is updatable, fact-check coverage and elite behavior can change over time, potentially affecting stability of scores.