Susceptibility to online misinformation: A systematic meta-analysis of demographic and psychological factors

This study investigates who is susceptible to online misinformation and why, focusing on the ability to distinguish true from false news (discrimination ability) versus general tendencies to judge news as true or false (response bias). The authors synthesize individual participant data from 31 US-based news-headline experiments (256,337 veracity judgments; 11,561 participants) using signal detection theory (SDT) to disentangle these mechanisms. They examine four demographic factors (age, gender, education, political identity) and four psychological factors (analytical thinking, ideological congruency, motivated reflection, familiarity). Based on prior literature, they hypothesized higher discrimination ability for older adults, those with higher formal education, and Democrats (vs. Republicans), and no effect of gender; higher discrimination ability for higher analytical thinking and unfamiliar headlines; and a true-news bias for ideologically congruent and familiar headlines. The work aims to inform targeted interventions and resolve conflicting findings by accounting for both discrimination and response bias.

Prior studies show mixed and sometimes conflicting results on who is susceptible to misinformation. Age: Although older adults visit more low-quality sites and share more misinformation, several studies suggest they are as good or better than younger adults at veracity judgments. Gender: Findings are mixed; some report null effects, others find females perform worse (though with methodological limitations that conflate bias and discrimination). Education: Often linked to critical thinking and better judgments, yet evidence suggests traditional education may not equip individuals for digital contexts, and even highly educated individuals can be misled by surface cues. Political identity: Research often finds Democrats outperform Republicans in assessing veracity, possibly due to differing information ecosystems. Analytical thinking: Rooted in dual-process theory, higher analytical thinking generally improves discrimination and may increase a false-news bias (caution). Ideological congruency: Participants tend to judge ideologically congruent headlines as true and incongruent as false, affecting response bias. Motivated reflection: Higher analytical thinkers may show greater congruency effects, though evidence is mixed and operationalizations vary (motivated numeracy/reasoning/reflection). Familiarity: The illusory truth effect is robust—familiar information is more likely judged true, primarily affecting response bias and sometimes reducing discrimination. Across these domains, much prior work conflates accuracy with response bias, underscoring the need for SDT to distinguish mechanisms for better theory and interventions.

The meta-analysis was preregistered (OSF: https://osf.io/s2ejg/) and followed PRISMA guidelines. Searches across Web of Science, Scopus, PsycINFO (2004–Nov 2022), plus PsyArXiv and mailing lists, yielded 4,666 records; 21 articles (31 studies) met eligibility criteria. Inclusion criteria: US-based adult samples; English-language headlines; experimental news-headline paradigm with individual, one-time veracity judgments of both true and false headlines; veracity framing as accuracy/real/fake/trustworthy/believable/credible/manipulative; binary or even-numbered Likert responses; no immediate feedback; data available. Exclusions included sharing-intention outcomes and rerating designs. Data extraction covered study/participant characteristics, headline features (veracity, ideological leaning), demographics, psychological measures, and judgments. Preprocessing standardized response formats (binarization), education recoded into secondary/undergraduate/graduate and converted to ridit scores; missing data were mean-imputed for education (10.71%), political identity (6.52%), analytical thinking (31.3%), and familiarity (80.23%; excluded from main model, analyzed separately). Ideological congruency required headline leaning; for studies lacking this, GPT-4 categorized headline leanings into seven levels (strongly Republican to strongly Democratic), validated with high agreement (Cohen’s kappa 0.78), and combined with participant identity (Democrat/Republican) to compute congruency. Main analysis used a Bayesian generalized linear mixed-effects model (GLMM) for SDT with a Bernoulli outcome (veracity_response_binary) and probit link (brms in R). Predictors: item_veracity (true/false), age, gender (male/female), education ridit score, political identity (Democrat/Republican), analytical thinking (CRT score 0–1), ideological congruency (7-level alignment), and motivated reflection (CRT × congruency). Random effects: participant intercept and item_veracity slope; study intercept with random slopes for fixed effects; headline intercept. Predictors were mean-centered; continuous predictors divided by two SDs. Four MCMC chains (10,000 iterations; 5,000 burn-in) with convergence checks (Rhat, visual). The intercept reflected response bias; interactions with item_veracity indexed discrimination ability. Familiarity was analyzed in a separate complete-case SDT model (N=2,619; 50,701 choices), adding familiarity_binary (unfamiliar/familiar) as predictor and study random intercept with familiarity slope. Additional analyses tested headline topic (politics/COVID-19/health), platform (Lucid vs MTurk), and source display (present vs not). All anonymized data were deposited on OSF.

Overall performance and baseline SDT: Participants showed above-chance discrimination ability (β=1.19, 95% CI: 1.03–1.36), with overall accuracy 67.87%; true-headline accuracy 68.51%, false-headline accuracy 67.24%. Baseline response bias was not credibly different from zero (β=−0.03, CI: −0.11–0.05). Age: Older age increased discrimination ability (β=0.38, CI: 0.29–0.47) and was associated with a false-news bias (β=−0.19, CI: −0.24 to −0.14), yielding higher accuracy for false than true headlines among older adults. Gender: No credible effect on discrimination (β=0.01, CI: −0.05–0.07); females showed a small false-news bias (β=−0.07, CI: −0.12 to −0.03). Education: No credible effect on discrimination (β=0.07, CI: 0.00–0.13); a small true-news bias (β=0.10, CI: 0.05–0.14) led to higher accuracy for true and lower for false headlines with more education. Political identity: Republicans had lower discrimination ability than Democrats (β=−0.42, CI: −0.51 to −0.32) and a small true-news bias (β=0.12, CI: 0.06–0.17). Analytical thinking (CRT): Strong positive effect on discrimination (β=0.66, CI: 0.52–0.81) and a false-news bias (β=−0.19, CI: −0.26 to −0.12). Ideological congruency: No credible effect on discrimination (β=0.10, CI: −0.01–0.20); strong true-news bias (β=0.29, CI: 0.22–0.37), increasing belief in congruent and disbelief in incongruent headlines. Motivated reflection (CRT × congruency): No discrimination effect (β=0.04, CI: −0.06–0.14); small positive effect on response bias (β=0.16, CI: 0.09–0.24), indicating stronger congruency-induced true-news bias among higher CRT scorers. Familiarity (complete-case): No credible effect on discrimination (β=0.16, CI: −0.03–0.30); strong true-news bias (β=1.03, CI: 0.67–1.34), increasing accuracy for familiar true and decreasing for familiar false headlines. Additional analyses: Headline topic (political/COVID-19/health) did not affect discrimination; MTurk vs Lucid—MTurk associated with higher discrimination (β=0.60, CI: 0.52–0.68); Source display increased discrimination (β=0.44, CI: 0.15–0.74), with Republicans benefiting more (β=0.17, CI: 0.09–0.26).

Disentangling discrimination ability from response bias clarifies who is susceptible to online misinformation and why. Older adults demonstrated better discrimination and a cautious false-news bias, challenging assumptions about digital naïveté and suggesting accumulated knowledge, heuristics, or source-use strategies may underlie their advantage. Younger adults showed lower discrimination and more naïveté, raising concerns about vulnerability despite higher digital literacy. Formal education did not improve discrimination and was linked to a true-news bias, highlighting potential gaps in current educational practices for digital contexts. Analytical thinking robustly improved discrimination and increased caution, reinforcing the role of deliberation and suggesting interventions to boost analytical processing could enhance veracity judgments. Political identity showed marked asymmetry: Democrats exhibited higher discrimination than Republicans, consistent with research on divergent information ecosystems and exposure patterns. Ideological congruency primarily shifted response bias toward believing congruent information, underscoring identity-driven processing and the need to promote evaluation independent of partisan alignment. Motivated reflection showed a small but credible effect, aligning higher analytical skill with stronger congruency-induced true-news bias; however, its modest size and mixed prior evidence warrant caution. Familiarity exerted a large true-news bias, emphasizing risks in environments where repetition and echo chambers amplify perceived truth. Platform differences (MTurk > Lucid) and benefits of source display (especially for Republicans) indicate methodological and design features can shape veracity judgments. Overall, distinguishing mechanisms informs tailored interventions—some factors improve discrimination (age, analytical thinking, source display), whereas others primarily shift bias (education, ideology, familiarity)—with implications for designing effective, targeted approaches to mitigate misinformation.

This preregistered IPD meta-analysis across 31 US-based experiments advances understanding of misinformation susceptibility by separating discrimination ability from response bias using SDT. It identifies robust predictors: older age and higher analytical thinking increase discrimination (cautionary false-news bias), while Republican identity, education, ideological congruency, motivated reflection, and familiarity predominantly shift response bias toward believing headlines. MTurk samples and source display further enhance discrimination, with source cues benefiting Republicans more. The findings resolve key debates, inform individualized and mechanism-specific interventions, and highlight the need for more representative, ecologically valid research. Future work should systematically include relevant variables, move beyond overall accuracy to SDT-like frameworks, incorporate reaction times and richer stimuli, expand beyond headlines and WEIRD samples, and improve data coding and open science practices to strengthen generalizability and intervention efficacy.

Headline stimuli may not represent the full spectrum of online news; studies typically used equal numbers of true and false headlines, unlike real-world base rates. Not all studies measured every demographic and psychological factor, necessitating mean imputation for some variables (education, political identity, analytical thinking) and a separate complete-case model for familiarity due to substantial missingness. The sample is US-only, with simplified binary coding of gender and political identity that excludes Independents and nuances. Response bias varied across studies, potentially reflecting study-level features such as headline selection and experimental demands. Additionally, older adults participating in online studies may be more digitally literate than the broader population, which could influence age-related effects.