Discerning true from false information is crucial in today's information landscape. Equally important is metacognition—awareness of one's own cognitive abilities. This study explores how political partisanship, age, education, and gender influence 1) discernment ability, 2) metacognitive efficiency (metacognitive ability given task performance), and 3) response bias (tendency to answer true or false regardless of veracity). It investigates whether the Dunning-Kruger effect (poor performers overestimating their abilities) manifests in this context. The study's significance stems from the potential link between poor discernment and poor decision-making in various domains (health, political participation), and the role of metacognitive ability in understanding the belief and sharing of misinformation. While individuals often overestimate their resistance to misinformation, this study seeks to identify who accurately assesses their ability to differentiate fact from fiction. Previous research on metacognition and misinformation has yielded mixed results, with some showing low insight, especially in poor performers, and others suggesting good insight. This research also examines demographic differences, noting prior studies that have associated political conservatism and older age with greater engagement with low-quality news. The study aims to clarify these findings by using a robust methodology, employing a signal detection theory-based model to independently measure discernment ability, metacognitive ability, and response bias.

Existing research on metacognition and misinformation shows mixed results. Some studies indicate low insight into the ability to discern true from false information, particularly among those with low actual discernment abilities, highlighting the Dunning-Kruger effect. Others report moderate to high insight. Demographic factors have also been linked to discernment ability and metacognition. Partisanship research suggests that Republicans may perform worse than Democrats in discerning true and false news, although some studies show this is more pronounced among strong partisans. Age is another factor, with older adults sometimes outperforming younger adults in truth discernment, but possibly exhibiting lower metacognitive abilities. Education and gender effects remain less clear. Methodological limitations in prior studies include small sample sizes or reliance on single-item measures, which may lead to unreliable measurement and underestimate participants' insight. This study addresses these limitations by using a more robust method and larger sample size.

533 participants were recruited through Prolific, aiming for equal numbers of men, women, Democrats, and Republicans in four age groups (18–32, 33–47, 48–62, 63+). After informed consent, participants rated the veracity (yes/no) and confidence (1–4 scale) of 140 news headlines (70 true, 70 false) in randomized order. Headlines covered topics reported within one year of data collection and were drawn from various sources and fact-checking websites. A pilot study ensured item quality. The stimuli set was not perfectly balanced for political favorability; a politically equated version was used for certain analyses. Data analysis employed signal detection theory (SDT) to independently measure discernment ability (d’), response bias (c value), and metacognitive ability (meta-d’ and m-ratio). Frequentist and Bayesian analyses were conducted. Three hypotheses were preregistered concerning metacognitive ability and its relation to political partisanship, age, and education. Several exploratory questions investigated the influence of headline political favorability and the relationship between discernment and metacognitive ability. The meta-d' model was employed using a hierarchical Bayesian approach. Statistical tests included ANOVAs and t-tests, with non-parametric alternatives used when data were not normally distributed. Bayes Factors were reported for null results.

Overall, participants achieved high accuracy (78.9% for facts, 81.3% for false statements), with an average d' of 1.82. Democrats demonstrated significantly higher discernment ability (d') than Republicans. This difference was more pronounced among strong partisans: weak Republicans performed better than strong Republicans, while strong Democrats outperformed weak Democrats. Older adults (63+) had significantly higher d' than younger adults (18-32), although this difference was less robust in the politically equated item set. Education was positively correlated with discernment ability, and men scored higher in discernment than women. Metacognitive efficiency (m-ratio) was high overall (M = 0.86), indicating good insight into one's discernment ability across all demographic groups. There were no significant differences in metacognitive efficiency based on partisanship, age, gender, or education, except when using the politically equated set, where Republicans slightly outperformed Democrats. Response bias (c values) showed Democrats exhibiting a slightly greater tendency towards a “false” bias than Republicans. No significant differences were found for age, education, or gender. In analyses of the politically equated dataset, examining the influence of political favorability showed Democrats were significantly more accurate than Republicans. Democrats showed a greater accuracy difference on pro-Republican items versus pro-Democrat items. Republicans displayed slightly higher metacognitive efficiency than Democrats in the equated set, with stronger partisans showing higher efficiency. Republicans also exhibited a "true" bias toward pro-Republican items, while Democrats showed little bias for either item type. Finally, contrary to a predicted Dunning-Kruger effect, the least accurate participants showed the highest metacognitive efficiency.

The findings demonstrate that Democrats outperformed Republicans in discerning true from false news, while older adults and men showed better performance than their younger and female counterparts, respectively. Education played a positive role in accuracy. Crucially, participants across all demographic groups displayed high metacognitive efficiency, indicating good awareness of their performance regardless of accuracy levels. This counters the notion of widespread overconfidence in misinformation judgment, particularly among Republicans. The slight superiority of Republicans in metacognitive efficiency using the equated set suggests that while they are less accurate, they are not necessarily overconfident in their judgments. The findings also challenge the presumed Dunning-Kruger effect, as low performers didn’t demonstrate the lowest metacognitive insight. The study suggests that differences in discernment ability and response bias, rather than metacognitive insight, may drive disparities in misinformation engagement. The differences observed between Republicans and Democrats in bias may partly explain their differences in discernment ability.

This research highlights the importance of considering both discernment ability and response bias when investigating misinformation engagement. The high levels of metacognitive insight found across all groups suggest that interventions focusing on improving discernment skills, rather than metacognitive awareness, may be more effective in combating misinformation. Future research could explore the underlying cognitive and emotional processes contributing to the observed demographic differences, and test the impact of interventions targeting discernment ability and response bias. The generalizability of these findings should be tested using stimuli representing more realistic proportions of true and false information.

The study's limitations include a slightly imbalanced stimuli set (more false news favorable to Republicans in the larger set), potentially influencing response bias. The equal number of true and false items might not fully reflect the real-world information ecosystem. The sample, although large, may not perfectly represent all demographic groups, particularly regarding the representation of political independents and the generalizability of online recruitment of older adults. Future studies should address these limitations.