Diverse misinformation: impacts of human biases on detection of deepfakes on networks

The research explores the complex interplay between human biases and the spread of misinformation, specifically focusing on deepfakes. Social media platforms often assume users can self-correct against misinformation; however, individual biases significantly impact susceptibility. The authors define "diverse misinformation" as the intricate relationship between human biases and the demographic representation within misinformation. Deepfakes are chosen as a case study due to their objective classification as misinformation, controllable demographic attributes of the personas presented, and their real-world harmful implications. The study aims to investigate how users' biases influence their susceptibility to deepfakes and their capacity to correct each other. The overall importance lies in understanding how individual biases impact the spread and correction of misinformation at scale, shaping a population's ability to self-correct. The study leverages an observational survey and a mathematical model to address these questions.

The paper reviews existing literature on distributed harm in online social networks, focusing on misinformation and the detection of computer-generated content. It highlights that vulnerability to harmful online content is not uniform across users, but rather is shaped by individual biases. The literature shows that homophily bias—a preference for interacting with similar individuals—influences the spread of misinformation, particularly within politically aligned groups. Previous work also demonstrates that biases impact accuracy in eyewitness accounts, such as the own-race bias (ORB) phenomenon. The paper also notes the existing efforts to detect and mitigate misinformation online, ranging from automated detection techniques to crowdsourced correction methods. Finally, the ethical considerations of deepfakes are addressed, including their potential impact on legal frameworks, consent issues, and the degradation of the epistemic environment.

The study employed an IRB-approved online survey (N=2016) using video clips from the DeepFake Detection Challenge (DFDC) Preview Dataset. Participants were not explicitly informed about the potential presence of deepfakes; instead, the survey was framed as a study on communication styles. This deceptive design aimed to mirror real-world scenarios where users encounter deepfakes organically. The survey assessed participants' ability to identify deepfakes and explored the relationship between participant demographics and their perception of video personas. The analysis focuses on the relationship between classification accuracy and demographic features of both videos and participants, avoiding interpretation of potential algorithmic biases in the video creation process. The researchers used the Matthews Correlation Coefficient (MCC) to measure the accuracy of deepfake detection and employed bootstrapping to test the credibility of observed biases. A Bayesian logistic regression was also used to explore the effects of matching demographics on detection accuracy. The mathematical model, inspired by epidemiological models, incorporates demographic heterogeneity, community structure (mixed-membership stochastic block model), susceptibility to misinformation, and the concept of "herd correction," where diverse social groups can mitigate misinformation spread. The model simulates the spread of multiple, independent streams of misinformation targeting different demographic groups.

The survey results revealed that unprimed participants had a 51% accuracy rate in detecting deepfakes, only slightly better than chance. However, accuracy varied significantly based on demographics. Specifically, strong evidence was found for a homophily bias: white participants were more accurate at classifying videos of white personas than those of persons of color; male participants were more accurate when classifying videos of male personas; and participants of color were more accurate when classifying videos of persons of color. Furthermore, younger participants (18-29 years old) demonstrated higher accuracy, even when evaluating videos of older personas. The mathematical model demonstrated that diverse neighborhoods can be protective against misinformation. In homogenous populations, misinformation spreads easily unless the correction rate exceeds a critical threshold. In heterogeneous populations, a steady state of misinformation is maintained, depending on the parameters and demographic composition. The model showed that highly susceptible individuals within homogeneous neighborhoods (echo chambers) are at the highest risk, while individuals with diverse neighborhoods are more likely to be corrected by their peers.

The findings highlight the significant influence of human biases on the detection of deepfakes and other forms of diverse misinformation. The low accuracy rate of unprimed participants underscores the challenge of relying solely on individual users to self-correct against such misinformation. The demographic biases observed are consistent with the own-race bias phenomenon and suggest the need for targeted interventions to address these disparities. The mathematical model supports the hypothesis that diverse social networks can enhance the self-correcting capacity of online communities, offering a potential strategy to mitigate misinformation spread. These findings have implications for the design of interventions aimed at improving deepfake detection and reducing the societal harm caused by misinformation. Future research should explore strategies to enhance individuals' ability to detect deepfakes and understand the mechanisms behind observed demographic biases.

This study demonstrates the significant impact of human biases on deepfake detection and the potential for diverse social networks to mitigate misinformation. The low accuracy of unprimed participants highlights the limitations of relying on individual users for self-correction. Future work should explore the effectiveness of interventions like education and machine-assisted detection in improving accuracy and addressing demographic disparities. Further research could explore the dynamics of misinformation spread in real-world social networks with varying levels of diversity.

The study's limitations include the use of a convenience sample from a Qualtrics panel, which might not fully represent the diversity of the broader US population. The deceptive nature of the survey, while designed to reflect real-world encounters, could have influenced participant responses. The mathematical model is a simplification of complex social network dynamics, and further research is needed to validate its predictions in real-world settings. Finally, the focus on deepfakes might not fully generalize to other forms of misinformation.