Behavioral Forensics in Social Networks: Identifying Misinformation, Disinformation and Refutation Spreaders Using Machine Learning

The study addresses the growing problem of misinformation on social networks by moving beyond binary classifications of spreaders (true vs. false) to infer intent. The key research question is how to identify and categorize users into nuanced groups—malicious (intentional spreaders of misinformation), maybe_malicious, naive_self_corrector, informed_sharer, and disengaged—by observing sequences of actions after exposure to both misinformation and its refutation. The context is the pervasive impact of misinformation (e.g., during COVID-19), and the purpose is to support fairer and more effective interventions: educating naive spreaders, stopping malicious actors, and leveraging informed sharers to amplify corrections. The importance lies in incorporating behavior after exposure to refutations to better capture intention, which prior work typically ignored, often mislabeling users based on a single sharing event without considering subsequent corrective behavior.

Prior work on misinformation detection has focused on content features (linguistic/stylistic, deep learning models), propagation structures, user profile features, and past sharing histories. Some works identify spreaders using profile, linguistic/personality, sentiment, and topic features, or analyze network interactions and structures. However, existing approaches largely perform binary classification (misinformation vs. refutation/true spreaders) and label users as misinformation spreaders upon a single share of false content, ignoring whether they later encountered and shared refutations. No prior work incorporates a sequence-of-behavior framework that conditions labels on exposure to both misinformation and its refutation to infer intent. This paper fills that gap by defining multiple behavior-based categories and leveraging network embeddings for classification.

Overview: The approach, termed behavioral forensics, consists of (1) behavior-based labeling after exposure to both misinformation and refutation, (2) learning network embedding features from follower–followee graphs, and (3) two-step machine learning classification. Labeling mechanism and classes: Users are labeled only if they are exposed to at least one pair of misinformation (m) and corresponding refutation (r). Exposure is operationalized as follows: when a user tweets/retweets m or r, all their followers are assumed exposed with exposure time set to the (re)tweet time; if multiple followees share the same message, exposure time is the earliest; if a user retweets prior to exposure by followees, exposure time equals the user’s first retweet time. Sequences of actions (share/not share) upon exposure are mapped to five classes using a state diagram: - malicious: share misinformation after exposure to both m and r (e.g., G→I, O→P, or C→D→F), indicating intentionality. - maybe_malicious: (i) shared m, then saw r but did not share r (stayed at D), or (ii) shared r and then later shared m (e.g., A→B→G→H→S; A→J→K→L→M; A→J→O→Q→U). Ambiguous intent but contributes to misinformation spread. - naive_self_corrector: shared m then later shared r after exposure to both (e.g., A→B→C→D→E; A→B→G→I→T; A→J→O→P→R). - informed_sharer: (i) shared r after exposure to both without sharing m (e.g., A→B→G→H; A→J→O→Q), or (ii) shared r, then after receiving m (K→L), did not share m (L) or shared r again (L→N). - disengaged: exposed to both m and r but shared neither (e.g., states G or O). Users can transition out of disengaged if they later share. Multi-event labeling consolidation: For users exposed to multiple m–r pairs, multiple labels may result. Disengaged labels are dropped if any non-disengaged labels exist. Remaining labels are mapped to integers (malicious=1, maybe_malicious=2, naive_self_corrector=3, informed_sharer=4). The median (choosing the larger in ties to avoid false positives with malicious as positive) yields a single consolidated label. Users exposed to only one m–r pair retain that label. Graph embeddings: Build a follower–followee network using labeled users and their connections. Learn low-dimensional node embeddings preserving topology and homophily using LINE (second-order for directed graphs) and PyTorch-BigGraph (PBG). Embedding dimensions tried: 4, 8, 16, 32, 64, 128. Embeddings are normalized. LINE captures shared-neighborhood similarity even without direct links; PBG scales to very large graphs. Features: For step-1 (disengaged vs others), use only normalized embeddings (profile features not needed due to strong separability). For step-2 (four-class classification among others), concatenate normalized embeddings with user profile features: follower count, followee (friend) count, statuses count, listed count, verified (1/0), protected (1/0), and normalized account age (days). Classification strategy: Due to extreme imbalance (disengaged ≈99.75%), employ two-step classification. Step-1: Disengaged vs Others with random undersampling of disengaged to balance (e.g., 4,059 disengaged vs 3,419 others). Step-2: Multiclass classification among malicious, maybe_malicious, naive_self_corrector, informed_sharer. Classifiers evaluated include Logistic Regression (OvR for multiclass), k-NN (k=5), SVM, Naive Bayes, Decision Tree, Random Forest (100 trees), and Bagged Decision Tree (base estimator SVM). For step-2 imbalance, use class_weights='balanced' when available or SMOTE for oversampling. Evaluation: Step-1 uses 5-fold cross-validation; Step-2 uses 10-fold cross-validation. Baselines: for step-1 a random classifier (50% accuracy) since no prior comparable method; for step-2 two baselines—predict-all-majority (naive_self_corrector) and random class. Visualization with t-SNE illustrates class clustering as embedding dimensionality increases. Code available at https://github.com/eunakhan/behavioralforensics.

Data and labels: Using a Twitter dataset of 8 political news events from 2019 (from altnews.in), 1,365,929 users were labeled. Class distribution: 99.75% disengaged (1,362,510); among non-disengaged (3,419): malicious=926, maybe_malicious=222, naive_self_corrector=1,452, informed_sharer=819. Step-1 (two-class: disengaged vs others) with LINE embeddings (128d): Logistic Regression achieved 99.251% accuracy (Bagged Decision Tree 99.157%, SVM 98.957%, Random Forest 97.794%, Decision Tree 86.936%, k-NN 95.318%, Naive Bayes 81.236%), far surpassing the 50% random baseline. ROC curves show excellent separability. Step-2 (multiclass among non-disengaged) with 128d LINE embeddings + profile features: Bagged Decision Tree performed best with accuracy 73.637% and weighted F1 72.215%, outperforming baselines (majority-class and random). Class-wise performance (Bagged Decision Tree): - malicious: precision 77.446%, recall about 75% (stated as 75.80%); k-NN precision 75.812% with similar recall. - maybe_malicious: precision 92.078%. - naive_self_corrector: precision 64.246%. - informed_sharer: precision 69.073%. Embedding dimensionality: Increasing embedding dimension improved performance, with noticeable cluster formation from 16d onward; gains plateaued around 64d. t-SNE visualizations show emergent clustering, e.g., malicious and informed_sharer forming distinct regions at higher dimensions. Headline result: Detecting malicious (intentional) spreaders achieved 77.45% precision and 75.80% recall, enabling targeted interventions against disinformation agents.

The findings demonstrate that intent-aware, behavior-based labeling—requiring exposure to both misinformation and refutation—combined with network embeddings can effectively identify nuanced categories of spreaders. Network structure alone provides strong signal to distinguish disengaged from others, and, when combined with profile features, supports meaningful multiclass separation across malicious, maybe_malicious, naive_self_corrector, and informed_sharer. The approach addresses shortcomings of prior binary labeling that ignored corrective behavior and thus conflated naivety with malice. The precision–recall trade-offs suggest policy-aligned model selection: prioritize precision if banning malicious accounts to avoid false positives, or prioritize recall if the goal is to preemptively treat followers of malicious users with refutations. Similarly, maximizing recall for naive_self_corrector and informed_sharer can facilitate peer correction and amplification of refutations to mitigate misinformation spread.

The work introduces a novel behavior-based, multi-class labeling framework for identifying misinformation, disinformation, and refutation spreaders, requiring exposure to both misinformation and refutation to better infer user intent. Leveraging follower–followee network embeddings and user profile features within a two-step classification pipeline, the model can categorize users even without direct behavioral histories. Results on a large Twitter dataset show high accuracy in separating disengaged users and strong performance in multiclass prediction, particularly for identifying malicious actors with 77.45% precision and 75.80% recall. The approach is general and applicable across social platforms. Future work includes dynamically updating user labels as new behaviors emerge to reflect behavior changes over time, and studying how user categories vary across topics, enabling topic-sensitive and adaptive interventions.

- Exposure approximation: Assumes all followers are exposed when a followee (re)tweets, with exposure time set to the earliest (re)tweet; actual view data and timestamps are unavailable via the API, which may introduce exposure misclassification. - Data scope: Twitter-only, 8 political news events in 2019; retweet times missing for two events (excluded). Findings may not generalize across platforms, languages, or domains. - Class imbalance: Extreme skew (≈99.75% disengaged) required undersampling and synthetic oversampling, which can affect classifier stability and generalizability. - Behavioral sparsity: Very few users exhibit non-disengaged actions; labeling across multiple events is limited, prompting consolidation heuristics (median of ordinal labels with tie-breaking) that may introduce bias. - Intent inference: Intent is inferred from observed sequences and may be imperfect; users’ motivations cannot be directly observed. - Feature limits: Behavior defined via retweets only; content semantics and temporal dynamics beyond exposure/retweet timing are not fully leveraged. - Embedding dependence: Performance depends on embedding dimension/quality and the assumption that homophily/topology reflect behavioral categories.