Cognitive reflection correlates with behavior on Twitter

The study examines how individual differences in analytic (reflective) versus intuitive thinking relate to real-world behavior on social media. Against the backdrop of dual-process theory and debates between intuitionist perspectives (which argue reasoning mainly serves post-hoc justification and has limited real-world impact) and reflectivist perspectives (which posit that analytic thinking improves accuracy and influences everyday behaviors), the authors investigate whether cognitive reflection predicts discernment and content engagement on Twitter. Prior work has linked social media use to personality traits and demographics and has shown in surveys that higher CRT associates with reduced belief in misinformation and greater acceptance of science. The authors aim to move beyond self-reports to observe naturally occurring behavior, testing whether reflective thinking meaningfully shapes what users follow and share online, and whether cognitive-style-based echo chambers exist beyond political ideology.

The paper reviews research connecting social media behavior to personality (Big Five, Dark Triad), partisanship, gender, and age; debates within cognitive science about the roles of intuition versus reflection in everyday judgments; and evidence that higher cognitive reflection correlates with religious disbelief, reduced conspiracy endorsement, increased acceptance of science, lower reliance on internet searches, and lower belief in and self-reported sharing of fake news. It also notes work on actively open-minded thinking and Twitter behavior. The authors highlight limitations of prior survey-based studies (demand effects, small curated stimulus sets, focus on blatantly false headlines) and the need to test whether analytic thinking predicts real-world sharing decisions and content consumption on social media.

Design: Hybrid lab–field study linking survey measures to behavioral Twitter data. Participants: Convenience sample recruited via Prolific (N = 1,901; 55% female; median age = 33; predominantly UK/US-based, with others from Canada, Spain, Italy, Portugal). Recruitment occurred between April 15 and June 20, 2018. Participants provided their Twitter usernames. Measures: Participants completed a seven-item Cognitive Reflection Test (combining a reworded three-item numeric CRT with four non-numeric CRT items). CRT scores were computed as the proportion correct (0–1). A demographic survey assessed education, political ideology (social and economic), ethnicity, income, religiosity, and other background variables. Twitter data collection: Using the Twitter API, the authors retrieved public profile information (e.g., total tweets, followers, following, favorites, listed count), up to ~1,200 most recent tweets per user, and the list of accounts each user follows (capped at 2,000 per user; only 6 exceeded this cap). Primary data pull was on August 1, 2018; tweets/retweets were also tracked on April 10, 2021 to maximize observations. Of the 1,901 users, 1,619 had accessible public tweets (totaling 1,871,963 tweets). Data were cleaned and matched to survey records. Key behavioral measures:

• Profile characteristics: number of accounts followed, followers, total tweets, tweets in past two weeks, favorites, lists.
• Content consumption: which accounts each user follows (co-follower network analysis).
• Content sharing: links to news domains and linguistic content of tweets. Analytic approach:
• Regressions used z-scored CRT as the main predictor, with controls for z-scored age, gender (0 = male, 1 = female), ethnicity (0 = non-white, 1 = white), political ideology (liberal–conservative scale), US residency, education (college degree), income, and log time to complete survey; standard errors clustered at the user level as appropriate; month fixed effects in tweet-level analyses.
• Co-follower network: Nodes are accounts followed by at least 25 participants (186 nodes; robustness to other thresholds reported). Edges weighted by number of shared followers among participants. Community detection identified clusters; the average CRT of followers for each account was used to predict cluster membership (logistic regression on account-level data).
• News sharing quality: Focused on tweets/retweets linking to one of 60 news websites with trust ratings from professional fact-checkers. Linear/logistic models predicted whether users shared news and the trust score of shared sources as a function of CRT and controls; robust SEs clustered by user; random/fixed effects included.
• Topic modeling: Structural Topic Modeling (STM) on English tweets/retweets from users with at least 10 English tweets (N = 1,424). All tweets per user were merged into a document; CRT used as a document-level covariate to estimate topic prevalence differences across CRT.
• Language analysis: LIWC dictionaries assessed the presence of words in categories including Insight, Inhibition, Positive emotion, Negative emotion, Morality, and Political. Logistic regressions related CRT to the probability that a user’s tweets contained words from each category, controlling for demographics. Significance assessed with appropriate multiple standard errors; robustness checks documented in supplementary materials. Ethics: Approved by Yale Human Subjects Committee, IRB Protocol #2002022539. Data availability restricted due to confidentiality; materials via journal links.

• Higher CRT users followed fewer accounts, indicating greater selectivity (incidence rate ratio = 0.867, p = 0.01).
• Co-follower network revealed two distinct clusters differing in followers’ CRT: Cluster 1 (higher CRT) mean follower CRT = 0.515 (SD = 0.075; 35% of nodes) vs. Cluster 2 (lower CRT) mean follower CRT = 0.419 (SD = 0.032; 65% of nodes); effect size Cohen’s d = 1.66. The average CRT of an account’s followers strongly predicted cluster membership (logistic regression OR = 0.545 for being in the low-CRT cluster per 1 SD increase in followers’ average CRT, p = 0.004), implying a one SD decrease in followers’ average CRT increased the odds of being in the low-CRT cluster by ~83.5%.
• News sharing: Among those who shared links to the rated sites, higher CRT was associated with sharing higher-quality news sources (β = 0.078, p = 0.019). Higher CRT users were more likely to share BBC links (OR = 1.232, p < 0.001) and less likely to share Daily Mail links (OR = 0.787, p < 0.001). Political conservatism and longer survey completion time were negatively related to the quality of shared content; US residency also showed a negative relation in reported models.
• Topic modeling: CRT positively correlated with a politics-related topic (e.g., words like people, vote, trump, brexit) and negatively correlated with a topic involving giveaways/“get rich quick” schemes (e.g., win, enter, giveaway, prize). Results were robust across topic-number choices.
• LIWC word categories: Higher CRT predicted greater use of Insight words (OR = 1.138, p < 0.001) and Inhibition words (OR = 1.133, p < 0.001). Contrary to predictions, CRT was positively related to Negative emotion words (OR = 1.124, p < 0.001) and not significantly related to Positive emotion words (OR = 0.966, p = 0.235). Relationships between CRT and Morality and Political word categories were also significant when controlling for demographics.
• Demographic controls: Several controls showed associations (e.g., conservatism negatively related to news quality; various LIWC associations by age, US residency, income), detailed in supplementary tables.

Findings support the reflectivist view that analytic thinking influences real-world behavior: higher cognitive reflection is linked to more discerning social media use—following fewer accounts, sharing higher-quality news, and engaging more with weighty topics like politics. The co-follower network reveals cognitive echo chambers beyond partisan lines, suggesting users cluster by cognitive style, which may shape information exposure and diffusion. The alignment of CRT with reduced engagement in giveaway/scam-like content supports the notion that lower reflection associates with greater gullibility in practice, extending survey-based evidence into observed behavior. Results also refine understanding of political engagement: rather than social desirability in self-reports, higher CRT users actually engage more with political content on Twitter. Overall, reflective thinking appears to be a positive force in everyday online judgment and decision-making, challenging strong intuitionist claims that reasoning has limited functional relevance outside the lab.

The study demonstrates that cognitive reflection predicts several facets of Twitter behavior: selective following patterns, sharing from more trustworthy news sources, and prioritization of substantive (especially political) topics. These results extend prior survey-based findings to naturally occurring behaviors, underscoring the practical importance of reasoning in daily life online. Future research directions include: modeling how cognitive-style homophily shapes social media networks and information flow; testing generalizability across platforms and non-Western contexts; developing machine learning models to infer CRT-like measures from online behavior; employing richer language models beyond dictionary methods; and assessing whether results generalize to other measures of cognitive sophistication beyond the CRT.

• Non-representative convenience sample (mostly UK/US) limits generalizability.
• Analyses rely on users who provided and had accessible public Twitter accounts; some participants lacked usable Twitter data.
• Observational design precludes strong causal inference about CRT effects on behavior.
• Dictionary-based LIWC approach may inadequately capture complex psychological constructs, contributing to mixed emotion-related findings.
• Focus on Twitter; uncertain generalizability to other platforms or cultural contexts.
• Data-sharing constraints (Twitter data confidentiality) limit external replication with the same dataset.