A simulation-based analysis of the impact of rhetorical citations in science

Citations are widely used in science to measure the impact of papers and researchers under the assumption that authors cite prior work to acknowledge intellectual debts. However, decades of research show that many citations are made for rhetorical reasons—authors cite to persuade readers and reviewers regardless of whether the cited work influenced them. We refer to these as rhetorical citations. While some rhetorical citations provide context, differentiation, or are coerced during peer review, the practice is often viewed as undesirable and potentially corrupting incentives. The implicit comparison is between the current world with rhetorical citing and a counterfactual world with only substantive citing, but such a rigorous comparison has been lacking. Theory and evidence suggest that when searching for papers to read and potentially cite substantively, researchers use status and citations as heuristics for perceived quality, creating feedback that concentrates attention on highly cited works. In contrast, rhetorical citing depends on rhetorical value, which can involve quality and status but also other factors, potentially redistributing attention to a broader set of papers. The first contribution of this paper is to compare a counterfactual world with only substantive citing to a more realistic world with both substantive and rhetorical citing, focusing on three community health metrics: correlation between citations and quality, citation churn, and citation inequality. The second contribution is a behavioral agent-based model of reading and citing that integrates both substantive and rhetorical motivations with cognitively realistic search and reading practices. We simulate communities where rhetorical citing can be turned on or off to measure causal effects on these metrics and examine how reading budgets, citing budgets, and literature size moderate the outcomes.

The paper synthesizes two main theories of citing: the normative theory, in which researchers cite to acknowledge intellectual debts (substantive citing), and the social constructivist theory, in which authors cite to persuade regardless of influence (rhetorical citing). Researchers do not read all potentially relevant papers and select strategically, relying on heuristics such as status and citation counts to infer quality, which can lead to cumulative advantage and concentration of attention among highly cited works. Substantive citing presupposes reading and is thus tied to these selection dynamics. Rhetorical citing, which does not require influence or careful reading, is supported by evidence of mischaracterizations and superficial use of references. Papers’ rhetorical value varies by time-invariant characteristics (e.g., quality, outlet) and time-varying signals (e.g., citations, author status), and differs across researchers based on topic fit and how well cited claims support their narratives. Empirical work indicates that longer reference lists are associated with fewer uncited papers and a redistribution of attention beyond elite works, consistent with rhetorical citing weakening status-based feedback. The paper situates its model within efforts to synthesize citation theories (e.g., social systems citation theory) and addresses empirical challenges in classifying citation types and establishing causal effects by using simulation.

Design: The authors develop an agent-based model of reading and citing in discrete time. At each timestep t, a new agent j enters, selects papers to read, and makes citation decisions. Agents have a reading budget m and a citing budget n. Papers have underlying quality q_i ~ Beta(1,6). Agents have person-specific thresholds τ_j ~ Uniform(0,1) for substantive citing and perception errors ε_ij (e.g., [-0.15, 0.15]). Paper-agent fit f_ij ~ Normal(0,0.05) adjusts perceived usefulness of a paper’s quality for agent j. Perceived quality before reading is S_ijt = q_i + f_ij + α·c_it + ε_ij, with α = 0.001 capturing reinforcement of perceived quality by citations. Agents read the m papers with highest S_ijt, after which perception error disappears for those papers, and the perceived quality becomes q_ij = q_i + f_ij. Citing process: Step 1 (substantive). Agents substantively cite all read papers whose post-reading quality q_ij exceeds threshold τ_j. Step 2 (rhetorical). If reference slots remain, agents rank all papers by rhetorical value r_ijt and fill remaining slots with the highest r_ijt, allowing a paper to be both substantively and rhetorically cited by the same agent. Rhetorical value consists of an underlying person-specific component r_ij (initialized Beta(1,6)) and a status premium proportional to perceived quality: if unread, r_ijt^unread = r_ij + β·S_ijt; if read, r_ijt^read = r_ij + β·(S_ijt − ε_ij). β = 0.3 governs the strength of rhetorical reinforcement. After citations are made, citation counts c_it are updated for all papers. Models: The full model includes both substantive and rhetorical citing. Two null models include only substantive citing based on q_ij: (1) Null-fixed-reference: agents fill all n slots with the n highest q_ij even below threshold; (2) Null-fixed-threshold: agents cite only papers with q_ij ≥ τ_j, leaving unused slots if necessary. Metrics: At each timestep t, the model computes (a) Pearson correlation between citation counts c_it and underlying quality q_i (citation–quality correlation), (b) citation churn: number of papers cited at time t not cited at time t−1, and (c) citation inequality: Gini coefficient of the citation distribution at t. Parameters and runs: Baseline parameters: literature size N = 600, reading budget m = 120, citing budget n = 40, timesteps = 1000 (each paper can accrue at most 1000 citations). Robustness checks explore N ∈ [200, 800], m ∈ [50, 150], n ∈ [20, 100], and alternative distributions. Heterogeneous agents (varying τ_j, f_ij, r_ij) constitute the main analyses; homogeneous agents are reported in Supplementary Information. Statistical comparisons use OLS regressions (one-tailed for full vs. null in main effects; two-tailed for moderation analyses) over 20 simulation runs.

Main effects of rhetorical citing (full vs. null models): - Citation–quality correlation: After 1000 timesteps, correlation is higher in the full model by +2.5% relative to both nulls (Full vs. Null-fixed-reference: t(39998) = 120.526, Cohen’s d = 1.205, 95% CI = (0.019, 0.019); Full vs. Null-fixed-threshold: t(39998) = 126.128, d = 1.261, 95% CI = (0.019, 0.020)). - Citation churn: Average churn across 1000 iterations is 2.36× higher than Null-fixed-reference (t(39958) = 484.613, d = 4.849, 95% CI = (15.217, 15.340)) and 2.17× higher than Null-fixed-threshold (t(39958) = 272.673, d = 2.728, 95% CI = (15.103, 15.322)). - Citation inequality: After 1000 iterations, the Gini coefficient is 30–31% lower than both nulls (Full vs. Null-fixed-reference: t(39998) = −1295.954, d = 12.960, 95% CI = (−0.292, −0.291); Full vs. Null-fixed-threshold: t(39998) = −1308.778, d = 13.088, 95% CI = (−0.295, −0.294)). Mechanism: Focusing on high-quality (top 40) vs. mid-quality (ranks 41–150) papers—which comprise 25% of the literature but draw ~85% of citations—null models send only ~18% of citations to mid-quality papers (due to perception error and fit variability). In the full model, substantive citations shift away from the very top, and mid-quality papers gain substantial rhetorical citations; low-quality papers remain rarely cited even rhetorically. Rhetorical citing thus redistributes attention from a stable elite to a broader, more dynamic mid-to-high-quality set. Moderation by citing budget (n from 20 to 100; m = 120, N = 600): - Correlation increases: Full +35.3% (0.68→0.92, p < 0.001); Null-fixed-reference +31.3% (0.67→0.88, p < 0.001); Null-fixed-threshold +35.8% (0.67→0.91, p < 0.001). - Churn increases: Full 4.59× (13.12→60.23, p < 0.001); Null-fixed-reference 2.76× (6.46→17.80, p < 0.001); Null-fixed-threshold 3.83× (6.47→24.81, p < 0.001). Full model churn advantage rises from ~2.03× to 3.38× over Null-fixed-reference and from ~2.03× to 2.43× over Null-fixed-threshold. - Inequality decreases: Full −30.0% (0.70→0.49, p < 0.001); Null-fixed-reference −14.6% (0.96→0.82, p < 0.001); Null-fixed-threshold −14.6% (0.96→0.82, p < 0.001). Moderation by reading budget (m from 50 to 150; n = 40, N = 600): - Correlation changes weakly (Full 0.79→0.80, p = 0.010; Null-fixed-reference 0.76→0.78, p < 0.001; Null-fixed-threshold 0.76→0.78, p < 0.001). - Churn changes slightly (Full 25.51→26.91, p = 0.005; Null-fixed-reference 8.38→10.89, p < 0.001; Null-fixed-threshold 11.38→11.67, p = 0.938). - Inequality changes minimally (Full 0.63→0.61, p = 0.016; Null-fixed-reference 0.92→0.91, p < 0.001; Null-fixed-threshold 0.93→0.92, p < 0.001). Moderation by literature size (N from 200 to 800; m = 120, n = 40): - Correlation decreases as N grows (Full −22.1%, 0.95→0.74, p < 0.001; Null-fixed-reference −21.7%, 0.92→0.72, p < 0.001; Null-fixed-threshold −22.6%, 0.93→0.72, p < 0.001). - Churn increases (Full +26.3%, 22.52→28.43, p < 0.001; Null-fixed-reference +29.3%, 9.21→11.91, p < 0.001; Null-fixed-threshold +22.8%, 10.16→12.48, p < 0.001). - Inequality increases (Full +30.6%, 0.49→0.64, p < 0.001; Null-fixed-reference +22.1%, 0.77→0.94, p < 0.001; Null-fixed-threshold +20.5%, 0.78→0.94, p < 0.001).

Turning rhetorical citing on in the model increases the correlation between citations and underlying quality, raises citation churn, and reduces citation inequality. The proximate mechanism is a redistribution of citations from a small, stable set of elite-quality papers to a broader set of mid-to-high-quality papers with higher rhetorical value for different authors. This weakens status-based feedback loops and helps deconcentrate attention, potentially making it easier for new or alternative ideas to gain recognition. Policy-relevant levers behave asymmetrically: expanding citing budgets (longer reference lists) amplifies these benefits by creating space for rhetorically useful, non-elite papers; in contrast, increasing reading budgets alone has little impact on formal recognition because citation slots are the bottleneck. Expanding literature size reduces citation–quality alignment and increases inequality but also increases churn as more high-quality candidates exceed thresholds, enabling substitution among them. While prior work has linked growing volume to stagnation, these simulations suggest that some seemingly undesirable practices (rhetorical citing, longer reference lists) can mitigate concentration and enhance dynamism in citation patterns. However, the study does not endorse rhetorical citing as universally beneficial; broader outcomes (e.g., misinformation, reward allocation) lie beyond the current metrics and time horizon. Effective policy should consider the entire pipeline—how researchers search, read, and then cite—rather than only the final act of referencing.

The paper contributes a synthetic behavioral model of citing that integrates substantive and rhetorical motivations with realistic search and reading constraints, enabling counterfactual comparisons by switching rhetorical citing on and off. Simulations show that rhetorical citing can plausibly improve three aspects of scientific community health—higher citation–quality correlation, greater citation churn, and lower citation inequality—by redistributing attention beyond a stable elite. Increasing citing budgets strengthens these benefits, whereas increasing reading budgets has limited effect; larger literatures decrease citation–quality alignment and increase inequality while also increasing churn. Future research should: (1) broaden community health metrics (e.g., misinformation prevalence, reward allocation efficiency, epistemic diversity), (2) endogenize agent behavior and paper production in response to incentives, (3) incorporate long-run dynamics such as obsolescence, decaying reinforcement, and entry of new works, (4) model heterogeneous dimensions of quality with distinct citation dynamics, and (5) explore parameter distributions and reinforcement forms to further align models with empirical practices.

The metrics used—citation–quality correlation, churn, and inequality—are not exhaustive, and optimal levels of churn and inequality are ambiguous. The model holds agent characteristics and paper types fixed across worlds, omitting behavioral adaptation to citation incentives (e.g., producing for rhetorical value vs. quality). It focuses on short-to-medium-term dynamics without ceilings on reinforcement or long-run processes such as obsolescence, decaying influence, and continual entry of new publications. Quality is modeled as a single underlying dimension, whereas empirical work indicates multiple types of quality with distinct citation patterns. Parameter choices, distributions, and reinforcement strengths shape outcomes; while robustness checks show qualitative consistency, further exploration is warranted. The study does not estimate the total effect of rhetorical citing on science (e.g., potential for misinformation, biases from early vs. late-stage references), so results should not be interpreted as an overall endorsement of rhetorical citing.