Altmetrics, particularly those based on social media, were initially conceived to measure the societal impact of research. However, existing altmetrics primarily capture engagement from academics, failing to reflect broader societal impact. This limitation arises because social media mentions of research articles are largely generated by academics themselves, not the general public. To address this, the study proposes examining 'second-order citations,' which are social media mentions of news articles that discuss research. News media plays a critical role in public discourse and serves as a key source of science information for the public. While some studies have examined news media coverage of research, they have not analyzed the subsequent social media engagement with those news stories. This study aims to investigate whether second-order citations provide a more effective means of understanding non-academic audiences' engagement with research content.

Existing research shows a weak correlation between social media mentions and traditional citations, with social media activity often dominated by academics. Studies consistently demonstrate that researchers tend to tweet about topics similar to their publications, indicating social media activity as an extension of scholarly communication. While platforms like Altmetric and PlumX track news media mentions of research, few studies have analyzed how these mentions impact broader audiences. The concept of 'second-order citations' has been proposed, suggesting that news stories mentioning research could reach a wider, non-academic audience than the research articles themselves.

The study analyzed COVID-19 research articles published between January 1, 2020, and December 31, 2020, from bioRxiv, medRxiv, the Journal of Virology, and the British Medical Journal. News stories mentioning these articles were collected from five major news outlets (BBC, MSN, The New York Times, The Guardian, and The Washington Post) using Altmetric Explorer. Social media posts (tweets and Facebook posts) linking to both the research articles (first-order citations) and the news stories (second-order citations) were collected using Python scripts and Crowdtangle. The dataset included 344 research articles, 1,221 news stories, 50,299 tweets and 6,420 Facebook posts linking to the research articles (first-order citations), and 97,235 tweets and 14,081 Facebook posts linking to the news stories (second-order citations). Twitter accounts sharing research articles were compared to those sharing news stories to determine the overlap with a dataset of known researchers on Twitter. Statistical analyses were conducted using Python's Pandas package to compare the volume of social media engagement and the overlap in audiences between first- and second-order citations.

The study found that second-order citations (social media mentions of news stories about research) were shared and engaged with on social media approximately twice as much as first-order citations (direct mentions of research articles). News stories from the five analyzed outlets were shared twice as often and by twice as many unique accounts as the research articles themselves. The engagement levels (retweets/shares, likes/reactions, and replies/comments) were also significantly higher for second-order citations. Correlations between Facebook shares and tweets were high for both news stories and research articles, but correlations between first- and second-order citations were low, indicating distinct audiences. Overlaps between accounts sharing first- and second-order citations were small on both Twitter and Facebook. Finally, the proportion of researcher accounts among those sharing first-order citations was more than twice as high as the proportion among those sharing second-order citations, suggesting that the latter reached a broader, non-academic audience.

The findings challenge the assumption that social media engagement with research is largely confined to academic communities. The significantly greater reach and engagement of second-order citations demonstrate the potential of news media to disseminate research to wider audiences. Several factors likely contributed to this, including journalists' ability to make research more accessible and relevant to non-academic audiences and the public's reliance on news as a key source of scientific information, particularly during the pandemic. The low correlation and limited overlap between first- and second-order citations suggest different selection criteria are at play for what research is deemed ‘newsworthy’. Journalists often prioritize practical value, societal relevance, and emotional impact when selecting stories, whereas academics might focus on novelty or academic significance. Further, the analysis of highly shared articles revealed that studies focusing on treatment/prevention measures, high-risk situations, and controversial subjects tended to attract more second-order citations.

This study provides strong evidence for including second-order citations in altmetrics to better capture the societal impact of research beyond academic circles. The methodology developed for identifying and analyzing second-order citation data is a valuable contribution to the field, despite challenges posed by recent changes in social media APIs. Future research should explore this methodology further, investigate the role of journalists and influencers in shaping social media engagement, and analyze broader patterns across different research domains and time periods.

The findings are based on a case study of COVID-19 research and may not be generalizable to other research areas or time periods. The focus on a limited set of news outlets may underestimate the total social media engagement with second-order citations. Changes in social media APIs may make direct replication of the methodology difficult.