How analysis of mobile app reviews problematises linguistic approaches to internet troll detection

The detection of internet trolls, particularly those engaged in state-sponsored disinformation campaigns, is a significant challenge. Existing research has focused on linguistic features of troll writing, often characterized by repetitive messaging masked through varied phrasing. Previous work by Monakhov (2020a, 2020b) demonstrated an algorithm capable of identifying troll tweets with high accuracy based on the ratio of repeated words to repeated word pairs (the "troll coefficient"). However, applying this algorithm to app reviews yielded unexpected results. Both positive (potentially suspicious) and negative (likely genuine) reviews showed high troll coefficients, suggesting that the linguistic features used for identification are not solely indicative of malicious intent. This paper hypothesizes that the observed high troll coefficients in negative app reviews stem from a collaborative effect, where users adapt their language to avoid repetition in response to existing comments. The study aims to investigate this hypothesis by analyzing a large dataset of app reviews and conducting an experiment to verify the findings in a controlled setting.

The study of internet trolling has a long history, evolving from early investigations in computer-mediated communication (Sia et al., 2002; Douglas and McGarty, 2001; Siegel et al., 1986) and hate speech (Carney, 2014; Chakraborti, 2010; Herring et al., 2002; Fraser, 1998) to its current relevance in hybrid warfare (Lundberg and Laitinen, 2020; Zannettou et al., 2019; Elyashar et al., 2018; Egele et al., 2017; Volkova and Bell, 2016). Monakhov’s (2020a, 2020b) previous work provided a crucial foundation for this study, demonstrating that the inherent limitations in the discourse of trolling—the need to convey a message repeatedly without detection—lead to unique linguistic patterns. This work established the "troll coefficient" as a promising metric. The current study builds upon this foundation by exploring the limitations of the coefficient in contexts beyond the previously analyzed tweets and examining the potential influence of collaborative writing styles on the algorithm’s accuracy.

Study 1 involved analyzing one-star reviews of two apps—one Russian and one English—from the Google Play Store. Over 4000 Russian and nearly 2000 English reviews were collected. The reviews were chronologically aligned and divided into samples of 250 reviews each, with the sampling window incrementally increasing. The troll coefficient (q = w/W, p/P, where w is the number of repeated content words, W is the total number of content words, p is the number of repeated content word pairs, and P is the total number of content word pairs) was calculated for each sample, along with the average time distance between reviews within each sample. This process was repeated with randomly shuffled reviews for comparison. Further, 3000 English reviews from 59 diverse apps were analyzed to assess generalizability. A Bayesian multiple linear regression model was constructed to predict troll coefficients, considering sample range, rate of change (reviews per unit time), average time distance, and the average number of words per review as predictors. Study 2 involved a controlled experiment using Yandex.Toloka, a crowdsourcing platform. 192 and 193 Russian-speaking participants described the same picture in two conditions: one where they could not see others’ responses and one where they could. The troll coefficient was calculated for each group's comments, and regression models were used to analyze the relationships between troll coefficient and the order of responses. Data analysis involved calculating correlation coefficients, linear and polynomial regression modeling, and Bayesian inference.

Study 1 revealed a strong negative correlation between the average time distance between reviews and the troll coefficient in chronologically aligned reviews (r = -0.87, p < 0.0001 for Russian; r = -0.84, p < 0.0001 for English). This correlation was absent in the randomly shuffled reviews. Analysis of 59 apps demonstrated that the relationship between time distance and troll coefficient varied depending on review frequency. Apps with infrequent reviews sometimes showed positive correlations. The Bayesian regression model successfully predicted troll coefficients based on time distance, sample range, rate of change, and word count, suggesting that time plays a significant role in shaping the troll coefficient. Study 2 showed significant differences in troll coefficients between the two experimental conditions. Participants who could see prior responses had significantly higher troll coefficients, exhibiting a positive correlation between troll coefficient and response order when sorted chronologically. When sorted inversely, an inverted U-shape correlation was observed, indicating that later participants considered only recent responses when choosing their language. These findings strongly suggest that the 'troll-like' effect observed in app reviews is influenced by collaborative repatterning and time.

The results strongly support the hypothesis that collaborative repatterning, not malicious intent, contributes significantly to high troll coefficients in app reviews. The negative correlation observed in Study 1 between time distance and troll coefficient, and the experimental results from Study 2, clearly indicate that users modify their language choices in response to recent contributions, leading to lower repetition and a reduced troll coefficient. This aligns with concepts like 'schema refreshing,' 'heteroglossia,' and strategic language deployment from language creativity studies, highlighting the interplay between creative language use and the perception of ongoing discourse. The study extends the understanding of online communication dynamics and indicates the importance of contextual factors in linguistic troll detection.

This study demonstrates that linguistic features previously used to identify internet trolls can be influenced by collaborative writing styles, complicating the development of effective troll detection algorithms. The model developed to predict troll coefficients based on temporal factors offers a potential solution, allowing for differentiation between malicious behavior and collaborative repatterning. Future research should focus on refining this model, incorporating more nuanced linguistic features, and investigating the cognitive mechanisms underlying collaborative repatterning in online discussions.

The study primarily focused on English and Russian language app reviews, limiting the generalizability of the findings to other languages. The definition and measurement of "troll-like" behavior could be further refined. While the experiment provided valuable insights, the artificial nature of the task might not perfectly capture the complexity of real-world app review writing. Additionally, the model's predictive accuracy could be improved by incorporating more sophisticated contextual information and potentially non-linear relationship among features.