From sociology of quantification to ethics of quantification

The pervasive use of quantification in modern society, from scientific modeling to social media algorithms, has raised significant ethical concerns. The authors begin by illustrating how the COVID-19 pandemic starkly revealed the problematic aspects of relying on quantitative models for policy decisions, particularly concerning the lack of transparency and the underestimation of uncertainty in the Imperial College model. This highlights a broader crisis of reproducibility in science, exacerbated by perverse incentives. The authors then introduce the concept of social system theory to explain how different systems (science, economy, media, politics) interact and how the media's appetite for quantified information accelerates pervasive quantification. The authors investigate the different forms of quantification (rankings, indicators, models, algorithms), and their commonalities, exploring how algorithms and big data pose societal threats when decisions are outsourced to computation without consideration for the inherent biases, uncertainties, and political implications. The authors emphasize the urgency of developing a better understanding of, adaptation to, and defense from pervasive datafication and the need for societal control over the use of quantification in policy making. They aim to identify commonalities within diverse forms of quantification, setting the stage for a broader discussion and a call for an encompassing ethics of quantification.

The paper reviews existing literature on the ethics of quantification, highlighting fragmented efforts in various domains. It examines works on the ethics of AI, statistical malpractice, the misuse of metrics (Goodhart's Law), and the challenges in mathematical modeling. The authors discuss the work of Espeland and Stevens, who emphasize the social nature of quantification and the need for an ethics of numbers. They also reference works by O'Neil on the negative consequences of algorithms, Supiot on the numerification of society, and the French 'statactivistes' who use numbers to fight unjust quantification. The paper draws on Jasanoff's distinction between 'technologies of hubris' and 'technologies of humility' to categorize the different approaches to quantification. It also touches on the techno-optimistic narratives that often overshadow concerns about the potential harms of quantification and discusses the concept of value of statistical life (VSL) and its limitations. It explores works on data activism and the need for more transparency and accountability in the use of data.

This is a perspective article, not an empirical study. The methodology consists of a review of existing literature on the ethics of quantification, drawing upon various disciplines including sociology, philosophy, statistics, computer science, and political science. The authors synthesize arguments and examples from different fields to identify common issues and propose an overarching framework for an ethics of quantification. The analysis integrates insights from social system theory and post-normal science to understand the complex interplay between science, policy, and society in the context of quantification. The paper uses a qualitative approach, drawing on case studies and examples to illustrate the points discussed, including the COVID-19 pandemic and the use of predictive models. The authors do not use statistical methods or conduct original empirical research.

The paper identifies several key recurring problems associated with quantification: 1. **Lack of Transparency and Explainability:** Many algorithms and models operate as 'black boxes', making it difficult to understand their decision-making processes and identify biases or errors. This opacity makes it challenging to hold those responsible accountable for the outcomes. 2. **Underestimation of Uncertainty:** The authors argue that many quantitative models fail to adequately account for uncertainty, leading to overconfidence in their predictions and potentially flawed policy decisions. The COVID-19 pandemic serves as a prime example of this issue. 3. **Bias and Discrimination:** Algorithms and metrics can embed and amplify existing social biases, leading to discriminatory outcomes. For example, algorithms used in criminal justice or loan applications can perpetuate racial and socioeconomic disparities. 4. **Goal Displacement:** The use of metrics can lead to a focus on optimizing the metrics themselves, rather than the underlying goals they are meant to measure, potentially resulting in unintended and negative consequences (Goodhart's Law). 5. **Erosion of Human Agency:** The increasing reliance on quantitative models and algorithms for decision-making can diminish the role of human judgment and critical thinking. Political problems may be converted into technical ones, thus avoiding or limiting political debate. 6. **Lock-in and Path Dependence:** Once certain quantification practices are established, they can be very difficult to change, even if they are demonstrably flawed. This creates lock-in effects, hindering progress towards more ethical and effective forms of quantification. The paper proposes several principles for an ethics of quantification: investigating societal relevance, vigilance about framing assumptions, navigating the humility-hubris axis, probing for missing numbers and blind spots, supporting data activism, and fostering quality as fitness for societal purpose.

The authors argue that the fragmented approaches to ethical considerations within specific domains of quantification hinder progress. They contend that an encompassing 'ethics of quantification' is crucial to address the shared problems and develop effective strategies. The COVID-19 pandemic underscores the urgency of this need, as the pandemic's narrative was dominated by numbers (deaths, infections), often presented with an unwarranted precision, while other crucial aspects of the crisis were obscured. The authors use the example of weather forecasting, where models and society have a more mature relationship. They advocate for a move away from 'technologies of hubris' towards 'technologies of humility' promoting greater transparency, awareness of uncertainty, and participation. The authors’ perspective emphasizes the symbiotic relationship between quantification and trust and the need for societal control over the production and use of numbers to prevent the misuse and manipulative use of data.

The paper concludes that an encompassing ethics of quantification is urgently needed to address the widespread challenges arising from the pervasive use of numbers, models, algorithms, and rankings. The authors suggest several policy implications, emphasizing the roles of organized labor, institutions, and the European Commission's Responsible Research and Innovation (RRI) framework. They propose several principles for an ethics of quantification that should guide the development of more responsible and ethical practices in the future. Further research should focus on developing practical tools and frameworks for implementing an ethics of quantification across different domains.

As a perspective article, the paper's main limitation is its reliance on a review of existing literature, without conducting original empirical research. The arguments are largely theoretical and rely on case studies and examples to support the claims. The authors acknowledge the complexity and diversity of quantification practices, and their framework might not capture all nuances of specific applications. The policy recommendations are relatively broad and require further development before practical implementation.